Category: Audio Formats

Every digital audio workstation, streaming platform, mastering converter, and optical disc format relies upon the mathematical foundations established by Harry Nyquist, Claude Shannon, and Vladimir Kotelnikov in the early twentieth century. Linear Pulse Code Modulation (LPCM) transforms a continuous-time, infinitely variable analog acoustic voltage $v(t)$ into a discrete sequence of binary numerical words by performing two orthogonal mathematical operations: uniform time discretization (Sampling) along the horizontal axis and amplitude discretization (Quantization) along the vertical axis.

Despite decades of widespread deployment, profound misconceptions persist regarding how digital sampling reconstructs continuous waveforms—particularly the pervasive myth that digital audio outputs a “stair-step” approximation of the original analog wave. This comprehensive engineering reference dissects the Nyquist-Shannon Sampling Theorem, Dirac delta impulse trains, Whittaker-Shannon sinc interpolation, aliasing and anti-aliasing filter topologies, bit-depth signal-to-quantization-noise ratio (SQNR), 32-bit IEEE 754 floating-point headroom, and Triangular Probability Density Function (TPDF) dithering with psychoacoustic noise shaping.

The Nyquist-Shannon Sampling Theorem states a remarkable, mathematically exact truth: if a continuous-time analog signal $x(t)$ is strictly bandlimited such that it contains no spectral frequency components at or above a maximum frequency $B$ Hertz, then $x(t)$ is completely and uniquely determined—with zero loss of information or inter-sample detail—by capturing discrete instantaneous amplitude samples spaced at uniform intervals of $T_s = \frac{1}{f_s}$ seconds, provided the sampling frequency $f_s$ strictly exceeds twice the maximum signal bandwidth:

1. Fundamental Nyquist Sampling Inequality:
   f_s > 2 * B      (or Nyquist Limit: f_Nyquist = f_s / 2 > B)

2. Ideal Impulse Sampling (Multiplication by Dirac Comb):
   x_s(t) = x(t) * sum_{n=-inf}^{+inf} delta(t - n * T_s)

3. Exact Continuous-Time Reconstruction (Whittaker-Shannon Formula):
   x(t) = sum_{n=-inf}^{+inf} x[n] * sinc( (t - n * T_s) / T_s )
   where normalized sinc(u) = sin(pi * u) / (pi * u)

Why is the “stair-step” mental model of digital audio completely false? A “stair-step” waveform (known in signal processing as a Zero-Order Hold) contains sharp 90-degree vertical edges at every sample transition. By Fourier analysis, sharp rectangular edges consist of high-frequency ultrasonic harmonics extending far above the Nyquist limit ($f_s / 2$). When the discrete samples pass through the Digital-to-Analog Converter’s (DAC) low-pass reconstruction filter, every frequency above $f_s / 2$ is stripped away. Mathematically, passing discrete impulses through an ideal brick-wall low-pass filter in the frequency domain is strictly equivalent to convolving each sample with a continuous $\text{sinc}(t) = \frac{\sin(\pi f_s t)}{\pi f_s t}$ impulse response in the time domain.

Because only one unique bandlimited continuous curve can pass simultaneously through a given set of sample points without introducing frequencies above $f_s / 2$, the reconstructed analog output of a properly filtered DAC is an exact, smooth, continuous replica of the original bandlimited input waveform—even for high-frequency sine waves represented by just $2.2$ samples per cycle.

What happens if an analog signal containing ultrasonic energy above the Nyquist frequency ($f_{in} > f_s / 2$) enters an Analog-to-Digital Converter unattenuated? Because sampling in the time domain creates periodic mirror images of the baseband spectrum centered around integer multiples of the sampling frequency ($k \cdot f_s$) in the frequency domain, any input frequency $f_{in}$ exceeding $f_s / 2$ folds back downward into the audible passband as an inharmonic, unremovable distortion tone at the difference frequency:

$$f_{alias} = |f_s – f_{in}|$$

For example, if an ADC operating at $f_s = 44.1\text{ kHz}$ (Nyquist limit $= 22.05\text{ kHz}$) is fed an ultrasonic synthesizer harmonic or cymbal overtone at $25\text{ kHz}$, that ultrasonic tone folds back into the audible recording at $|44.1\text{ kHz} – 25\text{ kHz}| = 19.1\text{ kHz}$. Worse still, once an alias frequency folds into the audible spectrum, it becomes indistinguishable from legitimate musical content and can never be filtered out downstream.

If human hearing extends up to $20,000\text{ Hz}$, why couldn’t early digital engineers simply sample at $f_s = 40,000\text{ Hz}$? An analog low-pass Anti-Aliasing Filter placed before an ADC cannot instantly drop from $0\text{ dB}$ attenuation at $20,000\text{ Hz}$ to $-100\text{ dB}$ attenuation at $20,001\text{ Hz}$. Real-world filters require a finite Transition Band ($f_{stop} – f_{pass}$) over which their response rolls off smoothly without inducing severe phase distortion and group delay ringing in the audible treble.

• 44.1 kHz (Red Book CD Standard): Selected by Sony and Philips in 1979 to maintain compatibility with both 525-line NTSC ($29.97\text{ fps}$) and 625-line PAL ($25\text{ fps}$) helical-scan video tape recorders used to master early digital audio ($3 \text{ samples/line} \times 245 \text{ active lines} \times 60 \text{ fields/s} = 44,100\text{ Hz}$). It provides a $2,050\text{ Hz}$ transition band between $20\text{ kHz}$ and the $22.05\text{ kHz}$ Nyquist limit.
• 48 kHz (AES5 Professional Broadcast & Film Standard): Chosen because $48,000$ divides cleanly into integer sample counts per frame for $24\text{ fps}$ cinema film ($2,000\text{ samples/frame}$), $25\text{ fps}$ PAL ($1,920\text{ samples/frame}$), and $30\text{ fps}$ video ($1,600\text{ samples/frame}$), while offering a wider, gentler $4,000\text{ Hz}$ filter transition band ($20\text{ kHz}$ to $24\text{ kHz}$).
• 96 kHz and 192 kHz (High-Resolution Production Rates): Modern converters sample internally at megahertz rates (oversampling) and decimate in DSP. Operating DAW sessions at $96\text{ kHz}$ pushes the Nyquist limit to $48\text{ kHz}$—which drastically reduces aliasing inside non-linear digital audio plugins (such as tape saturators, guitar amp simulators, and aggressive compressors) that generate harmonic overtones inside the DAW, while cutting round-trip ADC/DAC monitoring buffer latency in half for live tracking musicians.

While sampling rate ($f_s$) exclusively governs frequency bandwidth and time resolution, Bit Depth ($N$) exclusively governs the vertical amplitude resolution—specifically, the ratio between the maximum undistorted full-scale signal ($0\text{ dBFS}$) and the residual quantization noise floor.

In uniform linear PCM, an $N$-bit binary word encodes $L = 2^N$ discrete quantization steps spanning from $-V_{max}$ to $+V_{max}$, with each Least Significant Bit (LSB) step having height $\Delta = \frac{2 V_{max}}{2^N}$. When a continuous analog voltage falls between two adjacent quantization steps, it must be rounded to the nearest integer step, introducing an instantaneous rounding error $e(t)$ bounded between $-\frac{\Delta}{2}$ and $+\frac{\Delta}{2}$. Assuming the input signal is sufficiently complex and active across many steps, the quantization error behaves as an uncorrelated uniform random variable with Root-Mean-Square (RMS) error voltage $V_{noise,RMS} = \frac{\Delta}{\sqrt{12}}$:

1. Quantization Step Size (Delta):
   Delta = (2 * V_peak) / (2^N)

2. RMS Quantization Error Voltage (Uniform Distribution across [-Delta/2, +Delta/2]):
   V_error_RMS = sqrt( (1 / Delta) * integral_{-Delta/2}^{+Delta/2} (e^2) de ) = Delta / sqrt(12)

3. RMS Voltage of a Full-Scale Sine Wave (0 dBFS):
   V_signal_RMS = V_peak / sqrt(2)

4. Signal-to-Quantization-Noise Ratio (SQNR in dB):
   SQNR (dB) = 20 * log10( V_signal_RMS / V_error_RMS )
             = 20 * log10( 2^N * sqrt(6 / 4) )
             = 6.0206 * N + 1.761 dB

In a traditional 16-bit or 24-bit fixed-point integer system, the binary number represents an absolute fraction of full scale. Once all bits are set to 1, the number reaches $0\text{ dBFS}$; adding even $0.1\text{ dB}$ of additional gain wraps or flat-tops the waveform into hard digital clipping. In contrast, 32-bit IEEE 754 Floating-Point PCM divides each 32-bit word into three components: 1 Sign Bit, an 8-Bit Exponent (which shifts the binary decimal point dynamically over a range of $2^{-126}$ to $2^{+127}$), and a 23-Bit Mantissa (plus 1 implied leading bit = 24-bit precision).

Because the 8-bit exponent scales the signal across $254$ octaves of amplitude, $0\text{ dBFS}$ in 32-bit float is merely an arbitrary reference value ($\pm 1.0$). If a mixing engineer accidentally pushes a DAW channel fader $+50\text{ dB}$ above $0\text{ dBFS}$, the floating-point exponent simply increments to represent a peak value of $\pm 316.2$, preserving the full 24-bit mantissa resolution with zero waveform clipping! As long as the master bus fader or final export gain brings the peak level back below $0\text{ dBFS}$ before hitting the physical 24-bit fixed-point DAC output, the audio remains $100\%$ undistorted.

Earlier, we noted that quantization error behaves like benign white noise only when the input signal is large and swings rapidly across many quantization steps. What happens when a reverb tail fades down to $-85\text{ dBFS}$ in a 16-bit file, spanning only the lowest two or three LSB steps? At low signal levels, the rounding error is no longer random—it becomes deterministically correlated with the input signal waveform! A smooth sine wave oscillating across a single quantization threshold gets truncated into a harsh, buzzing square wave (Quantization Distortion), and once the signal drops below $\pm 0.5\text{ LSB}$, the output freezes at zero (Digital Gate-Off).

Why do mastering engineers specifically select TPDF Dither rather than simple Rectangular (RPDF) white noise? If you inject a single uniform random number spanning $[-0.5\text{ LSB}, +0.5\text{ LSB}]$ (RPDF dither), the mean quantization distortion is eliminated, but the variance (loudness) of the noise floor still fluctuates up and down slightly with the instantaneous amplitude of the audio signal—a flaw known as Noise Modulation. By summing two independent uniform random noise generators together ($[-0.5, +0.5] + [-0.5, +0.5]$), the resulting probability distribution forms a triangle spanning $[-1.0\text{ LSB}, +1.0\text{ LSB}]$. Mathematically, TPDF dither renders both the first moment (mean error) and the second moment (noise variance) completely independent of the input signal, totally eliminating both harmonic quantization distortion and noise floor modulation at a cost of just $4.77\text{ dB}$ in broadband RMS noise floor ($98.1\text{ dB} \rightarrow 93.3\text{ dB}$ in 16-bit).

To recover and exceed that $4.77\text{ dB}$ noise penalty, mastering dithering algorithms wrap a high-order error-feedback loop around the quantizer. By subtracting the previous sample’s rounding error $e[n-1]$ from the next sample before quantization, Psychoacoustic Noise Shaping acts as a high-pass filter on the quantization noise floor—scooping noise energy out of the $1\text{ kHz – }5\text{ kHz}$ ear-canal sensitivity zone (where human hearing is most acute per the Fletcher-Munson equal-loudness contours) and pushing that noise energy up into the $15\text{ kHz – }22\text{ kHz}$ region where human hearing is relatively insensitive. As a result, a 16-bit CD mastered with 9th-order psychoacoustic noise-shaped dither achieves an audible dynamic range equivalent to 19-to-20-bit linear PCM across the critical midrange.

A critical pitfall in modern digital mastering is assuming that because no individual discrete sample in a PCM file exceeds $0.0\text{ dBFS}$, the reconstructed analog waveform will never exceed $0.0\text{ dBFS}$. Consider four consecutive samples of a high-frequency wave near Nyquist: $[-0.707, +0.990, +0.990, -0.707]$. Neither peak sample exceeds $1.000$ ($0\text{ dBFS}$). However, when the DAC’s sinc interpolation filter reconstructs the smooth continuous curve arcing between those two $+0.990$ samples, the apex of the continuous curve shoots up to $+1.21\text{ (+1.65 dBFS)}$!

These overshoots are called Inter-Sample Peaks (ISPs). If a mastered track is brick-wall limited to $-0.1\text{ dBFS}$ sample peak, its continuous reconstructed waveform frequently hits $+1.5\text{ to }+3.0\text{ dBTP}$ (Decibels True Peak), overloading the internal oversampling filters of consumer DACs and causing severe audible crunch in lossy AAC/MP3 encoders. To prevent inter-sample clipping, ITU-R BS.1770 True Peak limiters oversample the audio stream by $4\times$ or $8\times$ ($192\text{ kHz}$ or $384\text{ kHz}$) inside the limiter detection sidechain, allowing engineers to cap the true continuous ceiling at $-1.0\text{ dBTP}$.