How audio is represented on computers

1. From pressure wave to discrete-time signal

Pipeline:

sound in air → microphone → analog voltage $x_{\text{analog}}(t)$ → anti-alias filter → sampler → ADC → samples $x[n]$

• Microphone: produces a voltage proportional (roughly) to air pressure.

• Anti-alias filter: low-pass with cutoff just below $f_s/2$​​.

• Sampler: takes values at $t=n{T}_s$, $T_s=1/f_s$​.

You get a discrete-time, continuous-valued signal:

$x[n]=x_{\text{analog}}(n{T}_s),n\in \mathbb{Z}$

At this point, still real-valued.

---

2. Quantization: continuous amplitude → discrete levels

ADC then maps each real $x[n]$ to one of $2^B$ levels, where $B$ = bit depth.

For uniform PCM quantization over a symmetric range $[-X_{\max },X_{\max }]$:

• Step size: $\Delta =\frac{2X_{\max }}{2^B}$​​.
• Quantizer index (conceptually):
  • $k=\text{round}\left(\frac{x[n]}{\Delta }\right)$
• Stored integer: something like $k\in \{-2^{B-1},\dots ,2^{B-1}-1\}$

Classic rule of thumb for a full-scale sine in an ideal uniform quantizer:

$\text{SNR}\approx 6.02B+1.76\text{dB}$ For a full explanation and derivation, see Quantization, SNR, Bits, and Sample Rate

So:

• 16-bit ≈ 98 dB dynamic range.
• 8-bit ≈ 50 dB → this is where companding (like μ-law) starts to matter.

In memory, this might be:

• int16
• int24 packed into 3 bytes
• float32 typically used in ML frameworks → values normalized to [−1,1][-1,1][−1,1]

Most “raw” audio APIs & WAV files are linear PCM integers; ML code often converts to float.

---

3. How it’s actually laid out in memory / files

At the lowest level, audio is just contiguous samples, e.g.:

• Mono, 16-bit PCM at 44.1 kHz → array of int16: $x[0],x[1],x[2],\dots$

• Stereo → interleaved by default:

  $L[0],R[0],L[1],R[1],L[2],R[2],\dots$

A typical WAV file =

1. Header (metadata)

   • sample rate (e.g. 44100)
   • bit depth (e.g. 16)
   • number of channels (1, 2, …)
   • encoding (e.g. PCM, µ-law, etc.)

2. Data chunk = raw PCM samples in the chosen format.

So everything you care about at the DSP/ML level is basically:

• sample rate
• bit depth / sample format (int or float, μ-law or linear)
• number of channels

4. Typical ML representation

When you load audio with something like torchaudio, librosa, etc., you usually get

• A tensor of shape
  • (channels, time) or (time,) for mono
• dtype:
  • float32 in $[-1,1]$ (most common), or
  • int16 if you ask for raw PCM.

Then you might transform it further:

• STFT → complex spectrogram
• |·| → magnitude spectrogram
• mel filterbank → mel-spectrogram
• $\log$ → log-mel spectrogram