A drop-in, differentiable spectrogram layer for PyTorch that decodes back to the waveform exactly, with no trained weights. Linear, interpretable, CPU-fast. Any scale: linear, mel, Bark, ERB.
iSincNet turns a waveform into an interpretable 2-D spectrogram (a deterministic SincNet filterbank) and back into the waveform. The decoder is a closed-form inverse of the encoder: no vocoder, no training, no checkpoints. Pick any frequency scale (linear, mel, Bark, ERB); reconstruction is length-exact and, for a well-conditioned bank, near-exact (about 100+ dB, up to roughly 125 dB).
Filterbank kernels and example spectrograms: docs/filters.md. For a controlled comparison and an explanation of why warped banks need adaptive bandwidth, see why the sinc envelope matters.
pip install . # core (torch, numpy, librosa)
pip install .[demo] # adds matplotlib + soundfile for the demo notebookimport torch
from sincnet.model import SincNet
# nothing to download: the decoder is the analytical inverse of the encoder
model = SincNet(
fs=16_000, fps=128, n_bins=256,
scale="mel", # any scale: "lin" | "mel" | "bark" | "erb"
component="complex", causal=False, decoder_type="exact",
).eval()
wav = torch.randn(1, 16_000) # (B, T)
with torch.no_grad():
spec = model.encode(wav) # (B, 2, F, T) signed real/imag spectrogram
recon = model.decode(spec, length=wav.shape[-1]) # (B, T) exact, training-free, length-preserving
# optional mu-law quantization for a compact representation:
q, scale = model.mulaw.quantize(spec)
recon = model.decode(model.mulaw.dequantize(q, scale), length=wav.shape[-1])See the demo notebook.
All share the same encode / decode API and are length-exact (pick via decoder_type):
decoder_type |
reconstruction | weights | differentiable | remark |
|---|---|---|---|---|
fast (default) |
~37 dB, single-pass conv_transpose + equalizer |
none | yes | fast path; fine for training losses |
exact |
~100+ dB, conjugate-gradient pseudo-inverse | none | yes | implicit backward, O(1) memory in iterations; for end-to-end training |
Whether decode(encode(x)) is exact is fixed, before any training, by n_bins, fps, and the
kernel length. The bank is per-frame invertible (a clean closed-form inverse exists) once:
2 * n_bins >= kernel_size = coverage * fs / fps <=> n_bins * fps >= (coverage / 2) * fs
At fps = 128 that means n_bins >= 256. SincNet warns at construction when a config sits below
the line. Comparing STFT(x) with STFT(decode(encode(x))):
128 bins (below the line): visible residual.
512 bins (well above the line): indistinguishable.
Full derivation and more figures: docs/invertibility_constraint.md.
- [1] Ravanelli & Bengio, Speaker Recognition from raw waveform with SincNet arXiv
- [2] Filterbank design for end-to-end speech separation arXiv, which decomposes SincNet into a
cos * sinproduct (the formulation used here) - [3] Toward end-to-end interpretable convolutional neural networks for waveform signals arXiv
- [4] PF-Net: Personalized Filter for Speaker Recognition from Raw Waveform arXiv
Related discussion, SincNet vs STFT: mravanelli/SincNet#74