---
license: mit
library_name: stftvae
pipeline_tag: audio-to-audio
tags:
  - audio
  - codec
  - vae
  - stft
  - neural-audio
---

# STFT-VAE

Continuous (VAE) neural audio codec built on the STFT. Audio is transformed to a
complex spectrogram, a transformer encoder downsamples it to a low-rate
continuous latent, and a transformer decoder + ISTFT reconstruct the waveform —
phase is carried by the STFT/ISTFT rather than a learned vocoder.

- **Sample rate:** 24 kHz
- **Latent:** 3.125 Hz frame rate, 128-dim continuous (no quantization)
- **Params:** ~116M
- **Files:** `config.json` (architecture) + `model.safetensors` (fp32 weights)

## Usage

```bash
pip install "stftvae[cli]"
```

```python
import torch
from stftvae import STFTVAE

vae = STFTVAE.from_pretrained("fluxions/stftvae", device="cuda")

audio = torch.randn(1, 24000)          # (1, T) mono @ 24 kHz
recons = vae.reconstruct(audio)        # (1, 1, T)

# or encode / decode separately
latent, length = vae.encode(audio)     # (1, 128, T_latent)
recons = vae.decode(latent, length=length)
```

Command line:

```bash
python -m stftvae input.wav output.wav --model fluxions/stftvae
```

The encoder returns the deterministic posterior mean (no sampling noise at
inference). Code: https://github.com/fluxions-ai/stftvae

## License

MIT