metadata
license: mit
library_name: stftvae
pipeline_tag: audio-to-audio
tags:
- audio
- codec
- vae
- stft
- neural-audio
STFT-VAE
Continuous (VAE) neural audio codec built on the STFT. Audio is transformed to a complex spectrogram, a transformer encoder downsamples it to a low-rate continuous latent, and a transformer decoder + ISTFT reconstruct the waveform — phase is carried by the STFT/ISTFT rather than a learned vocoder.
- Sample rate: 24 kHz
- Latent: 3.125 Hz frame rate, 128-dim continuous (no quantization)
- Params: ~116M
- Files:
config.json(architecture) +model.safetensors(fp32 weights)
Usage
pip install "stftvae[cli]"
import torch
from stftvae import STFTVAE
vae = STFTVAE.from_pretrained("fluxions/stftvae", device="cuda")
audio = torch.randn(1, 24000) # (1, T) mono @ 24 kHz
recons = vae.reconstruct(audio) # (1, 1, T)
# or encode / decode separately
latent, length = vae.encode(audio) # (1, 128, T_latent)
recons = vae.decode(latent, length=length)
Command line:
python -m stftvae input.wav output.wav --model fluxions/stftvae
The encoder returns the deterministic posterior mean (no sampling noise at inference). Code: https://github.com/fluxions-ai/stftvae
License
MIT