SNAC 24 kHz β Hindi
A SNAC 24 kHz multi-scale neural audio codec wh ose decoder has been fine-tuned on Hindi / Hinglish speech. It serves as the vocoder (SNAC-token β waveform detokenizer) for a Orpheus styled TTS stack.
Only the decoder was fine-tuned β the encoder and quantizer are frozen from the base
hubertsiuzdak/snac_24khz. The code
space is therefore identical to the base model, so this checkpoint is a drop-in decoder
replacement: any SNAC code24khz` decode here with no retraining and no re-tokenizing.
Model details
| Base | `hubertsiuzdak/snac |
| Output | 24 kHz, mono |
| Codebooks | 3 hierarchical levels, 4096 entries each |
| Frame layout | 7 tokens / ine) |
| Parameters | ~19.8 M |
| Fine-tuned | decoder only |
Installation
pip install snac torch sound
Usage
1. Round-trip: audio β c
import torch, soundfile as s
from snac import SNAC
device = "cuda" if torch.cud
model = SNAC.from_pretrained("nullHawk/snac-24khz-hindi-hp").eval().to(device)
wav, sr = sf.read("input.wavt be 24 kHz (resample first if
not)
audio = torch.from_numpy(wavce) # shape [1, 1, T]
with torch.inference_mode():
codes = model.encode(aud LongTensors (coarse -> fine)
recon = model.decode(codes) # [1, 1, T'] waveform
sf.write("recon.wav", recon[
2. Decode from a flat SN
TTS LLMs (Orpheus SNAC tokens, 7 per frame, interleaved across the 3 lev:
import torch
def decode_snac_tokens(ids,
"""ids : flat list of
offset: subtract your LLM's SNAC base id (use 0 if ids are already in [0, 4096)).
"""
ids = [(t - offset) % 40
frames = len(ids) // 7
l1, l2, l3 = [], [], []
for i in range(frames):
s = ids[i * 7:(i + 1) * 7]
l1.append(s[0])
l2 += [s[1], s[4]]
l3 += [s[2], s[3], s
dev = next(model.parameters()).device
codes = [torch.tensor(x,)[None] for x in (l1, l2, l3)]
with torch.inference_mode():
z_q = model.quantize
audio = model.decoder(z_q)[0, 0].cpu().numpy()
return audio[2048:] ms decoder warm-up
# offset = the SNAC base tokg. 128266 for maya1 / Orpheus).
wav = decode_snac_tokens(token_ids, model, offset=128266)
import soundfile as sf
sf.write("tts.wav", wav, 24000)
Notes
- Only the decoder changed, tsiuzdak/snac_24khz` by swappin g the repo id β the encoded code
- Output is 24 kHz mono. Trim the first ~2048 samples of a freshly decoded clip to remov e the decoder warm-up transient.
Acknowledgements
Built on SNAC by Hubert Siuzdak.
- Downloads last month
- 17
Model tree for nullHawk/snac-24khz-hindi-ft
Base model
hubertsiuzdak/snac_24khz