scarxity/fish-speech-s2-pro-nf4

Fish Speech S2-Pro — NF4 (bitsandbytes 4-bit)

A bitsandbytes NF4 4-bit quantization of fishaudio/s2-pro, the 4B-parameter flagship Fish Audio S2 text-to-speech model.

Roughly 4.9 GB on disk against 8.5 GB of bf16 safetensors, and it loads directly as 4-bit — no quantization step at startup.

Everything the base model does is preserved: voice cloning from a short reference clip, ~50 languages, and inline expressiveness tags such as [laugh], [whispers], [sighs].

Intended use

This checkpoint is built for scarxity/fish-speech-int4-patch, a fork whose server-4gb / webui-4gb Docker profiles run S2-Pro on a 4 GB laptop GPU (measured 3294 MiB peak on a simulated 4096 MiB budget, faster than realtime on an RTX 4060 Ti).

The 4 GB figure comes from the fork's serving code, not from these weights alone. The quantization is what fits ~4.9 GB on disk; getting into 4 GB of VRAM additionally needs the fork's decode-only codec, host-resident embedding table, sliced lm_head, and chunked codec decode. Stock upstream code with this checkpoint will need more VRAM than that.

git clone https://github.com/scarxity/fish-speech-int4-patch
cd fish-speech-int4-patch

huggingface-cli download scarxity/fish-speech-s2-pro-nf4 --local-dir checkpoints/s2-pro-nf4

docker compose --profile server-4gb build
docker compose --profile server-4gb up -d

Then POST to http://localhost:8880/v1/tts:

{ "text": "Hello, world.", "reference_id": "my-voice" }

An OpenAI-compatible /v1/audio/speech endpoint and a Gradio WebUI (port 7860) are included. See the fork's 4 GB guide for reference-voice setup and long-text handling.

Requirements

• NVIDIA GPU (bitsandbytes NF4 is CUDA-only)
• bitsandbytes >= 0.43
• Loading flags --bnb4 --half

This is NF4, not GPTQ, AWQ, or int4 — those formats are not interchangeable, and loaders for them will not read this checkpoint.

Quantization details

Scheme	bitsandbytes NF4 (normal-float 4-bit)
Double quantization	enabled (compress_statistics=True)
Compute dtype	fp16
Quantized	the 36-layer slow backbone and the 4-layer depth transformer
Effective	~4.13 bits/weight

Architecture

S2-Pro is a Dual-AR model over an RVQ audio codec (10 codebooks, 21.53 Hz):

• Slow AR (36 layers, dim 2560) predicts the semantic codebook along time — words, pronunciation, language, and the inline emotion tags.
• Fast AR (4 layers, dim 2560) runs nine times per frame for the residual codebooks — timbre and fine acoustic texture.
• Codec upsamples the codes ×2048 to a 44.1 kHz waveform.

Notes

• tokenizer_config.json in this repo declares PreTrainedTokenizerFast. Some S2-Pro copies ship TokenizersBackend, a transformers v5 name that older loaders reject with UnboundLocalError: tokenizer.
• Voice cloning is in-context: the reference clip's codes are prepended to every request rather than fine-tuned in. Keep reference clips near 10 seconds — a 31.7 s clip costs 683 prompt tokens on every request against 220 for a 10.2 s one.

License

Released under the Fish Audio Research License, inherited from the base model. Review it before any commercial or redistributive use.

Credits

• Fish Audio — the S2 model, all core research and architecture
• groxaxo/fish-speech-int4-patch — the original NF4 quantization path
• Technical report