Drama 3 OrbitQuant W4A4

Built with Fish Audio.

This is an OrbitQuant deployment of Fish Audio Drama 3, a multilingual speech model with a hybrid-attention MoE backbone and a 10-codebook audio decoder. All 10,240 routed experts use packed four-bit weights and 4-bit activations. Attention, routers, shared experts, embeddings, output heads, the audio decoder, and the codec retain their source weights.

Components

Component	Precision	Stored size
Routed expert gate, up, and down projections	OrbitQuant W4A4, RPBH rotation, Lloyd–Max codebooks	Included in transformer shards
Remaining backbone and audio decoder	BF16 with source FP32 state where required	Included in transformer shards
Transformer checkpoint	Mixed precision above	23.54 GB, versus 39.60 GB in the source
Audio codec	Unmodified source checkpoint, BF16 at inference	1.87 GB

The deployment bundle is approximately 25.44 GB. GB denotes decimal file bytes. Memory measurements below use GiB.

Usage

Use Linux, Python 3.12, a CUDA 12.8-compatible NVIDIA driver, Git, and uv. The complete pipeline was tested on one NVIDIA L40S 48 GB. Smaller physical GPUs were not tested.

hf download WaveCut/Drama-3-OrbitQuant-W4A4 --local-dir drama3-orbitquant
cd drama3-orbitquant
python3 install_runtime.py
.venv/bin/python run_tts.py --model . --text "Hello. The train leaves in five minutes. Please bring your ticket." --output english.wav

Russian speech:

.venv/bin/python run_tts.py --model . --text "Добрый вечер! Поезд отправляется через пять минут. Пожалуйста, приготовьте билет." --reference-audio assets/original-english.wav --reference-text "Hello. The train leaves in five minutes. Please bring your ticket." --output russian.wav

Two speaker tags:

.venv/bin/python run_tts.py --model . --text "<|speaker:0|> We found the key. <|speaker:1|> Great! Now we can finally open the door." --reference-audio assets/original-english.wav --reference-text "Hello. The train leaves in five minutes. Please bring your ticket." --reference-audio assets/orbitquant-english.wav --reference-text "Hello. The train leaves in five minutes. Please bring your ticket." --eager --output dialogue.wav

References are synthetic recordings; repeated reference arguments map to speakers in order. Voice similarity is not scored. The included loader and OrbitQuant wheel are required; generic Transformers, unmodified Fish Speech and SGLang-Omni loading are unsupported. The default limit is 500 frames. Use --max-new-tokens for longer outputs. Keep the model loaded for repeated requests.

Examples

English is unconditioned; Russian and dialogue use matching synthetic voice references. Both configurations use seed 42, temperature 1.0, top-p 0.9, top-k 30 and a 500-frame limit.

Text	Original	OrbitQuant
Hello. The train leaves in five minutes. Please bring your ticket.
Добрый вечер! Поезд отправляется через пять минут. Пожалуйста, приготовьте билет.
Two speaker tags: We found the key. Great! Now we can finally open the door.

Performance

GPU-resident execution on an otherwise idle L40S 48 GB, Torch 2.8.0/CUDA 12.8, CUDA graphs, batch size 1. The workload is the reference-conditioned Russian prompt above. Values are means of three warm requests after first use. Time includes prefill, audio-code generation and WAV decoding; loading, reference encoding and WAV writing are excluded. RTF is generation time / audio duration.

Configuration	VRAM after loading	Peak generation VRAM	Mean warm generation	Audio duration	RTF
Original FP8/BF16	41.23 GiB	42.76 GiB	3.243 s	6.13 s	0.529
OrbitQuant W4A4 / BF16	26.69 GiB	27.83 GiB	3.112 s	6.08 s	0.512

Device memory includes the CUDA context and allocator reservations, sampled every 0.1 s. Output lengths differ across configurations. These measurements do not establish an identical-work speedup or concurrent serving throughput.

Limits

Use voice references for Russian speech and dialogue. For W4A4 dialogue, use --eager; the CUDA-graph path can reach the frame limit. Whisper large-v3-turbo WER is 0% for the selected English and dialogue examples, and 10% for Russian because ASR writes 5 instead of пять. ASR does not measure fluency or speaker similarity. These examples do not establish broad quality equivalence or long-form reliability. Smaller physical GPUs are untested.

Model offload

Add --cpu-offload to the commands above. This mode stores backbone weights in pinned CPU RAM and stages one layer at a time on GPU. The audio tower and attention state remain on GPU. The codec is staged for reference encoding and WAV decoding. CUDA graphs are disabled.

CPU: EPYC 9354, PCIe 4.0 ×16, eight threads. GPU and runtime match the table above. All rows use the same Russian prompt, voice reference, seed 42 and offload policy. BF16 expands the source FP8 expert weights; it cannot recover their original pre-quantization values.

Configuration	VRAM loaded / peak, GiB	Host RSS loaded / peak, GiB	Warm time [range], s	Audio, s	RTF
BF16 reference	2.84 / 7.88	71.11 / 71.30	410.07 (408.53–411.61)	6.69	61.32
W4A4	2.75 / 7.33	26.05 / 26.24	112.17 (112.16–112.18)	6.04	18.58
W2A2	2.73 / 7.22	18.47 / 18.67	73.73 (73.31–74.15)	6.08	12.12

Two warm requests per configuration after first use. Time includes CPU-to-GPU transfers; other timing boundaries match the GPU-resident table. Driver and RSS peaks are sampled every 0.2 s. Output lengths differ.

One separate synchronized profile per configuration measures active PyTorch tensor peaks:

Configuration	Reference encoding, GiB	Audio-code generation, GiB	WAV decoding, GiB
BF16 reference	6.30	3.99	6.57
W4A4	6.30	3.34	6.54
W2A2	6.30	3.34	6.54

WAV decoding determines the total peak. The codec holds 3.85 GiB of tensors, including three 32768 × 32768 boolean masks totalling 3 GiB. The codec and GPU-resident tower use the same precision in both releases. The BF16 token embedding limits their audio-code generation peak. After W2A2 decoding, active tensors occupy 2.38 GiB; PyTorch retains a 6.68 GiB allocator pool. Reserved memory already includes active memory.

Russian with the same voice reference	Audio
Source FP8 expanded to BF16, offload
W4A4, offload

Files and license

model-layer-*.safetensors stores the backbone layers. model-components.safetensors stores the remaining transformer components. The index maps every tensor to its shard. codec.pth, the tokenizer files, the deployment loader, and the included OrbitQuant wheel complete the runtime. The machine-readable manifest records component precision, source provenance, rotation settings, and runtime hashes.

The model and its derivative weights use the Fish Audio Community License, Version 1.0. The license permits research and non-commercial use and limited commercial use below its revenue threshold. Hosted services have separate terms and exceptions. Read the license before commercial use or serving third parties. Preserve the included attribution in Notice. This is a WaveCut derivative release. Fish Audio did not endorse or officially release it.