Drama 3 OrbitQuant W2A2

Built with Fish Audio.

This is an OrbitQuant deployment of Fish Audio Drama 3, a multilingual speech model with a hybrid-attention MoE backbone and a 10-codebook audio decoder. All 10,240 routed experts use packed two-bit weights and two-bit activation quantization. Attention, routers, shared experts, embeddings, output heads, the audio decoder, and the codec retain their source weights.

Components

Component	Precision	Stored size
Routed expert gate, up, and down projections	OrbitQuant W2A2, RPBH rotation, Lloyd–Max codebooks	Included in transformer shards
Remaining backbone and audio decoder	BF16 with source FP32 state where required	Included in transformer shards
Transformer checkpoint	Mixed precision above	15.49 GB versus 39.60 GB in the source
Audio codec	Unmodified source checkpoint, BF16 at inference	1.87 GB

The deployment bundle is approximately 17.39 GB. GB denotes decimal file bytes. Memory below uses GiB. Two-bit quantization applies to routed experts. The complete pipeline retains higher-precision components.

Usage

Use Linux, Python 3.12, Git, uv, and a CUDA 12.8-compatible NVIDIA driver. The complete pipeline was tested on one L40S 48 GB. Smaller physical GPUs were not tested.

hf download WaveCut/Drama-3-OrbitQuant-W2A2 --local-dir drama3-orbitquant
cd drama3-orbitquant
python3 install_runtime.py
.venv/bin/python run_tts.py --model . --text "Hello. The train leaves in five minutes. Please bring your ticket." --reference-audio assets/original-english.wav --reference-text "Hello. The train leaves in five minutes. Please bring your ticket." --output english.wav

Russian with one voice reference:

.venv/bin/python run_tts.py --model . --text "Добрый вечер! Поезд отправляется через пять минут. Пожалуйста, приготовьте билет." --reference-audio assets/original-english.wav --reference-text "Hello. The train leaves in five minutes. Please bring your ticket." --output russian.wav

Dialogue with two voice references:

.venv/bin/python run_tts.py --model . --text "<|speaker:0|> We found the key. <|speaker:1|> Great! Now we can finally open the door." --reference-audio assets/original-english.wav --reference-text "Hello. The train leaves in five minutes. Please bring your ticket." --reference-audio assets/reference-speaker-1.wav --reference-text "Hello. The train leaves in five minutes. Please bring your ticket." --output dialogue.wav

References are synthetic recordings; repeated reference arguments map to speakers in order. Voice similarity is not scored. The included loader and OrbitQuant wheel are required; generic Transformers, unmodified Fish Speech and SGLang-Omni loading are unsupported. The default limit is 500 frames. Use --max-new-tokens for longer outputs. Keep the model loaded for repeated requests.

Examples

English and Russian use one matching synthetic voice reference; dialogue uses two. Source execution is eager; W2A2 uses CUDA graphs. Both configurations use seed 42, temperature 1.0, top-p 0.9, top-k 30 and a 500-frame limit.

Text	Original	W2A2
Hello. The train leaves in five minutes. Please bring your ticket.
Добрый вечер! Поезд отправляется через пять минут. Пожалуйста, приготовьте билет.
We found the key. Great! Now we can finally open the door.

Performance

GPU-resident execution on an otherwise idle L40S 48 GB, Torch 2.8.0/CUDA 12.8, CUDA graphs, batch size 1. The workload is the reference-conditioned Russian prompt above. Values are means of three warm requests after first use. Time includes prefill, audio-code generation and WAV decoding; loading, reference encoding and WAV writing are excluded. RTF is generation time / audio duration.

Configuration	VRAM after loading, GiB	Peak generation VRAM, GiB	Mean warm generation, s	Audio duration, s	RTF
Original FP8/BF16	41.23	42.76	3.243	6.13	0.529
OrbitQuant W2A2 / BF16	18.74	20.31	3.032	5.80	0.522

Device memory includes the CUDA context and allocator reservations, sampled every 0.1 s. Output lengths differ across configurations. These measurements do not establish an identical-work speedup or concurrent serving throughput.

Limits

Use voice references for English, Russian and dialogue; unconditioned English can produce excess silence. Whisper large-v3-turbo WER is 0% for the selected English and dialogue examples, and 10% for Russian because ASR writes 5 instead of пять. ASR does not measure fluency or speaker similarity. These examples do not establish broad quality equivalence or long-form reliability. Smaller physical GPUs are untested.

Model offload

Add --cpu-offload to the commands above. This mode stores backbone weights in pinned CPU RAM and stages one layer at a time on GPU. The audio tower and attention state remain on GPU. The codec is staged for reference encoding and WAV decoding. CUDA graphs are disabled.

CPU: EPYC 9354, PCIe 4.0 ×16, eight threads. GPU and runtime match the table above. All rows use the same Russian prompt, voice reference, seed 42 and offload policy. BF16 expands the source FP8 expert weights; it cannot recover their original pre-quantization values.

Configuration	VRAM loaded / peak, GiB	Host RSS loaded / peak, GiB	Warm time [range], s	Audio, s	RTF
BF16 reference	2.84 / 7.88	71.11 / 71.30	410.07 (408.53–411.61)	6.69	61.32
W4A4	2.75 / 7.33	26.05 / 26.24	112.17 (112.16–112.18)	6.04	18.58
W2A2	2.73 / 7.22	18.47 / 18.67	73.73 (73.31–74.15)	6.08	12.12

Two warm requests per configuration after first use. Time includes CPU-to-GPU transfers; other timing boundaries match the GPU-resident table. Driver and RSS peaks are sampled every 0.2 s. Output lengths differ.

One separate synchronized profile per configuration measures active PyTorch tensor peaks:

Configuration	Reference encoding, GiB	Audio-code generation, GiB	WAV decoding, GiB
BF16 reference	6.30	3.99	6.57
W4A4	6.30	3.34	6.54
W2A2	6.30	3.34	6.54

WAV decoding determines the total peak. The codec holds 3.85 GiB of tensors, including three 32768 × 32768 boolean masks totalling 3 GiB. The codec and GPU-resident tower use the same precision in both releases. The BF16 token embedding limits their audio-code generation peak. After W2A2 decoding, active tensors occupy 2.38 GiB; PyTorch retains a 6.68 GiB allocator pool. Reserved memory already includes active memory.

Russian with the same voice reference	Audio
Source FP8 expanded to BF16, offload
W2A2, offload

Files and license

model-layer-*.safetensors stores backbone layers. model-components.safetensors stores the remaining transformer components. The index maps every tensor to its shard. The codec, tokenizer, deployment loader, and included OrbitQuant wheel complete the runtime. The manifest records source provenance, component precision, rotation settings, and runtime hashes.

The model and derivative weights use the Fish Audio Community License, Version 1.0. It permits research and non-commercial use and limited commercial use below its revenue threshold. Hosted services have separate terms and exceptions. Read the license before commercial use or serving third parties. Preserve the attribution in Notice. This is a WaveCut derivative release, without Fish Audio endorsement.