Fish Audio · Quantized for ComfyUI
Drama 3.
Expressive speech. Compact expert weights.
Expert-only Q2_K and Q3_K quantizations of Fish Audio Drama 3, with custom ComfyUI nodes for packed expert RAM offload and CUDA inference.
Drama 3 is a 35B mixture-of-experts text-to-speech model for voice cloning, multi-speaker dialogue, and expressive speech. These builds reduce its routed-expert storage while retaining the original precision of the remaining model components.
Built with Fish Audio. This is an independent quantization and ComfyUI integration, based on PedroMarinhoDev/ComfyUI-Drama3.
Samples
| Prompt | Quantization | Audio Sample |
|---|---|---|
| I thought we were alone, but... [whisper] there's someone on the other side of that door. [angry] If this is another one of your little jokes, I swear— [pause] wait, is that music? [excited] You remembered my birthday! [laughs] I spent the whole evening thinking you'd forgotten. | fp8 | |
| q3_K | ||
| q2_K | ||
| You sold our spaceship for a sandwich? [angry] That was our only way off this planet! [pause] Wait... is that toasted sourdough? [whisper] Does it have the little pickles? [sighs] Move over. [laughs] We're splitting it. | fp8 | |
| q3_K | ||
| q2_K | ||
| I followed your recipe exactly, and the kitchen is still standing. [angry] You said two tablespoons of baking powder, didn't you? Two teaspoons? [sighs] Well, that explains the ceiling. | fp8 | |
| q3_K | ||
| q2_K |
Choose a quant
Smaller footprint
Q2_K
16.93 GB model download
10.57 GB packed expert banks in system RAM.
2.625 bits per expert weight, including scales.
Higher expert precision
Q3_K
20.20 GB model download
13.84 GB packed expert banks in system RAM.
3.4375 bits per expert weight, including scales.
| Model | Download size | Packed expert banks in system RAM | Expert bits/weight, including scales |
|---|---|---|---|
| drama3_q2_k_experts_bf16.safetensors | 16.93 GB | 10.57 GB | 2.625 |
| drama3_q3_k_experts_bf16.safetensors | 20.20 GB | 13.84 GB | 3.4375 |
Sizes above use decimal GB. The shared drama3_codec.safetensors adds 1.57 GB and works with either quant.
- Q2_K: the smallest option; about 16.2% smaller than Q3_K. Choose it when download size or system RAM is the priority.
- Q3_K: retains more expert precision and had lower sampled weight error. Choose it when you can accommodate the larger file; listen to both with your own prompts and reference voices.
The quant names describe the routed experts, not every tensor in the model. All 752 non-expert tensors retain their original dtypes and values, byte for byte between these builds. These include the shared experts, routers, attention/DeltaNet layers, embeddings, compact audio head, and audio tower. The codec is shared and unchanged between the quants.
Both builds were quantized directly from FP32 reconstructions of the official block-FP8 experts using GGML's reference encoders. Q2_K was not made by requantizing Q3_K or W4A8.
How the custom nodes work
The updated ComfyUI-Drama3 loader recognizes the packed Q2_K and Q3_K expert tensors. Experts remain packed in pinned system RAM, and a bounded LRU cache keeps recently used packed experts on the GPU. CUDA generation reads the packed blocks through fused Triton kernels; prefill expands the expert being processed to BF16.
Both quants use the same Load Drama 3, voice, generation, and codec nodes. Existing upstream node IDs are retained, and the bundle also retains support for the corrected W4A8 build.
These are ComfyUI .safetensors models using GGML K-quant block layouts. They require the included K-quant integration; GGUF, llama.cpp, audio.cpp, and a Transformers pipeline are not the inference path for these files.
Installation
Validated path: Linux + NVIDIA CUDA + Triton. Use the Python environment that runs ComfyUI.
Use an existing working ComfyUI installation. Linux, NVIDIA CUDA, and Triton are the validated execution path. Install or check dependencies using the Python environment that runs ComfyUI.
Fresh node installation
- Stop ComfyUI completely.
- Download ComfyUI-Drama3-KQuants.zip and extract it into
ComfyUI/custom_nodes/. The archive contains the completeComfyUI-Drama3/source folder, including K-quant support. - If a
ComfyUI-Drama3folder is already present, move it to a backup outsidecustom_nodesfirst. Keep only one active copy to avoid duplicate node registration. - Place your chosen model and the shared codec as shown below. Download both model files only if you want to switch between quants.
- Restart ComfyUI completely.
ComfyUI/
├── custom_nodes/
│ └── ComfyUI-Drama3/
├── models/
│ ├── diffusion_models/
│ │ ├── drama3_q2_k_experts_bf16.safetensors
│ │ └── drama3_q3_k_experts_bf16.safetensors
│ └── vae/
│ └── drama3_codec.safetensors
└── output/
└── drama3_voices/
└── synthetic_voice.safetensors # optional test reference
If you already have the compatible Drama 3 codec, reuse it.
First generation
For a quick test, download comfy_q2_workflow.json and synthetic_voice.safetensors. Place the voice file in ComfyUI/output/drama3_voices/, then drag the workflow into ComfyUI. The reference voice was generated synthetically for validation.
In Load Drama 3, select either model and use:
| Setting | Starting value |
|---|---|
experts |
in RAM — required for Q2_K/Q3_K |
expert_cache_gb |
8 — the measured cache size, in GiB |
second_gpu |
none — required for the current K-quant path |
Generation temperature |
1.0 |
Generation top_p |
0.9 |
Generation top_k |
30 |
Select drama3_codec.safetensors in Load Drama 3 Codec, confirm the saved voice is selected in Load Drama 3 Voice, edit the text in Drama 3 Generate, and queue the workflow. To compare quants, change only the model filename and keep the voice, prompt, seed, and sampling settings fixed.
expert_cache_gb = 0 enables automatic sizing and was also tested. The cache setting budgets expert cache VRAM, not total GPU memory. Dense layers, activations, and temporary buffers need additional space.
To use your own reference, start from the node bundle's example_workflows/drama3_voice_clone.json. Supply a clean speech clip and its exact transcript to Drama 3 Encode Voice, then save it with Save Drama 3 Voice for reuse. Qwen3-ASR weights are a separate optional download if you want automatic transcription; the saved-voice test above does not require them.
Example text:
[sighs] Well, that could have gone better. [laughs] Let's try again.
The model also supports speaker tags such as <|speaker:0|> and <|speaker:1|>. Match each supplied voice's transcript to its speaker tag when using multiple voices; see the upstream node documentation for the full prompt syntax.
Tested environment and memory
| Component | Validated version or configuration |
|---|---|
| OS / GPU | Linux / one NVIDIA RTX PRO 6000 Blackwell Workstation Edition |
| Python | 3.12 |
| PyTorch | 2.12.0+cu130 |
| Triton | 3.7.0 |
| ComfyUI | 0.41.0 |
| Comfy Kitchen | 0.2.37 |
| Expert offload | Pinned system RAM, 8 GiB GPU cache for benchmarks |
These versions record the tested environment; they are not a claim that every component must use exactly that version. CUDA K-quants require Triton. A CPU reference fallback exists, but CPU performance has not been qualified.
System RAM must accommodate the packed expert banks plus loading overhead, ComfyUI, and the operating system. The expert-bank sizes in the first table are not total RAM requirements. Minimum RAM and VRAM configurations were not established. The measured 8 GiB cache runs peaked at approximately 14.2 GiB of PyTorch-allocated VRAM, which excludes other GPU allocations.
Windows, AMD/Intel GPUs execution were not validated in this project. The current K-quant implementation supports single-GPU RAM offload; selecting experts in VRAM or a second GPU is unsupported.
Local benchmark results
The following results come from a matched three-way comparison on one RTX PRO 6000 Blackwell, using the same 8 GiB expert cache, reference voice, prompts, seeds, and sampling settings. Corrected W4A8 is included as a comparison baseline.
| Measurement | Q2_K | Q3_K | Corrected W4A8 baseline |
|---|---|---|---|
| Model file, decimal GB | 16.93 | 20.20 | 24.60 |
| Median steady generation, codec frames/s | 33.77 | 32.41 | 25.36 |
| Steady frames/s range | 31.99–35.51 | 28.13–33.78 | 20.78–26.66 |
| Median first codec frame, seconds | 0.650 | 0.641 | 1.158 |
| Generation real-time factor range | 0.714–0.902 | 0.739–1.116 | 0.977–1.571 |
| Peak PyTorch-allocated VRAM, GiB | 14.18 | 14.22 | 14.27 |
| ASR word edits / expected words | 1 / 59 | 0 / 59 | 0 / 59 |
| Sampled expert weight relative L2 error | 29.94% | 15.22% | 7.17% |
Four English cases—two seeds for a short utterance, narration, and a laughter-tagged utterance—were run twice per model after warmup, for eight timed takes per variant. Execution was serial, with reversed model order in the second round and no ASR running during timing. Each generation began with an empty expert cache. PyTorch's allocator was capped at 30 GiB.
The table compares complete implementations: K-quants use BF16 activations and Triton packed decode; W4A8 uses Comfy Kitchen's native path with INT8 activations. Equal cache byte budgets hold different numbers of experts. Q2_K and Q3_K timing ranges overlap, so the small median difference does not establish a consistent speed advantage.
All requested dialogue words were recovered in the first-round ASR check. Q2_K's laughter-tagged take also transcribed as “Haha,” counted as one insertion. Repeated runs produced identical code sequences within each quant. Both quants passed queued ComfyUI workflows; Q2_K additionally passed a visible UI Run. The focused loader patch was separately exercised with Q2_K, Q3_K, and corrected W4A8 using automatic cache sizing.
These checks establish intelligibility on a small English set. They do not establish equivalent voice similarity, emotion, nonverbal delivery, multilingual pronunciation, or long-form reliability. Weight error was measured over the same 640 expert-bank samples and is not an audio-quality score.
Audio is returned after generation and codec decoding. First codec frame timing measures generated codes, not first audible audio. Generation real-time factor is generation time divided by output duration and excludes codec decoding. The current ComfyUI workflow does not stream audio. These are not RTX 5090 benchmark results.
Troubleshooting
| Symptom | Action |
|---|---|
disable_weight_init has no attribute MoEExperts |
Install the K-quant node bundle or focused loader patch, then restart ComfyUI completely. The standard loader does not recognize these packed experts. |
Q2_K / Q3_K CUDA execution requires Triton |
Check that Triton is available in the Python environment actually running ComfyUI. Linux CUDA is the tested path. |
| Error about experts in RAM or a second GPU | Set experts = in RAM and second_gpu = none. |
| CUDA out of memory | Lower expert_cache_gb and account for other models using VRAM. The cache is only part of the GPU allocation. |
| Voice file is missing from the dropdown | Put the saved voice in output/drama3_voices/ and refresh the file list. |
| Silent or incomplete speech | Check that the reference transcript covers the clip exactly, use clean speech cut between sentences, and try another seed or reference. Keep the starting sampling settings above. |
| Duplicate nodes after installation | Keep one active ComfyUI-Drama3 folder. Move backups outside custom_nodes. |
Provenance and file integrity
- Base checkpoint: fishaudio/drama-3 at e0a9218.
- Full node bundle base: ComfyUI-Drama3 at 66c4fb9.
- Quantization layouts and reference encoders: ggml / llama.cpp. GGML is used for conversion; its encoder library is not required at inference time.
Model sizes and SHA-256 checksums:
drama3_q2_k_experts_bf16.safetensors bytes: 16927935960 sha256: ffd9c179fa71e72bfdce17370fe1efe1f6bb606109348f215cd43d4dfb39f444
drama3_q3_k_experts_bf16.safetensors bytes: 20199493096 sha256: 8c5bcaea481c24dbc4a7f417bebc3046f198bb2a02a53ef885fa9761ddaf8479 drama3_codec.safetensors bytes: 1565647580 sha256: 240f9e91b36b62efa544ffcd529e2b3954bae049107b4b93e064e3b9c2fc146a
License and credits
The weights and codec are distributed under the bundled Fish Audio Community License, Version 1.0, matching the license file accompanying the pinned source checkpoint. Read that agreement for the applicable use, distribution, commercial, and voice-consent terms.
The node bundle retains ComfyUI-Drama3's GPL-3.0 license and its component-specific licenses: Fish Audio-derived portions under the Fish Audio Community License, Qwen-derived portions under Apache-2.0, and GGML attribution under its MIT license. The archive includes the source, license files, and notices.
Credit to Fish Audio for Drama 3 and its codec, PedroMarinhoDev for the native ComfyUI integration, and ggml contributors for the K-quant layouts and reference encoders. This release adds Q2_K/Q3_K expert conversion, packed CUDA execution, RAM-offload integration, and the focused loader installer.
Built with Fish Audio. This project is not affiliated with, sponsored by, or endorsed by Fish Audio or the upstream node authors.
Model tree for UltimateIntent/Fish-Audio-Drama-3-Q2-Q3
Base model
fishaudio/drama-3