README.md

---
license: cc-by-nc-4.0
language:
  - en
datasets:
  - KRAFTON/Raon-OpenTTS-Pool
tags:
  - text-to-speech
  - tts
  - zero-shot-tts
  - f5-tts
  - diffusion-transformer
  - flow-matching
  - voice-cloning
  - speech-synthesis
  - english
  - comfyui
  - int8
  - convrot
pipeline_tag: text-to-speech
---

# Raon-OpenTTS for ComfyUI (fp32 / bf16 / INT8 ConvRot safetensors)

ComfyUI-ready repack of **[KRAFTON Raon-OpenTTS](https://github.com/krafton-ai/Raon-OpenTTS)** — open-weight, open-data zero-shot voice cloning (F5-TTS-style CFM/DiT, 16 kHz, English) — converted to clean inference-only safetensors, with an **INT8 ConvRot** build that runs natively through [comfy-kitchen](https://github.com/Comfy-Org/comfy-kitchen) quantized kernels.

**ComfyUI node pack:** <https://github.com/Saganaki22/ComfyUI-Raon-OpenTTS>

**Original models:** [KRAFTON/Raon-OpenTTS-1B](https://huggingface.co/KRAFTON/Raon-OpenTTS-1B) · [KRAFTON/Raon-OpenTTS-0.3B](https://huggingface.co/KRAFTON/Raon-OpenTTS-0.3B) · [GitHub](https://github.com/krafton-ai/Raon-OpenTTS) · [arXiv:2605.20830](https://arxiv.org/abs/2605.20830) · [Dataset](https://huggingface.co/datasets/KRAFTON/Raon-OpenTTS-Pool)

**Vocoder:** [speechbrain/tts-hifigan-libritts-16kHz](https://huggingface.co/speechbrain/tts-hifigan-libritts-16kHz) (mirrored here as `tts-hifigan-libritts-16kHz/generator.ckpt` for one-stop download)

## What's inside

```text
Raon-OpenTTS-1B/   (1042M params, dim 1408, depth 28, heads 24)
  config.yaml
  vocab.txt
  Raon-OpenTTS-1B-fp32.safetensors           4.17 GB   lossless EMA extraction (reference)
  Raon-OpenTTS-1B-bf16.safetensors           2.08 GB   half-precision runtime build
  Raon-OpenTTS-1B-int8-convrot.safetensors   1.41 GB   INT8 ConvRot (comfy-kitchen)
  Raon-OpenTTS-1B-int8-convrot-verify.json   per-layer quantization error table
Raon-OpenTTS-0.3B/ (339M params, dim 1024, depth 22, heads 16)
  config.yaml
  vocab.txt
  Raon-OpenTTS-0.3B-fp32.safetensors         1.35 GB
  Raon-OpenTTS-0.3B-bf16.safetensors         0.68 GB
  Raon-OpenTTS-0.3B-int8-convrot.safetensors 0.50 GB
  Raon-OpenTTS-0.3B-int8-convrot-verify.json
tts-hifigan-libritts-16kHz/generator.ckpt    HiFi-GAN vocoder (16 kHz)
```

All builds are the **EMA weights** from the official training checkpoints (`model_520000.pt` / `model_225000.pt`), which is what the official inference path uses. The fp32 extraction is verified **bit-exact** (every tensor `torch.equal` against the source EMA state dict; strict `load_state_dict` passes with 0 missing / 0 unexpected keys).

## INT8 ConvRot

Comfy-native `int8_tensorwise` + ConvRot format (`<layer>.weight` int8, `<layer>.weight_scale` fp32 per-row, `<layer>.comfy_quant` JSON marker). Only the repeated transformer-block GEMMs are quantized — attention Q/K/V/out and the two FFN projections; embeddings, AdaLN modulation, time MLP, input projection, ConvNeXt text blocks, norms, convolutions and the final projection stay full precision.

| | 1B | 0.3B |
|---|---|---|
| quantized layers | 168 (28 blocks x 6) | 132 (22 blocks x 6) |
| group sizes | 112x GS64 + 56x GS256 | 132x GS256 |
| quantized params | 686.3M (65.9%) | 184.5M (54.6%) |
| weight error (mean / max) | 0.84% / 1.26% | 0.80% / 0.89% |
| mel vs bf16 build (rel L2 / cosine) | ~0.95% / 0.99996 | ~0.87% / 0.99997 |

GS64 matters here: the 1B transformer width 1408 is not divisible by 256, so 1408-wide layers use ConvRot group size 64 (1408 % 64 == 0) while the 1536/5632-wide layers use 256.

Measured on RTX 5090 (load peak VRAM / inference peak VRAM / speed):

| 1B | load | inference | RTF |
|---|---|---|---|
| fp32 | 4.26 GB | 4.47 GB | 0.37 |
| bf16 | 2.22 GB | 2.39 GB | 0.22 |
| int8-convrot | 1.50 GB | 1.67 GB | 0.37 |

INT8 trades some speed for the smallest memory footprint at these GEMM sizes on this GPU; bf16 is the fastest build. Generated audio transcribes word-perfectly with Whisper across all builds (see the node pack repo for the validation harnesses).

## Usage

Install the node pack: <https://github.com/Saganaki22/ComfyUI-Raon-OpenTTS> — the **Raon OpenTTS Load Model** node downloads from this repo automatically when the files are missing, into `ComfyUI/models/raon_opentts/`.

Manual download:

```python
from huggingface_hub import snapshot_download
snapshot_download("drbaph/Raon-OpenTTS-comfyui",
                  local_dir="ComfyUI/models/raon_opentts")
```

## Vocab note (5559 vs 5555)

The shipped `vocab.txt` has 5559 tokens while both checkpoints embed 5555 rows. The node pack builds the model from the checkpoint shape (ground truth) and drops the four overhanging tokens — the four highest-codepoint entries of the sorted vocab (U+FDFA, U+FDFB, U+FFFD, U+1F3B5) — from the runtime map. All realistic text is unaffected (verified by Whisper transcription of generated audio).

## Citation

```bibtex
@article{kim2026raonopentts,
  title     = {Raon-OpenTTS: Open Models and Data for Robust Text-to-Speech},
  author    = {Kim, Semin and Chung, Seungjun and Moon, Taehong and Lee, Sangheon and Ahn, Minyoung and Lee, Keon and Kim, Nam Soo and Cho, Jaewoong and Schmidt, Ludwig and Lee, Kangwook and Park, Dongmin},
  journal   = {arXiv preprint arXiv:2605.20830},
  year      = {2026},
  url       = {https://arxiv.org/abs/2605.20830}
}
```

## License

This repository is licensed under the [Creative Commons Attribution-NonCommercial 4.0 International License](https://creativecommons.org/licenses/by-nc/4.0/), matching the upstream KRAFTON model weights. Please credit KRAFTON AI (see citation above) when using these weights.