| --- |
| license: other |
| base_model: BigBlueCeiling/MisoTTS-bf16 |
| tags: |
| - text-to-speech |
| - quantized |
| - torchao |
| --- |
| |
| # MisoTTS int8 (BigBlueCeiling) |
|
|
| A weight-only **int8** quantization of |
| [BigBlueCeiling/MisoTTS-bf16](https://huggingface.co/BigBlueCeiling/MisoTTS-bf16), |
| produced with torchao (`int8_weight_only`). Only the backbone/decoder Linear |
| layers are quantized; the embeddings, output heads, and projection stay bf16. |
|
|
| > Experimental. Weight-only int8; bf16 remains the reference. |
|
|
| ## What it is for |
|
|
| Lowering the hardware floor. Quantization here is a **memory** lever, not a speed |
| one: MisoTTS decodes one frame at a time, and those tiny per-step matmuls cannot |
| feed the GPU's low-precision tensor cores, so int8 dequantizes to bf16 for the |
| matmul. You get the VRAM saving, not a throughput win. |
|
|
| - **Fits:** ~16 GB VRAM cards (RTX 4060 Ti 16G, 4070 Ti Super, A4000, ...) |
| - **Quality:** Quality-preserving: mean CER 0.11, WER 0.14, UTMOS 3.96 - statistically even with bf16 (CER 0.10 / WER 0.15 / UTMOS 3.94). |
|
|
| ## Use |
|
|
| This checkpoint is a `torch.save`'d torchao state_dict (`model.pt`). The serving |
| core in the [MisoTTS repo](https://github.com/eoffermann/MisoTTS) pulls it |
| automatically when GPU-sense detects a matching VRAM tier. To load it directly: |
| |
| ```python |
| from generator import load_miso_8b # from the MisoTTS repo |
| gen = load_miso_8b("cuda", model_path_or_repo_id="BigBlueCeiling/MisoTTS-int8", |
| prequantized=True) |
| ``` |
| |
| Requires torch>=2.7 and a matching torchao (loading unpickles a torchao tensor |
| subclass, so `weights_only=False` is used; load only checkpoints you trust). |
|
|
| Model and original inference code are MisoLabs' work; see the upstream license. |
|
|
