lightsofapollo/omnivoice-mlx-q4-g64

OmniVoice — int4 g=64 (MLX)

4-bit, group-size-64 affine quantization of k2-fsa/OmniVoice, produced with mlx-audio for Apple Silicon.

Sizes

	Backbone	Total
original (bf16, this repo's audio_tokenizer/ is unchanged)	~1.2 GB	~1.6 GB
this repo (int4 g=64 backbone, bf16 tokenizer)	329 MB	724 MB

Quantization applies only to the Qwen3 backbone Linear layers (and the tied audio embedding/head matmuls). The Higgs Audio V2 acoustic tokenizer (decoder, RVQ, semantic) is left at bfloat16 to preserve audio fidelity.

Performance (M-series, mlx-audio 0.x)

Prompt	RTF (bf16)	RTF (this)
"Voice synthesis on Apple Silicon has come a long way. We can now generate full sentences in real time."	3.68×	4.59× (+25%)

Whisper-small round-trip: identical transcript to bf16 on the long prompt.

Usage (mlx-audio Python)

import json
import mlx.core as mx
import mlx.nn as nn
from huggingface_hub import snapshot_download
from mlx_audio.tts.models.omnivoice.config import OmniVoiceConfig
from mlx_audio.tts.models.omnivoice.omnivoice import Model

path = snapshot_download("lightsofapollo/omnivoice-mlx-q4-g64")
cfg_dict = json.load(open(f"{path}/config.json"))
model = Model(OmniVoiceConfig(**{k: v for k, v in cfg_dict.items() if k in OmniVoiceConfig.__dataclass_fields__}))

# IMPORTANT: quantize the model shape *before* loading weights.
q = cfg_dict["quantization"]
nn.quantize(model, group_size=q["group_size"], bits=q["bits"], mode=q.get("mode", "affine"),
            class_predicate=lambda _p, m: hasattr(m, "to_quantized"))

raw = dict(mx.load(f"{path}/model.safetensors"))
model.load_weights(list(model.sanitize(raw).items()))
mx.eval(model.parameters())

How this was made

python -m mlx_audio.tts.models.omnivoice.convert \
    --model k2-fsa/OmniVoice --output omnivoice-bf16 --dtype bfloat16

python -m mlx_audio.convert \
    --hf-path omnivoice-bf16 --mlx-path omnivoice-q4-g64 \
    --quantize --q-bits 4 --q-group-size 64

License

Apache-2.0 (inherited from k2-fsa/OmniVoice).