maya2-native-fast

Optimized CPU inference for maya-research/maya2-native: ONNX graphs, int8 backbone, fp16 depth, a torch-free numpy runtime.

This is not a fine-tune. The weights are converted from the upstream checkpoint. No training data was used.

What is in the repo

Path	Contents
assets/v1/backbone_wo8.onnx + .data	28-layer Qwen3 backbone, int8 weights (MatMulNBits, block 32), 1.6 GB
assets/v1/depth_frame_fp16.onnx + .data	Depth sampler, 15 levels, fp16 storage, Gumbel top-k sampling in-graph, 873 MB
assets/v1/mimi_decoder.onnx	Mimi decoder, fp32, dynamic frame count, 195 MB
assets/v1/embed_bf16.bin, audio_in_bf16.bin	Text and audio embedding tables, bf16, memory-mapped
assets/v1/prompt.npz, tokenizer.json, voices.json, controls.json	Prompt weights, extended tokenizer, voices, sentence controls
assets/v1/manifest.json	sha256 of every file above, runtime shapes. The runtime refuses mismatches.
assets/v2-lowram/	Optional lower-RAM set: same files as v1 except the depth graph depth_frame_q8a1_kg8.onnx + .data (int8 block 32, fp32 compute, group 8 kept fp16), 535 MB. About 640 MB less peak RAM, about 2.5x slower. Its manifest.json names the depth graph under graphs.
src/maya_fast/	Runtime (numpy, onnxruntime, tokenizers; no torch)
tools/maya_export/	Export, quantization and evaluation scripts, to rebuild the assets from the upstream checkpoint
tools/make_asset_variant.py	Builds a variant asset set (like v2-lowram) from v1 with one graph replaced, and rewrites its manifest
benchmarks/	End-to-end eval (maya_fast_eval.py), speaker similarity, blind listening pairs, test set, and result notes
tests/	Runtime and export tests

Measured results (2026-10-29, Apple M5, CPU, onnxruntime 1.31)

End to end: 25 Marathi test sentences x 3 seeds (75 utterances per set), voice Charvi, 8 threads. Error rate is the character error rate (CER) of an ASR transcript (IndicConformer) of the output. The reference is the upstream PyTorch model on the same sentences and seeds, scored the same way.

Check	v1	v2-lowram	Target	Status
CER	0.137	0.122	<= reference (0.147) + 0.01	pass, pass
Speaker similarity (ECAPA cosine) vs reference	-0.009	-0.013	>= -0.02	pass, pass
Peak RSS	4467 MB	3823 MB	3000 MB target, 5000 MB limit	over target, under limit
Mean RTF	2.21	5.46	offline RTF <= 4	pass, fail
Mean RTF, 4 threads (seed 0)	2.07	6.12	offline RTF <= 4	pass, fail

Component checks:

Check	Result	Target	Status
Depth fp16 vs fp32 reference, teacher-forced, 75 files	top-1 >= 0.9965 every level, KL <= 6e-6	top-1 >= 0.98, KL <= 0.02	pass
Depth int8 (v2-lowram) vs fp32 reference	min top-1 0.9806 (level 8)	top-1 >= 0.98, KL <= 0.02	pass (thin margin)
Depth fp32 vs upstream PyTorch, 20 frames	rel 1.6e-6	rel <= 1e-4	pass
Mimi decoder vs upstream	max abs 1.5e-6	<= 1e-4	pass
Prompt port vs upstream, 500 prompts	0 token mismatches, max embed error 7.5e-9	<= 1e-5	pass
Backbone int8 vs fp32, teacher-forced	top-1 0.9677, KL 0.0034	top-1 >= 0.99	fail (waived)

Known limits:

• Neither set meets the 3 GB RAM target. The backbone holds about 2 GB, and no smaller backbone passed its quality check (int4 RTN/HQQ: top-1 0.72 to 0.78).
• The int8 backbone misses top-1 agreement; expect small differences in generated codes.
• The RTF numbers above are about 2x slower than an earlier run of v1 on the same machine (RTF 0.92 to 1.04, 2 to 3 sentences). The cause is not known, so treat speed numbers as approximate.
• All numbers are from an Apple M5. An Intel i5 8th-gen laptop (the deployment target) has not been measured yet.
• Not yet checked by listening tests (blind A/B pairs are generated, not rated).
• Hindi (--lang hi) works mechanically; it has not been checked for quality.
• Real-time playback is not supported.

Speed

Per-frame cost in the 75-utterance run above (one frame = 80 ms of audio):

Stage	v1	v2-lowram
Backbone (int8, 28 layers)	53 ms	39 ms
Depth (15 levels)	117 ms (fp16)	394 ms (int8, fp32 compute)
Mimi decoder	3 ms	2 ms

The int8 depth is slow because onnxruntime has no fast kernel for 8-bit weights with fp32 compute. The faster int8-compute variants failed the quality check.

For comparison, the upstream PyTorch path measured in this project's spike: RTF 4.62 (best, bf16) and peak RSS 7.5 to 8.0 GB.

Install

python -m venv .venv
.venv/bin/pip install -r requirements-fast.txt

Requires Python 3.11 to 3.13 (onnxruntime 1.31 has no Python 3.10 wheels). Peak memory is about 4.5 GB with assets/v1, about 3.8 GB with assets/v2-lowram.

Use: command line

Run from the repo root (so src is importable):

PYTHONPATH=. .venv/bin/python -m src.maya_fast "नमस्कार, मी तुमची मदत करण्यासाठी इथे आहे." \
    --assets assets/v1 -o out.wav

Options:

• --voice NAME: named voice from assets/v1/voices.json (case-insensitive). Default Charvi.
• --lang mr|hi: language tag. Default mr.
• --seed N: random seed. Same seed and threads give the same audio.
• --threads N: ORT threads. Default min(8, cores).
• --assets DIR: assets/v1 (faster) or assets/v2-lowram (less RAM, slower).

Use: Python

from src.maya_fast.engine import MayaFast

engine = MayaFast(assets_dir="assets/v1", threads=8)
wav, metrics = engine.synthesize("नमस्कार, मी तुमची मदत करण्यासाठी इथे आहे.", voice="Charvi", seed=0, lang="mr")
# wav: float32, 24 kHz mono

Write it with soundfile or the standard wave module.

Assets are verified against manifest.json at load. A bad or partial download fails with the file name.

Rebuild the assets from upstream (optional)

Needs the upstream checkpoint (maya-research/maya2-native, kyutai/mimi, speechbrain/spkrec-ecapa-voxceleb) and an upstream maya3 install in a second venv (.venv-tts, torch 2.8, transformers 5.6.2). The export scripts write to $MAYA_FAST_WORK/assets/v1, default ~/.cache/maya_fast. See tools/maya_export/:

1. export_prompt_assets.py, export_backbone.py --precision wo8, export_depth_all.py, export_codec.py
2. finalize_assets.py writes the runtime manifest. Run each step under tools/memguard.py, which kills the process above a memory limit.

The low-RAM depth graph is built with tools/maya_export/quant_depth_frame.py; the exact commands are in benchmarks/depth_ram.md. Then tools/make_asset_variant.py derives assets/v2-lowram from assets/v1. The backbone quantization sweep (all variants failed) is tools/maya_export/quant_sweep_backbone.py, results in benchmarks/backbone_quant_sweep.md.

Evaluate

benchmarks/maya_fast_eval.py --assets assets/v1 synthesizes the test set, transcribes it with an ASR model in a separate process, and writes CER, RTF and peak RSS. It needs the ASR dependencies (sherpa_onnx, soxr) in addition to requirements-fast.txt. benchmarks/maya_fast_spk.py (needs torch + speechbrain) scores speaker similarity; benchmarks/maya_fast_listen.py makes blind A/B pairs and scores answers with --score answers.json.

License and attribution

• These weights and code are derived from Maya2 Native (Apache-2.0) by Maya Research.
• The Mimi decoder is derived from Kyutai Mimi (CC-BY-4.0). Keep this attribution.
• The text backbone is derived from Qwen3-1.7B-Base (Apache-2.0).
• Speaker conditioning uses SpeechBrain ECAPA-TDNN (Apache-2.0).
• Full notices in NOTICE.

The model adds no audio watermark.

Citation

Please cite the upstream model:

@misc{maya2native2026,
  title  = {Maya 2 Native: Streaming Multilingual Text-to-Speech},
  author = {{Maya Research}},
  year   = {2026},
  url    = {https://huggingface.co/maya-research/maya2-native},
  note   = {Release 1.0}
}