# Mini Native24 Evaluation Full corpus: 1,688 texts x six voices = 10,128 clips. These are completed local reproductions on a repeatedly used known/development corpus, not an official submission, a new independent test, or certified training-disjoint evidence. ## Configuration Evaluated public revision: `9f539a24df04fe28e3d4507934b813d6654ddb2e`. Original/base speech, native 24 kHz, PCM16 recording, seed 42, all six fixed voices. No events, enhancer, VoiceTuner or pitch polish; core weights are unchanged. ## Word Error Rate WER is **percent; lower is better**. Qwen3-ASR-1.7B uses the pinned official-source normalizer, scorer and corpus units, without a reference-text prompt. This is not an official leaderboard run. ### Local execution recipe Each voice job calls the pinned official transcriber on consecutive blocks of up to **24 rows**. Official duration sorting is applied **within each block manifest**, with batch size **8**, at most **256 new tokens**, a **30-second audio cap**, and **BF16 CUDA** inference. This is not duration sorting of the entire job as one global manifest. Execution used a local **NVIDIA RTX 5090**, not the upstream reference/default **NVIDIA H200**. The shared metric normalizer/source does not establish bit-for-bit equivalence to the official whole-manifest execution, a leaderboard submission, or a rank. | Aggregate | WER % | | --- | ---: | | Equal dataset, then equal six-voice mean | 0.739006 | | Pooled word counts (separate) | 0.875043 | Pooled counts: 1114 edits / 127308 reference words. Full and subset scopes are never mixed. | Voice | CV3 % | Seed-TTS % | MiniMax % | Dataset mean % | | --- | ---: | ---: | ---: | ---: | | hope | 1.220008 | 0.736957 | 0.263158 | 0.740041 | | sophia | 1.152230 | 0.820702 | 0.210526 | 0.727819 | | cassidy | 1.179341 | 0.762080 | 0.263158 | 0.734860 | | james | 1.043785 | 0.862574 | 0.315789 | 0.740716 | | jasper | 1.111563 | 0.812327 | 0.263158 | 0.729016 | | mark | 1.098007 | 0.870949 | 0.315789 | 0.761582 | ## Objective Quality Proxies A separately frozen 180-clip panel (30 per voice), reused without resynthesis. These are objective model scores, **not human MOS** or a claim of superior audible quality. Scoring uses a 30-second per-clip cap; UTMOSv2 averages three seeded predictions per clip. | Proxy | Six-voice mean | Clips | | --- | ---: | ---: | | UTMOS22Strong | 4.478976 | 180 | | UTMOSv2 | 3.916612 | 180 | ## Evidence and Limits Machine-readable counts, scope and receipt hashes: [evaluation.json](evaluation.json). Objective-quality proof includes each metric's reader-admission hash, independently pinned asset/source receipts and weight hashes; UTMOSv2 additionally records its verified runtime/source-tree receipt. These are saved pre-scoring checks, not a new remote-payload audit by this publication step. The known corpus has informed prior development. Reusing fixed voices and texts does not establish generalization, cloning quality, or all-platform compatibility. Saved receipts and arithmetic checked; this preparation does not rerun ASR, official normalization/alignment, quality models, or human listening. No official leaderboard submission. Saved audio sample inputs and container identities are separate: [audio provenance](assets/audio-provenance.json).