tigerquoll/kokoro-82m-coreml-cpu

Kokoro-82M — Faithful Core ML (CPU) Re-Export

A faithful Core ML re-export of the Kokoro-82M text-to-speech pipeline, built to run on the CPU compute unit of Apple devices — so neural TTS keeps working while the app is backgrounded or the screen is locked (where the GPU/MLX path is killed), with no quality loss versus reference PyTorch Kokoro.

These are modified derivatives of the original Apache-2.0 weights: the PyTorch modules were re-exported to Core ML ML Program format (FP32, dynamic sequence length), with the bidirectional duration LSTMs re-exported as a single native recurrent lstm op rather than token-bucketed unrolled graphs. No weights were retrained or altered in value.

TL;DR results

• Native recurrent duration op. The duration prediction LSTMs are exported as a real Core ML lstm (dynamic length, FP32, CPU), replacing bucketed unrolled-LSTM packages.
• ~26–40× faster cold load. First-use compile drops from the ~9 s uncacheable iPhone BNNS load to ~0.23–0.34 s.
• Bucket-free. Removes the maxChunkTokens = 62 ceiling that caused choppy ~4 s pacing; one dynamic graph handles up to the 510-token cap.
• Numerically faithful. AdaIN kept (not approximated to identity); zero integer-frame duration error vs PyTorch FP32 at every length, and output waveform NCC > 0.99 against reference Kokoro.

Benchmarks measured by the author on Apple Silicon / iPhone; reproduce with the conversion + verification scripts (see Reproduction).

What's in here

models/ — five ML Program packages (FP32, CPU-only, dynamic/flexible sequence length; the variable token/frame axis is the last axis, […, T]):

package	role	inputs	outputs
kokoro_duration_modelA.mlpackage	ALBERT + BERT-encoder + main TextEncoder: phoneme IDs → duration/text features	input_ids int32 [1,T], attention_mask int32 [1,T] (T≤510)	d_en f32 [1,512,T], t_en f32 [1,512,T]
kokoro_duration_modelB.mlpackage	ProsodyPredictor (DurationEncoder bi-LSTM + predictor LSTM + duration proj): per-token duration	d_en_cat f32 [1,640,T], style f32 [1,128]	pred_dur int32 [1,T], d f32 [1,T,640]
kokoro_f0ntrain_dynamic.mlpackage	shared bi-LSTM + F0/N AdaINResBlk: pitch + noise/energy curves	en f32 [1,640,T], s f32 [1,128]	F0_pred f32 [1,T_f0], N_pred f32 [1,T_f0] (T_f0 = 2·T)
kokoro_decoder_pre_dynamic.mlpackage	decoder front-end (AdaIN): fuse aligned text with F0/noise → pre-generator latent	asr f32 [1,512,T], f0_raw f32 [1,T_f0], n_raw f32 [1,T_f0], ref_s f32 [1,256]	x_pre f32 [1,512,T_f0]
kokoro_decoder_har_post_dynamic.mlpackage	iSTFTNet generator/post (AdaIN, CustomSTFT): latent + harmonic source → waveform	x_pre f32 [1,512,T_f0], ref_s f32 [1,256], har f32 [1,22,N_har]	waveform f32 [1,1,T_audio]

samples/ — {short,medium,long}_faithful_cpu.wav (this pipeline) paired with *_reference.wav (PyTorch Kokoro) for A/B comparison.

Pipeline order

input_ids, attention_mask
   └─► [A] ─► d_en, t_en
              d_en (+ tiled style, host-side concat) ─► d_en_cat
                 └─► [B] ─► pred_dur, d
   host-side frame alignment using pred_dur:
      d   ─► en  [1,640,T_frames]
      t_en ─► asr [1,512,T_frames]
   en + s ───────────────► [F0Ntrain] ─► F0_pred, N_pred
   asr + F0_pred + N_pred + ref_s ─► [decoder_pre] ─► x_pre
   host-side hn-nsf DSP: F0_pred ─► har
   x_pre + ref_s + har ──────────► [decoder_har_post] ─► waveform

The duration-based frame alignment and the hn-nsf harmonic source (har) are computed on the host (Swift), not in Core ML.

Voices / style embeddings

These packages take the style/voice as an input tensor (style/s [1,128], ref_s [1,256]); they do not bundle voices. Use the voice packs from the upstream model — see hexgrad/Kokoro-82M (voices/). They are re-hosted here only if a voices/ folder is present.

Usage (Core ML, Swift)

Load each package with MLModelConfiguration().computeUnits = .cpuOnly and drive them in the order above. A reference Swift implementation of the host-side glue (duration alignment, style concat, hn-nsf harmonic source, chunker) lives in the KokoroCoreMLPipeline package of the application repo.

let cfg = MLModelConfiguration()
cfg.computeUnits = .cpuOnly   // survives backgrounding / lock; deterministic
let durationA = try MLModel(contentsOf: urlA, configuration: cfg)
// … see KokoroCoreMLPipeline for the full pipeline wiring.

Reproduction

The full conversion + verification pipeline (g1_duration_reexport.py, s1_f0ntrain_reexport.py, s2_decoder_feasibility.py, s3_full_pipeline.py, finalize_models.py, verify_final.py) re-exports these from the official kokoro-v1_0.safetensors weights and checks parity against PyTorch FP32. Converted with coremltools 8.3.0 / torch 2.5.0.

License & attribution

Licensed under the Apache License, Version 2.0 (see LICENSE), as derivatives of hexgrad/Kokoro-82M (Apache-2.0). The original model's training data attribution is preserved (see NOTICE):

• Koniwa — CC BY 3.0
• SIWIS — CC BY 4.0

These Core ML packages are modified (re-exported) versions of the original weights. "Kokoro" is used for attribution only and does not imply endorsement.

Citation

@misc{kokoro_coreml_cpu,
  title  = {Kokoro-82M — Faithful Core ML (CPU) Re-Export},
  author = {Tigerquoll},
  note   = {Derivative of hexgrad/Kokoro-82M (Apache-2.0)},
  year   = {2026}
}