lineofflight/pocket-tts-coreml

Pocket TTS for Core ML

Kyutai's Pocket TTS, converted to Core ML for Apple Silicon Macs: the 6-layer Dutch, the 6-layer French, English 2026-10, and Kyutai's newer 6-layer Spanish, Portuguese, German and Italian. They are the packs voice, a speech tool for coding agents, speaks through on a Mac.

The packs have the layout, inputs and outputs of FluidInference's v2.1 packs in FluidInference/pocket-tts-coreml, which has no Dutch, no 6-layer French and no English 2026-10, and Kyutai's April 2026 models of Spanish, Portuguese, German and Italian. They were converted with FluidInference's own recipe.

What's here

constants_bin/mimi_init_state/  Mimi's initial state, a float32 file per tensor, which every pack shares
v2.1/<pack>/
  cond_prefill_ane.mlmodelc     the voice's and text's keys and values into the cache, in one call
  flowlm_step_ane.mlmodelc      the transformer, one frame a call
  flow_decoder_fused.mlmodelc   the flow's 8 Euler steps in one call
  mimi_decoder.mlmodelc         a latent to 80 ms of 24 kHz audio
  constants_bin/
    tokenizer.model             Kyutai's SentencePiece model for the language
    text_embed_table.bin        the text embeddings, float32 [4001, 1024]
    bos_emb.bin                 the BOS latent, float32 [32]
    bos_before_voice.bin        the latent before a voice's prompt, float32 [1024]
    <voice>.safetensors         a voice's KV-cache snapshot, Kyutai's own file

Pack	Kyutai model	Voices	Size
dutch	languages/dutch, 6 layers	daan and the 27 below	499 MB
french	languages/french, 6 layers	daan and the 27 below	499 MB
english_2026-10	languages/english_2026-10, 6 layers	the 27 below	496 MB
spanish	languages/spanish, 6 layers	daan and the 27 below	499 MB
portuguese	languages/portuguese, 6 layers	daan and the 27 below	499 MB
german	languages/german, 6 layers	daan and the 27 below	499 MB
italian	languages/italian, 6 layers	daan and the 27 below	499 MB

english_2026-10 is Kyutai's english_2026-09_2 training recipe retrained with a loader fix for an unaligned last word, so it reads sentence-final numbers. At the revision below it is also Kyutai's languages/english: the two folders hold the same files.

spanish, portuguese, german and italian are the models Kyutai has published under those names since October 2026, with their own tokenizers and voice snapshots. FluidInference's packs of the same names hold Kyutai's April 2026 models.

Mimi's initial state is the recipe's own, the same file for file for all seven packs and as constants_bin/mimi_init_state/ at the root of FluidInference's repository. It is here too, so the packs need nothing from elsewhere.

FILES.txt lists every file with its sha256 and size.

Models and inputs

Every model takes and gives float32. The first three compute in float16, Mimi in float32.

Model	Inputs	Outputs
cond_prefill_ane	conditioning [1, 256, 1024], valid_len [1], per layer i of 6: k_cache{i}, v_cache{i} [1, 512, 16, 64], position{i} [1]	new_k_cache{i}, new_v_cache{i}, new_position{i}
flowlm_step_ane	sequence [1, 1, 32] (the BOS latent on the first step, never NaN), the caches and positions	transformer_out [1, 1, 1024], is_eos [1, 1, 1] (a logit; stop above -4), the new caches and positions
flow_decoder_fused	transformer_out [1, 1024], latent_init [1, 32]	latent_final [1, 32]
mimi_decoder	latent [1, 32] and 24 state tensors	audio [1, 1, 1920] and the 24 states, under the traced var_ names of FluidInference's packs

The names and shapes match FluidInference's v2.1/english exactly.

Where it comes from

The weights are Kyutai's, from kyutai/pocket-tts-without-voice-cloning at revision a7056f505a4a8ca72d8b5fb1440071d6f1a35739, fetched with curl and checked against the sha256 the repository lists:

File	sha256
languages/dutch/model.safetensors	59b5c3249ab05e8b5fcb0a296e9b0cb0c0a20da5e1fd28c04bb90b2175612aeb
languages/french/model.safetensors	56366cfed6c25bfcd3bf4c61e66e06efaf52e78a5d06e1dcade1c588495bc699
languages/english_2026-10/model.safetensors	4c56cbd171c63e8a26ea61f5d48b8a0cd8f3c51f4652c0ab9a19bfcced8c931d
languages/spanish/model.safetensors	c4988ff376692f2ab1e85b17350755b08f16e49c0b5f0de8b271efa9abd71de7
languages/portuguese/model.safetensors	28e20f32515ba8bd0eaec69671f11f8921414fcdc375aa072fa09f46cc1057c5
languages/german/model.safetensors	9fe42605604832349e3f495177bde56b77ae8fee73b2ed4b63236092ccde7621
languages/italian/model.safetensors	16365c43eaabee9b222cd414d6a48f0976b4800a21fe5d55253f645eff7eeb1f

The tokenizers and the voices are Kyutai's files from the same revision, copied unchanged.

The conversion is FluidInference's mobius recipe, models/tts/pocket_tts/coreml (Apache-2.0), on its branch feat/pocket-tts-flowlm-ane at commit 210d6b6d21177115babfece0de5e1795cffec2a2. That branch carries the _ane and fused converters (its Trials 16 to 21) beside everything on main at the time; main, at 26f5fa8059d5ab3d34449adce6ff6ed86734e522, still makes only the older cond_step, flowlm_step and flow_decoder.

The recipe ran with torch 2.9.1, coremltools 9.0 and Kyutai's pocket-tts 4.0.1, the first two as its uv.lock pins them, and was compiled with xcrun coremlc from Xcode 27.1. As a check, the same steps on Kyutai's english_2026-04 rebuilt FluidInference's own v2.1/english: cond_prefill_ane, flowlm_step_ane and mimi_decoder with identical weights, and all four giving the same outputs, Mimi's within 2e-4 for the reason below.

These edits to the recipe were needed, and are the only ones:

• _language_arg.py: dutch, french and english_2026-10 added to the languages it accepts. It already takes spanish, portuguese, german and italian.
• traceable_mimi_decoder.py: 3 changes for pocket-tts 4, which the newer packs need. Its attention state's new pad (0 at batch 1) is left out, the quantized latent skips the quantizer decode_from_latent now applies itself, and the attention takes the attn_mask argument the layers now pass.
• convert_mimi_decoder.py: its renaming of Mimi's outputs is skipped, so they keep the traced names FluidInference's published packs have.
• mlmodelc_all.sh: the three _ane and fused models added to what it compiles.

Two steps are not in the recipe. bos_before_voice.bin is flow_lm.bos_before_voice written as float32, and Mimi's initial state is the recipe's own TraceableMimiDecoder state, with the NaN cache fill as 0. Both match FluidInference's English pack byte for byte.

Kyutai's pocket-tts has used the tanh approximation of GELU since 3.1.0, and every model here came after it. The recipe's transformer models keep exact GELU, as FluidInference's packs do, while Mimi follows the package. The difference is about 1e-3 in the transformer's output, against 2e-2 to 3e-2 from float16, and 2e-4 in Mimi's audio.

The commands

From a checkout of mobius and a scratch directory:

git clone --depth 1 https://github.com/FluidInference/mobius.git mobius
git -C mobius fetch --depth 1 origin "refs/heads/feat/pocket-tts-flowlm-ane:refs/remotes/origin/feat/pocket-tts-flowlm-ane"
git -C mobius worktree add --detach ../mobius-ane origin/feat/pocket-tts-flowlm-ane
uv venv env --python python3.12
uv pip install --python env/bin/python torch==2.9.1 coremltools==9.0 pocket-tts==4.0.1 \
  numpy==2.4.1 safetensors==0.7.0 sentencepiece==0.2.1 scipy==1.17.0 einops==0.8.1 \
  huggingface-hub==1.3.1 pydantic==2.12.5 beartype==0.22.9 pyyaml

Then, for each of dutch, french, english_2026-10, spanish, portuguese, german and italian, in mobius-ane/models/tts/pocket_tts/coreml, with the model loaded from a local copy of pocket-tts's config for the language whose weights_path is the fetched file:

python convert_models/convert/convert_cond_prefill_ane.py --language L --t-max 256
python convert_models/convert/convert_flowlm_step_ane.py --language L
python convert_models/convert/convert_flow_decoder_fused.py --language L --num-steps 8
MIMI_SEMANTIC_NAMES=0 python convert_models/convert/convert_mimi_decoder.py --language L --compute-precision fp32
python convert_assets/export_constants.py --language L
python convert_assets/pack_constants_bin.py --language L
DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer LANGUAGES=L ./mlmodelc_all.sh

Each pack is then build/L/'s four .mlmodelc (with metadata.json for Mimi only, as FluidInference's packs have it) and its constants_bin/ files for the voices listed.

Checked

The recipe's own check speaks its 31 to 37 word sample text through each pack and has Whisper large-v3 hear it. Through these models, at the recipe's temperature of 0.7 and at voice's 0.3:

Pack	Voices	Words wrong, of 31 to 37, over 4 runs
dutch	alba, daan	1
french	alba, estelle	0
english_2026-10	alba, charles	0
spanish	alba, lola	0
portuguese	alba, charles	8
german	alba, juergen	4
italian	alba, giovanni	3
FluidInference's v2.1/english	alba, charles	0
FluidInference's v2.1/spanish	lola, charles	all 144
FluidInference's v2.1/portuguese	charles, 2 runs	all 74
FluidInference's v2.1/german	juergen, 2 runs	0
FluidInference's v2.1/italian	giovanni, charles	0

The recipe has no Dutch sample text, so the Dutch one says what the others do. Through Core ML, FluidInference's April Spanish and Portuguese turn to NaN partway through the text and never stop, so nothing of them is heard.

Voices

Voice	Source	License
alba	Alba MacKenna	CC BY 4.0
anna, vera, fantine, charles, paul, eponine, azelma, george, mary, jane, michael, eve	VCTK p228, p229, p244, p254, p259, p262, p303, p315, p333, p339, p360, p361	CC BY 4.0
marius, javert	Kyutai voice donations	CC0
estelle	Unmute	CC0
lola, giovanni	Common Voice	CC0
juergen	Kyutai, source not stated	CC BY 4.0 (Kyutai's)
daan	CML-TTS, Dutch	CC BY 4.0
emma	Kokoro-82M's bf_emma (hexgrad), synthetic speech, cloned with Kyutai's model	Apache-2.0 (Kokoro); the state made with Kyutai's CC BY 4.0 model
heart, bella, nicole	Kokoro-82M's af_heart, af_bella and af_nicole (hexgrad), English, synthetic speech, cloned with Kyutai's model	Apache-2.0 (Kokoro); the states made with Kyutai's CC BY 4.0 model
siwis	Kokoro-82M's ff_siwis (hexgrad), French, synthetic speech of a voice Kokoro trained on the SIWIS French Speech Synthesis Database, cloned with Kyutai's model	Apache-2.0 (Kokoro); SIWIS is CC BY 4.0 (Yamagishi, Honnet, Garner and Lazaridis, University of Edinburgh); the state made with Kyutai's CC BY 4.0 model
alex, dora, sara	Kokoro-82M's em_alex (Spanish), pf_dora (Portuguese) and if_sara (Italian) (hexgrad), synthetic speech, cloned with Kyutai's model	Apache-2.0 (Kokoro); the states made with Kyutai's CC BY 4.0 model

Each voice but the 8 from Kokoro-82M is Kyutai's snapshot for that language, from languages/<pack>/embeddings/. Kyutai's repository carries the CML-TTS sample cml_tts_nl_11290_9412_000030_0001.flac at its root.

The 8 voices from Kokoro-82M are clones of its bf_emma, af_heart, af_bella, af_nicole, ff_siwis, em_alex, pf_dora and if_sara, made here the way pocket_tts's get_state_for_audio_prompt makes a state, from a prompt of Kokoro's own output in the voice's language (its sentences 0.4 s apart) through Kyutai's voice-cloning model, the gated kyutai/pocket-tts at the revision each pack's config in pocket_tts names (db04f0e6 for english_2026-10, 3e82814a for the rest). Those gated files differ from the open ones only in Mimi's 42 encoder tensors, checked tensor by tensor before the clone, so their states are states of the model these packs are. emma's prompt is 20 s and the others' 10 s, cut at a pause: from a 20 s prompt Kyutai's Spanish model stops at a semicolon, and its Portuguese one at times, where from 10 s, about the length of Kyutai's own voices, it does not, so emma's spanish and portuguese states are made from the first 10 s of her prompt. Her other states are 249 positions; the rest are 103 to 126, Kyutai's voices' about 126. Kokoro's weights and voicepacks are Apache-2.0; its card names no real speaker behind any of them but ff_siwis, whose training data it gives as the SIWIS database, under CC BY 4.0. The gated weights themselves are not here.

License

Pocket TTS is Kyutai's, under CC BY 4.0, and so are these conversions of it: see kyutai/pocket-tts. Kyutai's model card sets these terms:

Prohibited use: Use of our model must comply with all applicable laws and regulations and must not result in, involve, or facilitate any illegal, harmful, deceptive, fraudulent, or unauthorized activity. Prohibited uses include, without limitation, voice impersonation or cloning without explicit and lawful consent; misinformation, disinformation, or deception (including fake news, fraudulent calls, or presenting generated content as genuine recordings of real people or events); and the generation of unlawful, harmful, libelous, abusive, harassing, discriminatory, hateful, or privacy-invasive content. We disclaim all liability for any non-compliant use.

The conversion recipe is FluidInference's mobius, Apache-2.0. The voices are under the licenses above.