Card: document the experimental ane/ build (FluidAudio #49)

f2679c4 verified 5 days ago

7.7 kB

---
license: other
license_name: lfm1.0
license_link: LICENSE
base_model: LiquidAI/LFM3-Audio-2B
library_name: fluidaudio
pipeline_tag: automatic-speech-recognition
tags:
  - coreml
  - apple-silicon
  - asr
  - tts
  - text-to-speech
  - streaming
  - on-device
---
# LFM3-Audio-2B for Core ML

[LiquidAI/LFM3-Audio-2B](https://huggingface.co/LiquidAI/LFM3-Audio-2B) converted to Core ML for Apple devices
(macOS 15 / iOS 18 and newer): offline speech recognition (long-form included), LiveASR (streaming English
transcription in 0.96 s chunks) and text-to-speech, all from one 2B audio language model. Run it with
[FluidAudio](https://github.com/FluidInference/FluidAudio) (`LFM3AudioManager`).

This is a modified version of the original model: the weights were converted to Core ML and the language model was
quantized to int8 (block size 32). See [Quantization and parity](#quantization-and-parity).

Core AI builds of the language model and the depth decoder (macOS 27 / iOS 27; faster ASR and TTS) are in
[FluidInference/lfm3-audio-2b-coreai](https://huggingface.co/FluidInference/lfm3-audio-2b-coreai); that backend uses the
encoder, codec and tables from this repo.

## Files

| Part | File | Precision | Compute unit (default) |
|---|---|---|---|
| LM backbone, layers 0-11 / 12-23 (stateful; functions `decode_q1`, `prefill_q256/64/16/13`) | `coreml/LFM3-A-C4160-int8b32.mlmodelc`, `coreml/LFM3-B-C4160-int8b32.mlmodelc` | int8 weights (block 32), fp16 compute | GPU |
| LM head (128k vocab, per-chunk top-64) | `coreml/LFM3-Head-int8b32.mlmodelc` | int8 | GPU |
| Causal audio encoder (functions `n12` / `n96` / `n375`, tokens per call) | `coreml/AudioEncoder-fp16.mlmodelc` | fp16 | GPU (offline), ANE (LiveASR `n12`) |
| MaskGIT depth decoder (`depth_in`, `depth_pass`) | `coreml/DepthDecoder-fp16.mlmodelc` | fp16 (+ host fp32 sampling) | GPU |
| LiquidCodec decoder (`f1` / `f4` / `f25`, frames per call) | `coreml/CodecDecoder-fp16.mlmodelc` | fp16 | ANE |
| Pack contract, tokenizer | `assets/config.json`, `assets/tokenizer.json` | | |
| Host tables: token embeddings, speech-frame embeddings, depth code embeddings, codec synthesis head | `assets/embed_tokens.f16.bin`, `assets/codebook_embedding.f16.bin`, `assets/depth_tables.f16.bin`, `assets/codec_head.f32.bin` | fp16 / fp32 | CPU |

ASR and LiveASR need about 2.5 GB of files; TTS adds the depth decoder, codec and tables (~0.4 GB). Context: 4096
positions. `ane/` holds an experimental Neural Engine build of the LM that FluidAudio does not download; see
[Experimental: Neural Engine LM](#experimental-neural-engine-lm-ane).

## Use (Swift)

```swift
import FluidAudio
let manager = try await LFM3AudioManager.downloadAndLoad()  // .coreML; includeSpeech: true
// Offline ASR (16 kHz mono); language: "English", "French", ... or nil to infer
let result = try await manager.transcribe(samples)
print(result.text)
// LiveASR (English)
try await manager.startLiveTranscription()
let partial = try await manager.appendLiveAudio(microphoneChunk)
let final = try await manager.finishLiveTranscription()
// TTS (24 kHz), streamed as it is decoded
let speech = try await manager.synthesize(
    "Hello from LFM3 Audio.", voice: .female, language: "American English",
    onAudio: { chunk in /* play */ })
```

CLI: `fluidaudiocli lfm3-audio transcribe|live|tts|benchmark`.

## Speed and accuracy

Apple M5 Pro (24 GB), macOS 27.0, FluidAudio release build, Core ML backend. Wall time includes model warm-up (the
first calls after load), audio decoding excluded.

| Task | Data | Result |
|---|---|---|
| Offline ASR | LibriSpeech test-clean, first 100 utterances (901 s) | WER 1.32%, RTFx 18.4; transcripts 100/100 identical to the original model (HF fp32 on MPS, greedy) |
| LiveASR (160 ms pieces) | LibriSpeech test-clean, first 20 utterances (157 s) | WER 1.46%, RTFx 10.4 |
| TTS | 20 phrases (72 s of speech, female voice) | RTFx 4.3-4.5 across runs, first audio after 37 ms (median); Parakeet TDT v3 round-trip WER 2.5% |

Per speech frame (80 ms): LM 9.4 ms + depth decoder 4.9 ms on the GPU; the codec decodes on the ANE in parallel.
Peak memory footprint for ASR: 1.9 GB.

## Quantization and parity

- LM: int8 per-block-32 weights. Teacher-forced TTS hidden states match fp32 at cosine 0.9999; offline ASR transcripts
  are identical to the original model on the 100 utterances above. (An int4 build failed the speech-frame check.)
- Audio encoder, depth decoder and codec: fp16. Depth decoder codes are 99.6% identical to the original on 319 golden
  frames (sampling noise injected); codec waveform SNR 43-49 dB vs fp32.
- Host log-mel matches the original feature extractor to 8.3e-5.

## Experimental: Neural Engine LM (`ane/`)

An all-ANE build of the language model (backbone + head), for testing. FluidAudio does not download or load it yet.
On the M5 Pro it is slower than the GPU pack above. It is here for iPhone-class testing, where the ANE should match
GPU latency at lower power; that has not been measured yet.

| Part | File | Weights | ANE placement | ms per call (M5 Pro) |
|---|---|---|---|---|
| Layers 0-11 | `ane/LFM3-L0-12-q1-S1024-lut8g8.mlmodelc` | LUT8, group 8 | 100% | 5.77 |
| Layers 12-23 + final norm | `ane/LFM3-L12-24-q1-S1024-lut8g8.mlmodelc` | LUT8, group 8 | 100% | 6.01 |
| Head (4 x 32,000 logits) | `ane/LFM3-Head-lut8g8.mlmodelc` | LUT8, group 8 | 100% | 2.08 |

- **Speed:** one text token takes 14.43 ms (GPU pack ~10.8 ms), and one speech frame (backbone only) 12.03 ms
  (9.07 ms).
- **Calls:** decode only, one row per call, positions up to 1024. Input `x` is fp16 [1, 1, 2048], and the output `h`
  feeds the next part. The host supplies the RoPE tables (`cos_g` / `sin_g`, `cos_s` / `sin_s`) and the additive
  masks (`mask_g`, `mask_s`).
- **State:** the K/V caches and conv states are MLState. Full-attention layers write at the position given by the
  `onehot` input. `ane/config.json` lists each part's inputs and states.
- **Head:** returns raw logits in four 32,000-row slices. Argmax, top-k and token suppression run on the host.
- **Parity vs fp32,** measured with the same driver for both builds (GPU pack in parentheses):
  - ASR: 19/20 golden clips token-identical (20/20)
  - TTS teacher-forced hidden cosine: min 0.99969 (0.99983)
  - Depth codes: 97.15% identical (97.39%)
- **ANE numerics:**
  - Weight rows are pre-scaled by a power of two, and the scale is undone after each linear. This avoids fp16 underflow.
  - Sigmoid is computed as 0.5 + 0.5·tanh(x/2).
- **Loading:** the first load compiles for the ANE in about 5 s per part; cached loads take 35-60 ms.

## License

[LFM Open License v1.0](LICENSE) (Liquid AI). Commercial use by organizations with annual revenue of USD 10 million or
more requires a separate license from Liquid AI; see the license for the full terms.

## Changelog

| Date | Files | Change | Issue | FluidAudio PR | HF commit |
|---|---|---|---|---|---|
| 2026-10-29 | all | Initial release: Core ML LM pack (int8 b32), audio encoder, depth decoder (per-pass functions), codec, host tables | [FluidAudio #49](https://github.com/FluidInference/FluidAudio/issues/49#issuecomment-6501658263) | [#1076](https://github.com/FluidInference/FluidAudio/pull/1076) | [21447f69](https://huggingface.co/FluidInference/lfm3-audio-2b-coreml/commit/21447f69cc14197e302d755ecc1aaeb416e24e55) |
| 2026-10-30 | `ane/*` | Experimental all-ANE LM (LUT8 group 8, decode q=1, span 1024); not loaded by FluidAudio | [FluidAudio #49](https://github.com/FluidInference/FluidAudio/issues/49#issuecomment-6501658263) | none | [e5259314](https://huggingface.co/FluidInference/lfm3-audio-2b-coreml/commit/e5259314f4efec09e4fc1011ac92795c26bceb1f) |

	---
	license: other
	license_name: lfm1.0
	license_link: LICENSE
	base_model: LiquidAI/LFM3-Audio-2B
	library_name: fluidaudio
	pipeline_tag: automatic-speech-recognition
	tags:
	- coreml
	- apple-silicon
	- asr
	- tts
	- text-to-speech
	- streaming
	- on-device
	---

	# LFM3-Audio-2B for Core ML

	[LiquidAI/LFM3-Audio-2B](https://huggingface.co/LiquidAI/LFM3-Audio-2B) converted to Core ML for Apple devices
	(macOS 15 / iOS 18 and newer): offline speech recognition (long-form included), LiveASR (streaming English
	transcription in 0.96 s chunks) and text-to-speech, all from one 2B audio language model. Run it with
	[FluidAudio](https://github.com/FluidInference/FluidAudio) (`LFM3AudioManager`).

	This is a modified version of the original model: the weights were converted to Core ML and the language model was
	quantized to int8 (block size 32). See [Quantization and parity](#quantization-and-parity).

	Core AI builds of the language model and the depth decoder (macOS 27 / iOS 27; faster ASR and TTS) are in
	[FluidInference/lfm3-audio-2b-coreai](https://huggingface.co/FluidInference/lfm3-audio-2b-coreai); that backend uses the
	encoder, codec and tables from this repo.

	## Files

	\| Part \| File \| Precision \| Compute unit (default) \|
	\|---\|---\|---\|---\|
	\| LM backbone, layers 0-11 / 12-23 (stateful; functions `decode_q1`, `prefill_q256/64/16/13`) \| `coreml/LFM3-A-C4160-int8b32.mlmodelc`, `coreml/LFM3-B-C4160-int8b32.mlmodelc` \| int8 weights (block 32), fp16 compute \| GPU \|
	\| LM head (128k vocab, per-chunk top-64) \| `coreml/LFM3-Head-int8b32.mlmodelc` \| int8 \| GPU \|
	\| Causal audio encoder (functions `n12` / `n96` / `n375`, tokens per call) \| `coreml/AudioEncoder-fp16.mlmodelc` \| fp16 \| GPU (offline), ANE (LiveASR `n12`) \|
	\| MaskGIT depth decoder (`depth_in`, `depth_pass`) \| `coreml/DepthDecoder-fp16.mlmodelc` \| fp16 (+ host fp32 sampling) \| GPU \|
	\| LiquidCodec decoder (`f1` / `f4` / `f25`, frames per call) \| `coreml/CodecDecoder-fp16.mlmodelc` \| fp16 \| ANE \|
	\| Pack contract, tokenizer \| `assets/config.json`, `assets/tokenizer.json` \| \| \|
	\| Host tables: token embeddings, speech-frame embeddings, depth code embeddings, codec synthesis head \| `assets/embed_tokens.f16.bin`, `assets/codebook_embedding.f16.bin`, `assets/depth_tables.f16.bin`, `assets/codec_head.f32.bin` \| fp16 / fp32 \| CPU \|

	ASR and LiveASR need about 2.5 GB of files; TTS adds the depth decoder, codec and tables (~0.4 GB). Context: 4096
	positions. `ane/` holds an experimental Neural Engine build of the LM that FluidAudio does not download; see
	[Experimental: Neural Engine LM](#experimental-neural-engine-lm-ane).

	## Use (Swift)

	```swift
	import FluidAudio

	let manager = try await LFM3AudioManager.downloadAndLoad() // .coreML; includeSpeech: true

	// Offline ASR (16 kHz mono); language: "English", "French", ... or nil to infer
	let result = try await manager.transcribe(samples)
	print(result.text)

	// LiveASR (English)
	try await manager.startLiveTranscription()
	let partial = try await manager.appendLiveAudio(microphoneChunk)
	let final = try await manager.finishLiveTranscription()

	// TTS (24 kHz), streamed as it is decoded
	let speech = try await manager.synthesize(
	"Hello from LFM3 Audio.", voice: .female, language: "American English",
	onAudio: { chunk in /* play */ })
	```

	CLI: `fluidaudiocli lfm3-audio transcribe\|live\|tts\|benchmark`.

	## Speed and accuracy

	Apple M5 Pro (24 GB), macOS 27.0, FluidAudio release build, Core ML backend. Wall time includes model warm-up (the
	first calls after load), audio decoding excluded.

	\| Task \| Data \| Result \|
	\|---\|---\|---\|
	\| Offline ASR \| LibriSpeech test-clean, first 100 utterances (901 s) \| WER 1.32%, RTFx 18.4; transcripts 100/100 identical to the original model (HF fp32 on MPS, greedy) \|
	\| LiveASR (160 ms pieces) \| LibriSpeech test-clean, first 20 utterances (157 s) \| WER 1.46%, RTFx 10.4 \|
	\| TTS \| 20 phrases (72 s of speech, female voice) \| RTFx 4.3-4.5 across runs, first audio after 37 ms (median); Parakeet TDT v3 round-trip WER 2.5% \|

	Per speech frame (80 ms): LM 9.4 ms + depth decoder 4.9 ms on the GPU; the codec decodes on the ANE in parallel.
	Peak memory footprint for ASR: 1.9 GB.

	## Quantization and parity

	- LM: int8 per-block-32 weights. Teacher-forced TTS hidden states match fp32 at cosine 0.9999; offline ASR transcripts
	are identical to the original model on the 100 utterances above. (An int4 build failed the speech-frame check.)
	- Audio encoder, depth decoder and codec: fp16. Depth decoder codes are 99.6% identical to the original on 319 golden
	frames (sampling noise injected); codec waveform SNR 43-49 dB vs fp32.
	- Host log-mel matches the original feature extractor to 8.3e-5.

	## Experimental: Neural Engine LM (`ane/`)

	An all-ANE build of the language model (backbone + head), for testing. FluidAudio does not download or load it yet.
	On the M5 Pro it is slower than the GPU pack above. It is here for iPhone-class testing, where the ANE should match
	GPU latency at lower power; that has not been measured yet.

	\| Part \| File \| Weights \| ANE placement \| ms per call (M5 Pro) \|
	\|---\|---\|---\|---\|---\|
	\| Layers 0-11 \| `ane/LFM3-L0-12-q1-S1024-lut8g8.mlmodelc` \| LUT8, group 8 \| 100% \| 5.77 \|
	\| Layers 12-23 + final norm \| `ane/LFM3-L12-24-q1-S1024-lut8g8.mlmodelc` \| LUT8, group 8 \| 100% \| 6.01 \|
	\| Head (4 x 32,000 logits) \| `ane/LFM3-Head-lut8g8.mlmodelc` \| LUT8, group 8 \| 100% \| 2.08 \|

	- Speed: one text token takes 14.43 ms (GPU pack ~10.8 ms), and one speech frame (backbone only) 12.03 ms
	(9.07 ms).
	- Calls: decode only, one row per call, positions up to 1024. Input `x` is fp16 [1, 1, 2048], and the output `h`
	feeds the next part. The host supplies the RoPE tables (`cos_g` / `sin_g`, `cos_s` / `sin_s`) and the additive
	masks (`mask_g`, `mask_s`).
	- State: the K/V caches and conv states are MLState. Full-attention layers write at the position given by the
	`onehot` input. `ane/config.json` lists each part's inputs and states.
	- Head: returns raw logits in four 32,000-row slices. Argmax, top-k and token suppression run on the host.
	- Parity vs fp32, measured with the same driver for both builds (GPU pack in parentheses):
	- ASR: 19/20 golden clips token-identical (20/20)
	- TTS teacher-forced hidden cosine: min 0.99969 (0.99983)
	- Depth codes: 97.15% identical (97.39%)
	- ANE numerics:
	- Weight rows are pre-scaled by a power of two, and the scale is undone after each linear. This avoids fp16 underflow.
	- Sigmoid is computed as 0.5 + 0.5·tanh(x/2).
	- Loading: the first load compiles for the ANE in about 5 s per part; cached loads take 35-60 ms.

	## License

	[LFM Open License v1.0](LICENSE) (Liquid AI). Commercial use by organizations with annual revenue of USD 10 million or
	more requires a separate license from Liquid AI; see the license for the full terms.

	## Changelog

	\| Date \| Files \| Change \| Issue \| FluidAudio PR \| HF commit \|
	\|---\|---\|---\|---\|---\|---\|
	\| 2026-10-29 \| all \| Initial release: Core ML LM pack (int8 b32), audio encoder, depth decoder (per-pass functions), codec, host tables \| [FluidAudio #49](https://github.com/FluidInference/FluidAudio/issues/49#issuecomment-6501658263) \| [#1076](https://github.com/FluidInference/FluidAudio/pull/1076) \| [21447f69](https://huggingface.co/FluidInference/lfm3-audio-2b-coreml/commit/21447f69cc14197e302d755ecc1aaeb416e24e55) \|
	\| 2026-10-30 \| `ane/*` \| Experimental all-ANE LM (LUT8 group 8, decode q=1, span 1024); not loaded by FluidAudio \| [FluidAudio #49](https://github.com/FluidInference/FluidAudio/issues/49#issuecomment-6501658263) \| none \| [e5259314](https://huggingface.co/FluidInference/lfm3-audio-2b-coreml/commit/e5259314f4efec09e4fc1011ac92795c26bceb1f) \|