| --- |
| license: cc-by-4.0 |
| language: |
| - en |
| library_name: coreml |
| pipeline_tag: automatic-speech-recognition |
| base_model: |
| - ConwayResearch/Underdog-Bark-3.0 |
| - ConwayResearch/Underdog-Bark-3.0-Clean |
| tags: |
| - coreml |
| - apple-neural-engine |
| - speech-to-text |
| - asr |
| - parakeet |
| - underdog |
| - fluidaudio |
| - on-device |
| --- |
| |
| # Underdog Bark 3.0 · Core ML |
|
|
| Full Core ML pipelines (preprocessor, encoder, decoder, joint) for |
| [Underdog Bark 3.0](https://huggingface.co/ConwayResearch/Underdog-Bark-3.0) and |
| [Underdog Bark 3.0 Clean](https://huggingface.co/ConwayResearch/Underdog-Bark-3.0-Clean) by Conway Research, laid |
| out in the [FluidAudio](https://github.com/FluidInference/FluidAudio) Parakeet TDT v3 component layout so they run |
| end to end in Swift with the encoder on the Apple Neural Engine. |
|
|
| Underdog already ships a Core ML encoder for Bark 3.0 (with the preprocessor and decoder in Python), and Bark 3.0 Pro |
| as a full Core ML pipeline. This repo adds the rest of the pipeline for the small models, plus Bark 3.0 Clean, which |
| had no Core ML build. |
|
|
| | Folder | Model | Output | |
| |---|---|---| |
| | [`bark-3.0/`](bark-3.0) | Underdog Bark 3.0 | Verbatim: keeps "um", "uh", repeats | |
| | [`bark-3.0-clean/`](bark-3.0-clean) | Underdog Bark 3.0 Clean | Cleaned: drops fillers, false starts and repeated words | |
|
|
| English only. Tokenizer and output conventions (punctuation, casing, numerals) are Parakeet TDT v3's. |
|
|
| ## Files (each folder) |
|
|
| | File | What it is | |
| |---|---| |
| | `Encoder.mlmodelc` | FastConformer encoder, 15 s window, **Neural Engine default** (sparse mask + 6-bit palette per 8 rows; 399 MB Bark 3.0, 410 MB Clean) | |
| | `Encoder_lut3.mlmodelc` | Same weights as a dense 3-bit palette per row (253 MB): **use this one on the GPU** (`.cpuAndGPU`) | |
| | `Decoder.mlmodelc` | 2-layer LSTM prediction network | |
| | `JointDecisionv3.mlmodelc` | TDT joint (token + duration), v3 interface | |
| | `Preprocessor.mlmodelc` | Parakeet TDT v3 mel front-end (Bark uses the original's) | |
| | `parakeet_vocab.json` | Parakeet TDT v3 vocabulary (8192 tokens) | |
| | `NOTICE` | Upstream attribution | |
|
|
| ### How the encoder is encoded |
|
|
| Bark stores each encoder linear as per-row levels: 260 of the 264 matrices take five values per output row |
| `{0, ±a, ±b}`, and layer 23's four feed-forward matrices take seven. Core ML has no entropy-coded weight format, so |
| the weights are re-encoded as Core ML palettes (`constexpr_lut_to_sparse` + `constexpr_sparse_to_dense`, iOS 18): |
| the palette of every group of 8 output rows holds those rows' values, and each weight stores its index. Nothing is |
| re-quantized: every palettized weight reconstructs **bit-exactly** to the value in the upstream `model.bark` (240 |
| matrices; the 24 positional projections act on the fixed positional table and are folded into constants at export). |
|
|
| Grouping 8 rows per palette matters on the Neural Engine, whose palette cost scales with the number of palettes: |
| per-row palettes run about 3× slower there. The GPU does not care, which is why `Encoder_lut3.mlmodelc` (per-row, |
| smallest dense file) is the GPU build; the sparse `Encoder.mlmodelc` is rebuilt on every GPU load (minutes), so do not |
| use it with `.cpuAndGPU`. |
|
|
| ## Benchmarks |
|
|
| Apple M5 Pro, macOS 27, FluidAudio `asr-benchmark`, encoder on the Neural Engine, **full** LibriSpeech test sets, |
| corpus WER (total edits / total reference words, lowercase, punctuation stripped), RTFx = audio / processing time. |
|
|
| | Model | test-clean WER | test-other WER | RTFx clean / other | |
| |---|--:|--:|--:| |
| | Bark 3.0 (this repo) | — | 4.96 % | — / 140× | |
| | Bark 3.0 Clean (this repo) | 2.37 %¹ | 4.66 % | 155×¹ / 143× | |
| | Bark 3.0 Clean, upstream MLX (`bark_mlx.py`, fp16), same files and scorer | 2.39 % | — | 55× / — | |
| | Parakeet TDT v3 (FluidInference, reference) | 2.27 % | 4.12 % | 150× / 140× | |
|
|
| Long-form: a one-hour Earnings-22 recording (four concatenated calls) transcribes in **6.5 s (558×)** with Bark 3.0 and |
| 7.0 s (514×) with Clean, models warm; Clean writes 7,787 words to Bark 3.0's 8,223 (fillers and repeats removed). |
|
|
| ¹ Measured with a dense 6-bit-palette build of the same exact encoder weights; the shipped sparse `Encoder.mlmodelc` |
| matched it on test-other (4.66 % both, 143× vs 142×). |
|
|
| Notes: |
| - LibriSpeech references are verbatim and Clean removes words on purpose, so read speech is not where Clean shines; |
| see the upstream cards for AMI, DisfluencySpeech and the other Open ASR sets. |
| - Bark 3.0 is evaluated upstream with 0.25 s of leading silence. Through FluidAudio's sliding-window pipeline it makes |
| no difference (test-other 5.02 % with it vs 4.96 % without), so none is added. |
| - Bark models emit `<unk>` (token 0) where their training text had a quotation mark. Upstream decoders skip special |
| tokens; FluidAudio drops it from the transcript as of [#1054](https://github.com/FluidInference/FluidAudio/pull/1054). |
| On older releases strip it from `result.text` (`replacingOccurrences(of: "<unk>", with: "")`); the WER numbers above |
| are scored without it. |
|
|
| ## Usage (Swift, FluidAudio) |
|
|
| With [FluidAudio](https://github.com/FluidInference/FluidAudio) ([#1054](https://github.com/FluidInference/FluidAudio/pull/1054) |
| adds `AsrModelVersion.bark3` / `.bark3Clean`, which download the right folder): |
|
|
| ```swift |
| import FluidAudio |
| |
| let models = try await AsrModels.downloadAndLoad(version: .bark3) // or .bark3Clean |
| let asr = AsrManager(config: .default) |
| try await asr.loadModels(models) |
| var state = TdtDecoderState.make(decoderLayers: await asr.decoderLayerCount) |
| let result = try await asr.transcribe(URL(fileURLWithPath: "meeting.wav"), decoderState: &state) |
| print(result.text) |
| ``` |
|
|
| ```bash |
| swift run -c release fluidaudiocli transcribe meeting.wav --model-version bark3 # or bark3-clean |
| ``` |
|
|
| On a FluidAudio release without #1054, download one folder and load it with the v3 layout: |
|
|
| ```bash |
| hf download FluidInference/bark-3.0-coreml --include "bark-3.0/*" --local-dir bark |
| ``` |
|
|
| ```swift |
| let models = try AsrModels.loadLocal(from: URL(fileURLWithPath: "bark/bark-3.0"), version: .ultra) |
| ``` |
|
|
| The first load compiles the encoder for the Neural Engine (up to a minute); later loads are cached by Core ML. |
| Requires iOS 18 / macOS 15 or later. Not yet tested on iPhone. |
|
|
| ## Credits and license |
|
|
| - **Underdog Bark 3.0 and Bark 3.0 Clean** by Conway Research ([Underdog](https://underdog.ai)), CC-BY-4.0. |
| - Derived from **nvidia/parakeet-tdt-0.6b-v3** (© NVIDIA Corporation, CC-BY-4.0) through |
| **moondream/parakeet-ultra** (© Moondream, CC-BY-4.0). |
| - This repo: Core ML conversion and re-encoding by Fluid Inference. The encoder's linear layers are bit-exact to the |
| upstream weights; the remaining tensors (subsampling, depthwise convs, norms, decoder, joint) are the upstream values |
| converted to Core ML at fp16. `NOTICE` in each folder carries the upstream attribution. License: [CC-BY-4.0](LICENSE). |
|
|
| ## Changelog |
|
|
| | Date | Files | Change | Issue | mobius PR | FluidAudio PR | HF commit | |
| |---|---|---|---|---|---|---| |
| | 2026-10-21 | `bark-3.0/*`, `bark-3.0-clean/*` | Initial release | — | — | [#1054](https://github.com/FluidInference/FluidAudio/pull/1054) | [`ca21bed2`](https://huggingface.co/FluidInference/bark-3.0-coreml/commit/ca21bed2ccd9c3fbcd104a6014b9a5638a764bc8) | |
|
|
