Card: FluidAudio usage via AsrModelVersion.bark3 / .bark3Clean (FluidAudio #1054)
Browse files
README.md
CHANGED
|
@@ -75,7 +75,8 @@ corpus WER (total edits / total reference words, lowercase, punctuation stripped
|
|
| 75 |
| Bark 3.0 Clean, upstream MLX (`bark_mlx.py`, fp16), same files and scorer | 2.39 % | — | 55× / — |
|
| 76 |
| Parakeet TDT v3 (FluidInference, reference) | 2.27 % | 4.12 % | 150× / 140× |
|
| 77 |
|
| 78 |
-
Long-form: a one-hour Earnings-22 recording (four concatenated calls) transcribes in **
|
|
|
|
| 79 |
|
| 80 |
¹ Measured with a dense 6-bit-palette build of the same exact encoder weights; the shipped sparse `Encoder.mlmodelc`
|
| 81 |
matched it on test-other (4.66 % both, 143× vs 142×).
|
|
@@ -86,22 +87,19 @@ Notes:
|
|
| 86 |
- Bark 3.0 is evaluated upstream with 0.25 s of leading silence. Through FluidAudio's sliding-window pipeline it makes
|
| 87 |
no difference (test-other 5.02 % with it vs 4.96 % without), so none is added.
|
| 88 |
- Bark models emit `<unk>` (token 0) where their training text had a quotation mark. Upstream decoders skip special
|
| 89 |
-
tokens
|
| 90 |
-
|
|
|
|
| 91 |
|
| 92 |
## Usage (Swift, FluidAudio)
|
| 93 |
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
```bash
|
| 97 |
-
hf download FluidInference/bark-3.0-coreml --include "bark-3.0/*" --local-dir bark
|
| 98 |
-
```
|
| 99 |
|
| 100 |
```swift
|
| 101 |
import FluidAudio
|
| 102 |
|
| 103 |
-
let
|
| 104 |
-
let models = try AsrModels.loadLocal(from: dir, version: .ultra) // same contract as Parakeet TDT v3
|
| 105 |
let asr = AsrManager(config: .default)
|
| 106 |
try await asr.loadModels(models)
|
| 107 |
var state = TdtDecoderState.make(decoderLayers: await asr.decoderLayerCount)
|
|
@@ -109,6 +107,20 @@ let result = try await asr.transcribe(URL(fileURLWithPath: "meeting.wav"), decod
|
|
| 109 |
print(result.text)
|
| 110 |
```
|
| 111 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 112 |
The first load compiles the encoder for the Neural Engine (up to a minute); later loads are cached by Core ML.
|
| 113 |
Requires iOS 18 / macOS 15 or later. Not yet tested on iPhone.
|
| 114 |
|
|
@@ -125,4 +137,4 @@ Requires iOS 18 / macOS 15 or later. Not yet tested on iPhone.
|
|
| 125 |
|
| 126 |
| Date | Files | Change | Issue | mobius PR | FluidAudio PR | HF commit |
|
| 127 |
|---|---|---|---|---|---|---|
|
| 128 |
-
| 2026-10-21 | `bark-3.0/*`, `bark-3.0-clean/*` | Initial release | — | — |
|
|
|
|
| 75 |
| Bark 3.0 Clean, upstream MLX (`bark_mlx.py`, fp16), same files and scorer | 2.39 % | — | 55× / — |
|
| 76 |
| Parakeet TDT v3 (FluidInference, reference) | 2.27 % | 4.12 % | 150× / 140× |
|
| 77 |
|
| 78 |
+
Long-form: a one-hour Earnings-22 recording (four concatenated calls) transcribes in **6.5 s (558×)** with Bark 3.0 and
|
| 79 |
+
7.0 s (514×) with Clean, models warm; Clean writes 7,787 words to Bark 3.0's 8,223 (fillers and repeats removed).
|
| 80 |
|
| 81 |
¹ Measured with a dense 6-bit-palette build of the same exact encoder weights; the shipped sparse `Encoder.mlmodelc`
|
| 82 |
matched it on test-other (4.66 % both, 143× vs 142×).
|
|
|
|
| 87 |
- Bark 3.0 is evaluated upstream with 0.25 s of leading silence. Through FluidAudio's sliding-window pipeline it makes
|
| 88 |
no difference (test-other 5.02 % with it vs 4.96 % without), so none is added.
|
| 89 |
- Bark models emit `<unk>` (token 0) where their training text had a quotation mark. Upstream decoders skip special
|
| 90 |
+
tokens; FluidAudio drops it from the transcript as of [#1054](https://github.com/FluidInference/FluidAudio/pull/1054).
|
| 91 |
+
On older releases strip it from `result.text` (`replacingOccurrences(of: "<unk>", with: "")`); the WER numbers above
|
| 92 |
+
are scored without it.
|
| 93 |
|
| 94 |
## Usage (Swift, FluidAudio)
|
| 95 |
|
| 96 |
+
With [FluidAudio](https://github.com/FluidInference/FluidAudio) ([#1054](https://github.com/FluidInference/FluidAudio/pull/1054)
|
| 97 |
+
adds `AsrModelVersion.bark3` / `.bark3Clean`, which download the right folder):
|
|
|
|
|
|
|
|
|
|
| 98 |
|
| 99 |
```swift
|
| 100 |
import FluidAudio
|
| 101 |
|
| 102 |
+
let models = try await AsrModels.downloadAndLoad(version: .bark3) // or .bark3Clean
|
|
|
|
| 103 |
let asr = AsrManager(config: .default)
|
| 104 |
try await asr.loadModels(models)
|
| 105 |
var state = TdtDecoderState.make(decoderLayers: await asr.decoderLayerCount)
|
|
|
|
| 107 |
print(result.text)
|
| 108 |
```
|
| 109 |
|
| 110 |
+
```bash
|
| 111 |
+
swift run -c release fluidaudiocli transcribe meeting.wav --model-version bark3 # or bark3-clean
|
| 112 |
+
```
|
| 113 |
+
|
| 114 |
+
On a FluidAudio release without #1054, download one folder and load it with the v3 layout:
|
| 115 |
+
|
| 116 |
+
```bash
|
| 117 |
+
hf download FluidInference/bark-3.0-coreml --include "bark-3.0/*" --local-dir bark
|
| 118 |
+
```
|
| 119 |
+
|
| 120 |
+
```swift
|
| 121 |
+
let models = try AsrModels.loadLocal(from: URL(fileURLWithPath: "bark/bark-3.0"), version: .ultra)
|
| 122 |
+
```
|
| 123 |
+
|
| 124 |
The first load compiles the encoder for the Neural Engine (up to a minute); later loads are cached by Core ML.
|
| 125 |
Requires iOS 18 / macOS 15 or later. Not yet tested on iPhone.
|
| 126 |
|
|
|
|
| 137 |
|
| 138 |
| Date | Files | Change | Issue | mobius PR | FluidAudio PR | HF commit |
|
| 139 |
|---|---|---|---|---|---|---|
|
| 140 |
+
| 2026-10-21 | `bark-3.0/*`, `bark-3.0-clean/*` | Initial release | — | — | [#1054](https://github.com/FluidInference/FluidAudio/pull/1054) | [`ca21bed2`](https://huggingface.co/FluidInference/bark-3.0-coreml/commit/ca21bed2ccd9c3fbcd104a6014b9a5638a764bc8) |
|