| # Validation |
|
|
| Validation was run on 2026-07-31 with an Apple M5 Pro, 24 GB unified memory, |
| macOS 26.6, Python 3.12.13, and MLX 0.32.0. |
|
|
| ## Checkpoint conversion |
|
|
| - Source: `Audio8/ARK-ASR-0.6B` |
| - Revision: `45776b56d58cdfb2e2eb632f7e110f38684633e0` |
| - Retained parameter tensors: 780 |
| - Missing or unexpected tensors: 0 |
| - Output dtype: BF16 |
| - Output size: 2.1 GiB |
| - Source SHA-256: |
| `57a86ce1c2f2c2d6ebb7ad9642c9e951a5109625122b48c9180126a28787673d` |
| - Output SHA-256: |
| `4f17d6fda7be51e489c1819c266ed946b2d3982368f0a231ceb0c1f211a3e56c` |
|
|
| ## PyTorch parity |
|
|
| The original remote-code model and the native MLX model were run on the pinned |
| `Narsil/asr_dummy/1.flac` LibriSpeech sample. |
|
|
| - Prompt token IDs: identical |
| - BF16 input features: maximum absolute difference `0.0` |
| - Adapted audio features cosine similarity: `0.9998097773` |
| - Initial decoder logits cosine similarity: `0.9999750792` |
| - Greedy generation token IDs: identical |
| - Final decoded text: identical |
|
|
| Both implementations produced: |
|
|
| > He hoped there would be stew for dinner, turnips and carrots, and bruised |
| > potatoes and fat mutton pieces to be ladled out in thick peppered |
| > flour-fattened sauce. |
|
|
| ## MLX benchmark |
|
|
| The same 12.1-second audio sample was measured after the checkpoint was already |
| present in the operating-system file cache: |
|
|
| - Model load: `0.362 s` |
| - Audio preprocessing: `0.396 s` |
| - First token: `0.110 s` |
| - Full 38-token generation: `0.384 s` |
| - Generation throughput: `98.94 tokens/s` |
| - MLX peak memory: `2.620 GB` |
|
|
| These numbers describe this machine and test clip; they are not portable |
| performance guarantees. |
|
|
