File size: 1,874 Bytes
c3ac013 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | # Validation
Validation was run on 2026-08-01 with an Apple M5 Pro, 24 GB unified memory,
macOS 26.6, Python 3.12.13, and MLX 0.32.0.
## Checkpoint conversion
- Source: `AutoArk-AI/ARK-ASR-3B`
- Revision: `1e28271b79edc97635783bea65abc89195a09ed3`
- Source tensors: 926
- Intentionally dropped tensors: 2
- Retained parameter tensors: 924
- Missing, unexpected, or shape-mismatched tensors: 0
- Output dtype: BF16
- Output size: 7.0 GiB
- Output SHA-256:
`a5e9431bdd648340a40c092e385c4fe1d445d8ad3311dd94609e72af36b0256d`
Source shard hashes are recorded in `conversion.json`.
## PyTorch parity
The original remote-code model and native MLX model were run on the pinned
`Narsil/asr_dummy/1.flac` LibriSpeech sample.
- Prompt token IDs: identical
- BF16 input features: maximum absolute difference `0.0`
- Adapted audio feature cosine similarity: `0.998567558665821`
- Initial decoder logits cosine similarity: `0.9999223476845636`
- Greedy generation token IDs: identical
- Final decoded text: identical
Both implementations produced:
> he hoped there would be stew for dinner turnips and carrots and bruised
> potatoes and fat mutton pieces to be ladled out in thick peppered flour
> fattened sauce
The validation command was:
```bash
python scripts/validate_parity.py /path/to/1.flac \
--model . \
--source AutoArk-AI/ARK-ASR-3B \
--revision 1e28271b79edc97635783bea65abc89195a09ed3 \
--min-adapter-cosine 0.998
```
## MLX benchmark
The same 12.1-second audio sample was measured after the checkpoint was already
present in the operating-system file cache:
- Model load: `1.011 s`
- Audio preprocessing: `0.499 s`
- First token: `0.885 s`
- Full 34-token generation: `1.965 s`
- Generation throughput: `17.30 tokens/s`
- MLX peak memory: `7.820 GB`
These numbers describe this machine and test clip; they are not portable
performance guarantees.
|