Rewrite model card: YAML metadata, artifact-form definitions, file layout, attribution

4134f13 verified 2 days ago

6.26 kB

license: apache-2.0
base_model: Audio8/ARK-ASR-3B
pipeline_tag: automatic-speech-recognition
language:
  - en
  - zh
  - de
  - ja
  - fr
  - ko
  - es
  - pl
  - it
  - ro
  - hu
  - cs
  - nl
  - fi
  - hr
  - sk
  - sl
  - et
  - lt
tags:
  - coreai
  - aimodel
  - aimodelc
  - arkasr
  - speech
  - audio
  - apple-silicon

ARK-ASR-3B Core AI

Apple Core AI conversions of Audio8/ARK-ASR-3B, made from source revision 1e28271b79edc97635783bea65abc89195a09ed3.

ARK-ASR combines a Whisper-style audio encoder, an MLP adapter, and a Qwen2.5 decoder. These files use Apple's Core AI runtime (macOS 27+); they are not GGUF and are not compatible with llama.cpp, whisper.cpp, or Apple's CoreAISpeech bundle layout. For GGUF runtimes see harshav/ARK-ASR-3B-GGUF.

Artifact forms

Term	Meaning
`.aimodel`	Portable Core AI source asset. Runs on any macOS 27+ device; the runtime specializes it for your chip on first use (one-time, cached).
`.aimodelc`	AOT-compiled variant for one specific silicon architecture, produced by `xcrun coreai-build compile --architecture <arch>`. Skips runtime specialization entirely (measured 74 s → ~0 s of model preparation per app launch). The runtime hard-rejects mismatched chips with `incompatibleCompiledAssetArchitecture`, so only use the variant matching your device (`h16c` = M4 Max family; list yours with `xcrun coreai-build list-architectures`). Other architectures compile from the `.aimodel` in minutes.
`fp16`	Full-precision weights (encoder).
`int8`	Weight-only quantization, per-block-32 symmetric with clipping (decoder). SDPA, RoPE, RMSNorm, embeddings, and the LM head stay in fp16.
`static`	The prefill graph's prompt shape is baked (2 + 375 audio slots + 3 = 380 tokens, the constant 30 s window). A dynamic-shape prefill costs ~3 GB more compiled package and ~4x slower gate load for zero behavior change.

The decoder is one unified bundle: prefill + decode entrypoints share a single weight copy, with the KV cache externalized and threaded host-side.

Runtime

coreai-arkasr-conversion.tar.gz is the complete conversion and verification toolkit used to produce and gate these artifacts (SHA-256: 8a24d7069ec64462703986445af5841541e626235cf03c53ece8d3aefc08d77f), including the port's STATE.md with full reproduction steps:

tar -xzf coreai-arkasr-conversion.tar.gz && cd ark_asr
uv run --python 3.12 make_oracle.py --audio <16k.wav>
uv run --python 3.12 --with-editable <coreai-models>/python export_encoder.py --dtype float16
uv run --python 3.12 --with-editable <coreai-models>/python export_unified.py --mode int8 --cache-len 1024 --static-prefill
uv run --python 3.12 gate_static.py --unified --mode int8 --cache-len 1024

The reference client is VoiceInk (Transcription/CoreAI/): mel frontend, prompt splicing (vocabSize + slot audio injection), host KV cache, greedy decode. runtime_config.json, mel_filters.f32, and the tokenizer files are the exact inputs that client consumes. Audio input is 16 kHz mono.

Files

encoder.aimodel/                 fp16 Whisper tower + MLP adapter
encoder.h16c.aimodelc/           AOT-compiled encoder (h16c / M4 Max only)
decoder.aimodel/                 int8 static unified prefill+decode
decoder.h16c.aimodelc/           AOT-compiled decoder (h16c / M4 Max only)
runtime_config.json              prompt/config constants for clients
mel_filters.f32                  exact slaney mel filterbank from ARK's feature extractor
tokenizer*.json, vocab.json, merges.txt, added_tokens.json, special_tokens_map.json
coreai-arkasr-conversion.tar.gz  conversion + gate toolkit

Payload	Size	SHA-256
`encoder.aimodel/main.mlirb`	1,329,458,070 bytes	`14bfbb6d1503c6cf26686a3aa855614cd33a1d0cc686698788697f542fb0055c`
`decoder.aimodel/main.mlirb`	3,572,562,869 bytes	`e324f285d79421f18baea5213a3b021b813df7d15e009f01260b64517bd90b4d`
`encoder.h16c.aimodelc` resources.bin	1,328,929,212 bytes	`7ba10b5317d5fb1eee88df8ec3a318ac43984cfa25221bbd8f2f8dbde7172f1b`
`decoder.h16c.aimodelc` resources.bin	12,966,484,252 bytes	`88385079c215b51ad522b62c436a3a1ab72987f10d2d99128e0cb54d53ad871e`

Do not recompute the mel filterbank: Whisper uses slaney-scaled normalized triangles (peak ≈ 0.042), and the textbook HTK formula diverges by ~24x, which measurably corrupts transcription. Use mel_filters.f32 as shipped.

Local validation

Gated on an Apple M4 Max (h16c), macOS 27.0 (26A5388g), coreai-core 1.0.0b2, coreai-torch 0.4.1, against an fp32 PyTorch golden of the official model. The pass condition is exact token-for-token equality, not similarity.

Stage	Result
encoder `.aimodel` vs fp32 golden, real-audio rows	cosine mean 0.999998, min 0.999901
full pipeline greedy decode	42/42 tokens identical to PyTorch

Performance on the same machine (12.75 s clip):

Metric	fp16 dynamic	int8 static (this repo)
gate load	62.7 s	3.4 s
prefill	1.75 s	0.31 s
decode	71.0 ms/tok	51.8 ms/tok
app prewarm, fresh process	127 s	74 s

Architecture note: ARK decodes autoregressively through a 36-layer 3B LLM, so ~52 ms/token is a bandwidth floor every framework hits, GGUF/llama.cpp included. Purpose-built ASR models (e.g. Cohere Transcribe) are structurally faster for short dictation.

Long audio

The model is a fixed 30-second-window architecture: the official feature extractor pads or trims mel to 3000 frames, and a 33.5 s probe confirms the reference implementation truncates at the window edge. Longer audio is a host-side concern — window the input (the GGUF runtime in harshav/ARK-ASR-3B-GGUF overlaps windows by 2 s and stitches) and concatenate transcripts.

License and attribution

The original model is Apache-2.0 licensed. See the official model card and AutoArk repository for architecture, training, and upstream attribution.