AuroraASR-1M-MLX

Native mlx.nn port of AuroraAI-Research/AuroraASR-1M (Apache-2.0, revision cd307b54b4d666d6ad7dc982a7a58a96d6e7f942) for Apple Silicon. It reproduces the PyTorch logits to 1.5e-4 at every length I tested, and it runs an 11 second clip through the acoustic model in about 2.0 ms on an M4 Max. Weights are the original FP32 values, only re-laid-out for MLX. This is a conversion, not a fine-tune, so recognition quality is exactly the upstream model's.

Runtime dependencies are mlx, numpy, sentencepiece, soundfile, and soxr. No ONNX Runtime and no PyTorch.

The earlier ONNX Runtime + MLX execution provider build that used to live in this repo moved to cowWhySo/AuroraASR-1M-ONNX-MLX-EP. A Core ML build is at cowWhySo/AuroraASR-1M-CoreML.

Quick start

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
python aurora_pipeline.py reference_jfk.flac

from aurora_pipeline import Transcriber, read_audio

asr = Transcriber(".")                      # compiled graph, NumPy frontend
print(asr.transcribe(read_audio("clip.wav")))
print(asr.transcribe_batch([wave_a, wave_b]))   # right-padded batch with the upstream masking

Input is any file soundfile reads. It is mixed to mono and resampled to 16 kHz with soxr HQ. bash run_mac.sh installs, runs the tests, and transcribes the bundled clip. Add --bench to also run the full benchmark.

Model

• 949,073 parameters, 4 unique Conformer-lite blocks applied twice each (weight shared), CTC head over 257 tokens (blank plus 256 SentencePiece pieces). English only.
• Input is 80-bin log-mel features, [B, T, 80]. Output is [B, ceil(T/4), 257]. Any length works, there is no fixed shape.
• The model is bidirectional and non-streaming. There is no incremental decoding.
• model.safetensors (FP32, MLX layout, safetensors metadata format: mlx) SHA-256: 45b91b4b54171270b13b8a8af9bd4ca789ac3e2a37392951b8c868f60346a2d1. Tensor data SHA-256: 80ed02b9541f47b56a0122ec5df997acf6d3d54105146518ef760dd12bf5438f. That second hash is the one a re-conversion reproduces, because the safetensors header does not serialize its metadata in a fixed order.
• config.json holds the model dimensions only. It is not a transformers config, so AutoModel.from_pretrained will not work on this repo. Use aurora_pipeline.Transcriber.

Results

Measured on 18 October 2026: Apple M4 Max, macOS 26.6.2, CPython 3.13.16, mlx 0.32.4. Raw numbers are in benchmarks/native_mlx_report.json.

Parity with PyTorch (torch 2.14.1)

Reference logits come from the upstream model and feature extractor. The clips are crops of the bundled JFK recording. Max error is the largest absolute difference over all logits when this repo's model gets the same features as the reference.

Clip	Mel frames	Max abs error	Argmax agreement	Transcript
0.5 s	51	2.7e-5	100%	match
2.0 s	201	3.0e-5	100%	match
3.37 s	338	4.3e-5	100%	match
5.5 s	551	2.9e-5	100%	match
11 s	1101	1.5e-4	100%	match

• A padded batch (2 s and 11 s together) matches the upstream padded output to 1.3e-4 on every valid frame.
• Going from the waveform with this repo's own NumPy frontend, max logit error is 0.0134 (mean under 2e-4) at worst, with identical argmax and transcripts. The MLX frontend peaks at 0.0033 on the same clips.
• Padded batches have an upstream quirk, covered under Padding below. By default this port reproduces it.

Padding

In the upstream model a clip's output can change depending on how much padding surrounds it in a batch. Cause, confirmed in both PyTorch and this port: five of the 16 conv1 channels have a positive bias, so after the ReLU the padded region holds a nonzero value instead of the zeros a lone clip sees past its end. The stride-2 depthwise conv reads one frame beyond the clip, and that frame lands in the last valid stem frame. Attention is bidirectional, so the one wrong frame shifts every logit.

• It triggers only when the clip's conv1 length, (mel_frames + 1) // 2, is odd and the batch is long enough to have more conv1 frames than the clip does. That is about half of all clip lengths.
• The amount of padding does not matter once there are at least 2 extra mel frames. A clip padded by 2 frames shifts by the same 4.29 logits as one padded by 900.
• Masking the padded conv1 output to zero removes it: the 2 s clip then matches its solo result to 3.3e-5 in PyTorch and 1.4e-5 here.
• Size of the effect on the JFK crops (16 lengths from 0.6 s to 10 s, each batched with the 11 s clip): logits moved by 2 to 7 on 12 of them, and the transcript changed on 6, by 1 or 2 characters at a time ("fllo" to "folo", "your cpran do" to "your cpr can do"). With the mask, 0 of 16 changed.
• I cannot say which version is more accurate. There is no labeled data here. The solo result is the deterministic one, and if upstream trained with padded batches, it trained through this quirk (I have not seen its training code).

Transcriber(exact_padding=False) is the default and matches upstream. Transcriber(exact_padding=True) applies the mask, so batched results equal solo results. I recommend it if you batch utterances of different lengths and want results that do not depend on batch composition. It changes nothing for a single unpadded clip.

Speed

Acoustic model only, 11 s clip (1101 mel frames), features already in host memory. Each backend ran in its own fresh process, 3 rounds of 60 timed runs, order shuffled per round, median of round medians.

Backend	P50	P95	Peak process RSS
ONNX Runtime CPU	5.615 ms	5.833 ms	91.7 MiB
ONNX Runtime + MLX EP	2.438 ms	2.632 ms	112.4 MiB
Native MLX, first release path (eager, logits to host)	2.195 ms	2.352 ms	67.4 MiB
Native MLX, this release (mx.compile, token ids to host)	1.995 ms	2.072 ms	69.5 MiB

Against the MLX execution provider, native MLX is 1.11x faster eager and 1.22x faster compiled. Against ONNX Runtime CPU it is 2.56x and 2.81x. The comparison script and its raw report are in the ONNX repo (bench_onnx.py). I did not run Core ML under this protocol, so I make no native MLX versus Core ML claim.

What changed in this release, from scripts/bench_native.py (same protocol, native variants only):

Variant	P50	vs first release
Eager, logits to host	2.206 ms	1.00x
Eager, token ids to host	2.260 ms	0.98x
mx.compile, logits to host	1.966 ms	1.12x
mx.compile, token ids to host	1.987 ms	1.11x
mx.compile, no host copy (diagnostic)	1.955 ms	1.13x

• mx.compile is the whole gain, about 11%. It fuses the many small elementwise kernels.
• Running the CTC argmax on the GPU and copying only the token ids did not help. I tested it because the logits copy looked like the obvious cost (284 KB against 1.1 KB). Eager with argmax is 0.05 ms slower, and the "no host copy" run is only 0.01 ms faster than compiled with logits. Unified memory makes that copy close to free at this size. I kept the id path as the default because it is never slower and moves less data on long audio or large batches.
• The earlier 3.27 ms native figure was measured in one process shared with two ONNX Runtime sessions, interleaved. Run in its own process, the same code takes 2.2 ms, and the MLX EP figure drops from 3.57 to 2.44 ms the same way.
• The model is launch-bound at these sizes. Latency is 1.4 ms at 2 s of audio and 2.0 ms at 11 s.

Whole pipeline (waveform to text, 11 s clip, warm)

Frontend	P50	P95
NumPy (default, closest to the reference features)	3.32 ms	3.92 ms
MLX (--frontend mlx / Transcriber(frontend="mlx"))	2.27 ms	2.44 ms

The MLX frontend computes the log-mel on the GPU and keeps features off the host. Its features differ from the PyTorch ones slightly more than the NumPy frontend does (max 0.018 against 0.009 on the 11 s clip, mean about 1e-5 for both). Transcripts matched on all five test clips, but that is one recording, so it is opt-in.

Longer audio and batching (compiled, token ids to host)

Audio	Acoustic P50	Speed
2 s	1.39 ms	1440x realtime
11 s	1.96 ms	5600x
22 s	3.05 ms	7210x
44 s	5.16 ms	8530x
88 s	13.5 ms	6520x

Attention is quadratic in length, which is why throughput falls off past 44 s. Batching 11 s clips helps: 495 clips/s at batch 1, 1275 at batch 8, 1431 at batch 16.

Memory

Measured in a process that loads only this model. Peak RSS is 67 to 69 MiB, of which 29 MiB is the Python and MLX runtime before the model loads. The MLX allocator peaked at 41.7 MiB, but 37 MiB of that is reusable cache. Live tensors after a run are about 4 MiB, which is the weights. The earlier 163 MiB figure came from one process holding three backends.

What this does not show

• Accuracy. Parity means this port matches the upstream model. The bundled JFK transcript has several wrong words ("fll ameraca", "coupr"), and they are wrong upstream too. No WER evaluation was run on this conversion. See the upstream card for its numbers.
• Other machines. Everything above is one M4 Max, one clip family, one OS version. Absolute latency will differ on other chips.
• Streaming, or any speaker or language beyond what upstream supports.

Files

File	What it is
aurora_mlx.py	The mlx.nn model and load_native
aurora_pipeline.py	Audio reading, log-mel frontends (NumPy and MLX), CTC decode, Transcriber, CLI
model.safetensors, config.json, native_manifest.json	Converted weights, model dimensions, and provenance (source revision, hashes, validation record)
scripts/convert_weights.py	Upstream checkpoint to MLX layout. Checks every tensor name and shape. Re-running it reproduces tensor_data_sha256
tests/test_native_parity.py	15 tests: multi-length and padded-batch parity, the padding quirk and its fix, frontends, CTC, weight file format and hashes
parity_fixtures.npz, scripts/make_parity_fixtures.py	PyTorch reference logits and the script that made them
reference_fixture.npz, reference_jfk.flac	The original single-clip fixture and audio
scripts/bench_native.py, benchmarks/native_mlx_report.json	Benchmark and its raw output

Run the tests with python -m unittest discover -s tests -v. Reproduce the tables with python scripts/bench_native.py --rounds 3 --runs 60.

Credit

Architecture, training, and weights belong to the AuroraAI-Research authors. This repo only changes the file format and the runtime.