aufklarer/FireRedLID-887M-MLX-8bit

FireRedLID 887M — MLX INT8

Native MLX conversion of FireRedTeam/FireRedLID, including its relative-position Conformer encoder, autoregressive LID decoder, and original beam-3, two-token hierarchical decoding. This is not a single-step classifier approximation.

This release applies MLX native 8-bit affine weight-only quantization with group size 64 to all eligible encoder and decoder linear matrices. Activations are not quantized, and convolution, normalization, embedding, and positional tensors retain the mixed-fp16-encoder-fp32-decoder release's validated precision. INT8 is numerically lossy; token parity and logit comparisons are recorded in validation.json. The audio frontend reproduces the official 80-bin Kaldi fbank and global CMVN path, including the Int16 PCM amplitude convention.

Usage

pip install mlx==0.32.0 numpy==1.26.4 scipy==1.14.1 \
  soundfile==0.13.1 kaldiio==2.18.1 kaldi-native-fbank==1.22.3

from pathlib import Path

import mlx.core as mx

from frontend import FireRedFrontend
from mlx_model import FireRedLIDModel

model_dir = Path(".")
frontend = FireRedFrontend(model_dir / "cmvn.ark")
features = frontend.extract_file("speech.wav")
model = FireRedLIDModel.from_directory(model_dir)
result = model.identify_features(mx.array(features[None]))
print(result[0][0].text, result[0][0].confidence)

Architecture and provenance

• 16 kHz mono audio; 80-bin Kaldi fbank; 25 ms window / 10 ms shift; snip_edges=true; zero dither; supplied global CMVN.
• Two stride-2 Conv2d layers (4x time subsampling).
• 16-layer, 1,280-dimensional relative-position Conformer encoder.
• 20 attention heads (64 dimensions per head) in both encoder and decoder.
• Six-layer autoregressive Transformer LID decoder over 120 tokens.
• Beam size 3, maximum output length 2, smoothing 1.25, GNMT length penalty 0.6, and EOS penalty 1.0.
• Source model revision: 1bb4d285c8456429385d9c0810300df4297bc11b.
• Source checkpoint SHA-256: 7dee2a280e9b11d5241a0e3d4fa60ee1520a036a2e8385f17960371cfea10093.
• Official source revision: 4e7d9aaf4482a47cec1724807026b9b151926eb5.

The official paper reports 97.18% on an 82-language FLEURS intersection, 92.07% on a 74-language Common Voice intersection, and 88.47% on its combined Chinese dialect/accent evaluation. Those intersections differ from other LID papers and should not be treated as directly comparable full-dataset scores.

Conversion validation

validation.json records independent frontend comparison, Conv2d subsampling parity, all 16 encoder-layer comparisons, both autoregressive decoder steps, decoded tokens, confidence, latency, and peak MLX memory. Validation runs the official PyTorch model in a separate process before loading MLX so the two large runtimes do not overlap in memory.

benchmark.json records fresh-process first-shape and repeated model-inference latency after separately timed frontend extraction, plus peak memory. Filesystem pages may already be cached, so its first inference timing is not represented as a cold-disk download/load benchmark.

The 1,134,584,042-byte artifact is 46.06% smaller than the validated 2,103,273,736-byte mixed-precision release. On four parity fixtures spanning 1.5 to 30 seconds, all decoded tokens match the pinned official PyTorch model, including the hierarchical zh mandarin result. Minimum cosine similarity is 0.9999585914 across all encoder layers and 0.9999605973 across both decoder steps; the maximum confidence difference is 0.00014848.

On Apple Silicon, the median of three fresh processes (seven warm repetitions each) reduced peak process RSS from 2.263 GB to 1.299 GB (42.6%). Model inference after frontend extraction improved from 89.3 to 53.7 ms first and 51.1 to 31.8 ms warm for a 1.5-second clip. For a 30-second clip, it improved from 260.2 to 251.9 ms first and from 255.6 to 242.4 ms warm. Frontend extraction is separately timed at roughly 1.1 ms and 18 ms and is unchanged by weight quantization. Long-input peak MLX memory fell from 3.904 GB to 2.753 GB (29.5%). See benchmark_comparison.json for raw medians and methodology.

These fixture measurements do not establish lossless quantization or unchanged corpus-level accuracy. The independent multilingual evaluation below broadens the release gate; the mixed-precision model remains the closest parity reference.

Independent FLEURS quantization evaluation

fleurs82_evaluation.json records a deterministic stratified sample from the FLEURS test set at revision 70bb2e84b976b7e960aa89f1c648e09c59f894dd: three clips for each of 82 mapped languages, evaluated clean and with 8 kHz telephony, 10 dB noise, and an energy-bearing 1.5-second crop. evaluate_fleurs.py reproduces the evaluation.

Condition	Clips	Dense accuracy	INT8 accuracy	Token agreement
Clean	246	95.93%	95.93%	100%
8 kHz telephony	246	97.15%	97.15%	100%
10 dB noise	246	92.68%	92.68%	100%
Highest-energy 1.5 s	246	36.59%	36.99%	99.59%
All conditions	984	80.59%	80.69%	99.90%

There are zero INT8 regressions and one INT8 recovery. Confidence absolute difference has median 0.00000191, mean 0.001441, p95 0.008888, p99 0.017291, and maximum 0.034719. The maximum occurs on a short low-margin input where both models return the same label. Applications using a confidence threshold should still calibrate that threshold directly on this INT8 release.

The upstream paper reports a full-test result over 82 languages but does not publish its language list or mapping. This release defines its intersection as 79 direct dictionary/FLEURS code matches, Mandarin and Cantonese hierarchical mappings, and the checkpoint's legacy iw/jw Hebrew and Javanese codes. The sampled result is not a reproduction of the paper's 97.18% headline and should not be compared as if it were the full test set.

Limitations

This is a closed-vocabulary decoder. It does not provide calibrated unknown- language detection and can return a confident known token for silence, music, unsupported speech, or code-switching. The upstream training range is 1.5 to 35 seconds. Production systems should use VAD, target-domain calibration, and an explicit reject policy. Language and regional-accent labels are model predictions, not reliable inferences of nationality or ethnicity.

See the official paper and source code for the training and evaluation methodology.