wcamon/catcher-asr-mlx-int8

Catcher ASR — MLX INT8 artifact

An unofficial community INT8 conversion of nvidia/nemotron-3.5-asr-streaming-0.6b for the Catcher Rust + MLX-C inference runtime on Apple Silicon.

This repository is not affiliated with, endorsed by, or sponsored by NVIDIA. Nemotron is referenced only to identify the source model. No NVIDIA logo or brand artwork is used.

What is included

• all 655 checkpoint tensors in the Catcher artifact layout;
• affine weight-only INT8 matrix and pointwise weights, group size 128;
• FP16 convolution, normalization, scale, bias, and other sensitive tensors;
• tokenizer and model configuration files;
• a machine-readable manifest.json describing every tensor;
• the upstream model card in NVIDIA_MODEL_CARD.md;
• OpenMDW-1.1 and origin notices.

Artifact size is approximately 629 MiB (659.6 MB decimal). The original F32 checkpoint is approximately 2.4 GiB.

Download

hf download wcamon/catcher-asr-mlx-int8 \
  --local-dir catcher-asr-mlx-int8

No Hugging Face token is required because this repository is public.

Catcher runtime usage

On an Apple Silicon Mac with the Catcher CLI built:

catcher transcribe \
  --model catcher-asr-mlx-int8 \
  --audio speech.wav \
  --language en-US \
  --lookahead 3

The current runtime accepts mono 16 kHz PCM or float WAV input. It implements the audio frontend, cache-aware 24-layer FastConformer encoder, language prompt, greedy RNNT decoder, and tokenizer in Rust, while MLX-C/Metal executes the accelerated tensor kernels.

Validation status

This artifact has been loaded by the Catcher runtime and tested end to end on Apple Silicon. For the included development reference utterance, Catcher's non-blank streaming RNNT token IDs exactly matched the official Transformers implementation and decoded as:

Hello, this is a streaming speech recognition test

The 4.151875-second reference utterance completed in 2.64 seconds in one local release run (real-time factor 0.64), using approximately 702.5 MiB maximum resident memory. These figures are hardware- and workload-specific.

This is an experimental community conversion. It has not yet been evaluated on a complete multilingual benchmark, and no claim is made that its WER is identical to the F32 source checkpoint. Contributions of reproducible language and accuracy evaluations are welcome.

Current runtime limitations

• Apple Silicon macOS only;
• greedy RNNT decoding;
• mono 16 kHz input;
• tested Catcher lookahead values: 0, 3, 6, and 13 encoder frames;
• not directly loadable by Transformers, NeMo, or Python MLX;
• requires the Catcher artifact loader and runtime.

License and attribution

The source model is Copyright NVIDIA Corporation and is distributed under the OpenMDW License Agreement version 1.1. This quantized distribution retains that license and the applicable notice of origin. See LICENSE, NOTICE.md, and NVIDIA_MODEL_CARD.md.

Model outputs are addressed by the OpenMDW-1.1 terms. Users remain responsible for reviewing the license and ensuring their use complies with applicable law.

The Catcher runtime is an independent community implementation and is not part of NVIDIA NeMo or the official Transformers implementation.