aufklarer/Qwen3-ForcedAligner-0.6B-CoreML-INT8

Qwen3-ForcedAligner-0.6B — CoreML INT8

CoreML conversion of Qwen/Qwen3-ForcedAligner-0.6B with INT8 kmeans palettization on the two transformer bundles, for Apple Silicon.

Predicts word-level timestamps for audio + text pairs in a single non-autoregressive forward pass.

Files

File	Role	Size
audio_encoder.mlmodelc / .mlpackage	24-layer block-attention encoder (INT8-palettized); fixed 30 s mel input	~304 MB
text_decoder.mlmodelc / .mlpackage	28-layer non-AR decoder + 5000-class classify head (INT8-palettized); fixed T=768	~427 MB
embed_tokens.fp16.bin	Raw little-endian fp16 token embedding table, shape [152 064, 1024] (not palettized)	~297 MB
config.json	Runtime config — variant, classify_num, timestamp_segment_time, fixed shapes	small
vocab.json, merges.txt, tokenizer_config.json	Qwen3 BPE tokenizer files	~5 MB

The embed-tokens table is shipped as a raw fp16 binary rather than a CoreML package — the Swift runtime memory-maps the file and gathers rows with vImageConvert_Planar16FtoPlanarF, which costs ~0.5 ms per alignment instead of the ~70 ms an mlpackage round-trip would cost. Keeping the embedding at fp16 (not palettized) preserves classify-time accuracy across the full vocabulary; the palettization budget is spent on the two large transformer bundles, where it matters most for disk and ANE memory.

Usage (Swift via speech-swift)

import Qwen3ASR

let aligner = try await CoreMLForcedAligner.fromPretrained(
    modelId: "aufklarer/Qwen3-ForcedAligner-0.6B-CoreML-INT8")
let aligned = try aligner.align(
    audio: pcmSamples,
    text: "Can you guarantee that the replacement part will be shipped tomorrow?",
    sampleRate: 16000,
    language: "English")

CLI: speech align audio.wav --engine coreml --aligner-model aufklarer/Qwen3-ForcedAligner-0.6B-CoreML-INT8

Performance (M2 Max, 64 GB, debug build, 20 s clip)

Metric	Value
RTF	0.014 (69× faster than real-time)
Peak RSS	697 MB
Median alignment time	~289 ms

Per-stage profile (COREML_ALIGN_PROFILE=1):

mel=150ms  encoder=78ms  embedding=0.5ms  splice=0ms  decoder=54ms  argmax=7ms

The decoder uses MLComputeUnits = .all because its 28 layers exceed the ~26-layer ANE graph cap; the encoder runs on .cpuAndNeuralEngine.

Architecture

Same model as the upstream Qwen3-ForcedAligner. Inference is non-autoregressive: one forward pass over a chat-template prompt (system + audio + assistant + <ts> word <ts> slots), then argmax at the timestamp positions, then LIS monotonicity correction.

The causal mask is baked into the exported graph as a constant with a finite -1e4 fill value (not -inf) so the fp16 softmax cannot produce NaN.

Conversion source: soniqo/speech-models/models/forced-aligner/export/convert_coreml.py.

Variants

• This repo: INT8 — kmeans-palettized text decoder + audio encoder, ~880 MB, similar RTF, ~35% lower peak memory than FP16.
• Companion: aufklarer/Qwen3-ForcedAligner-0.6B-CoreML-FP16 — full precision, ~1.7 GB, highest fidelity.

For pure MLX paths (no CoreML), see the 4bit, 5bit, 8bit, and bf16 siblings under the same HuggingFace org.