phequals/indic-transcribe-core-coreml

Indic-Transcribe Core — experimental CoreML port

Built with Indic-Transcribe-core from Bodhan AI / AI4Bharat. Derived from NVIDIA Canary-1B-v2. This is a format conversion, not a fine-tune.

Pinned upstream revision: 4d29eeb7a0990de4a8febf9a4d5a9c5c61134a0d.

Status

The three FP16 CoreML packages have been exported on JarvisLabs and executed through native Swift on macOS. Two Hindi fixtures (12.78 seconds and a 30-second synthetic workplace Hinglish crop) reproduce the upstream FP32 transcript exactly. This small smoke test does not establish multilingual quality, WER, robustness, or production readiness.

The 12.78-second sample took about 1.69 seconds warm in the native Swift harness. First-use compilation/warmup took about 25.6 seconds for prediction, in addition to model loading. Latencies are specific to the development Mac and sample. Longer token sequences cost more: the 30-second workplace sample took 6.59–9.67 seconds in two subsequent native runs. The autoregressive decoder dominates. Later short-sample runs varied between 2.3 and 5.8 seconds while still matching the transcript. These measurements are observations, not latency guarantees; controlled repeated benchmarking remains necessary.

Core renders English terms in Indic speech in the native script. The shipped upstream wrapper accepts a mixed flag, but it did not preserve Latin workplace terms in our test. Do not market this port as a mixed-script Hinglish model. Automatic language identification can choose the wrong script; on our Hindi poetry sample it chose Urdu. Preserve explicit language selection.

Layout and inference

• coreml/encoder.mlpackage: mel features + true frame mask → encoder states/lengths.
• coreml/cross.mlpackage: encoder states → cached cross-attention keys/values.
• coreml/decoder.mlpackage: stateful self-attention cache and fused vocabulary head.
• native-assets/frontend.bin: exact upstream Hann window and mel filterbank, little-endian float32.
• native-assets/tokenizer.json: upstream token pieces and native-mode language prompts.
• BodhanFrontend.swift, BodhanCoreML.swift: native validation implementation.

Requires macOS 15+. Use CPU+GPU for the measured configuration. Compile packages locally; .mlmodelc is deliberately not distributed. Keep models loaded, allocate a fresh decoder state for each utterance, and retain the projected cross-attention arrays for the complete decode. A fresh state is required after a LID-only probe.

The frontend accepts 16 kHz mono samples. The native validation runtime accepts up to 30 seconds per invocation and caps generation at 256 new tokens, reporting an error if EOS is not reached. Longer recordings require segmentation and careful boundary merging. The Muesli experimental adapter uses 28-second windows with one second of overlap; this policy still needs dedicated long-form evaluation.

The encoder supports 101–3001 mel frames; decoder prompt length is 1–16 with 512 cached positions. Preserve true lengths and exact frontend normalization. Do not zero-pad audio/features to a new bucket and assume identical encoder output.

Reproduction

Export dependencies used: PyTorch 2.11.0+cu130, transformers 4.57.6, coremltools 9.0. See coreml_port.py and export_coreml.py. The exporter expects an upstream snapshot and a source-generated parity fixture; private test audio and fixtures are not included in this model repository.

Native harness:

swiftc -O BodhanFrontend.swift BodhanCoreML.swift native_bench.swift -o native_bench

Supply your own 16 kHz float32 audio as described by the harness. This repository contains no evaluation recordings. Muesli exposes this model as Bodhan Core under Experimental, with a language picker including Auto-detect. Its standard model directory is ~/.cache/muesli/models/indic-transcribe-core-coreml; developers can override it with MUESLI_BODHAN_CORE_MODEL_DIR. It is not a default replacement.

License and attribution

The converted model remains under the unmodified Indic Open Model License v1.0 included here. Upstream NVIDIA Canary-1B-v2 is under CC BY 4.0. Preserve both attributions and the upstream notices. See NOTICE.md and the complete license.

Sources: Bodhan model, NVIDIA base model.

Experimental CoreML + MLX prototype

A standalone MLX decoder variant can reuse this CoreML encoder. It includes decoder weights and a Python benchmark runner; it is not yet the Muesli native backend. See the variant README for measurements and limitations.