cmsha/omi-med-stt-v1-coreml

Omi Med STT v1 — CoreML

CoreML conversion of omi-health/omi-med-stt-v1, an English medical speech-to-text model for clinical dialogue built from NVIDIA Parakeet TDT 0.6B v2 (FastConformer + Token-and-Duration Transducer). Runs fully on-device on Apple Silicon (Neural Engine + CPU) via FluidAudio.

The component layout, I/O contract, tokenizer (1024 SentencePiece tokens, blank id 1024), and file names are identical to FluidInference/parakeet-tdt-0.6b-v2-coreml — only the weights differ (medical fine-tune). Any runtime that loads the stock v2 CoreML build can load this one from a local directory.

Files

File	Role	I/O
Preprocessor.mlmodelc	waveform → log-mel	audio_signal, audio_length → mel, mel_length
Encoder.mlmodelc	FastConformer encoder (FP16, 15 s window)	mel, mel_length → encoder, encoder_length
Decoder.mlmodelc	RNNT prediction network (U=1, explicit LSTM state)	targets, target_length, h_in, c_in → decoder, h_out, c_out
JointDecision.mlmodelc	fused single-step joint + decision head	encoder_step, decoder_step → token_id, token_prob, duration
parakeet_vocab.json	token id → SentencePiece piece (dict format)	—

Audio: 16 kHz mono, fixed 15-second window (shorter audio is padded by the runtime). Precision: FP16 MLProgram, minimum deployment target iOS 17 / macOS 14. Total size ≈ 1.1 GB.

Conversion provenance

• Source checkpoint: omimedstt-v1.nemo from omi-health/omi-med-stt-v1 (sha256 eaf0ff7258133b4e597aef024f81a4db1779abeb966e53f00a2250bb54e0fbf4).
• Exported with the FluidInference mobius parakeet-tdt-v2-0.6b/coreml pipeline (convert-parakeet.py --nemo-path …, NeMo 2.3.1, coremltools 9.0b1, PyTorch 2.7.0), then compiled with xcrun coremlcompiler and renamed to the FluidAudio v2 layout. The embedded model author field reads "Fluid Inference" because it is set by that conversion tooling; the conversion of this checkpoint was performed independently.
• Vocabulary exported from the checkpoint's SentencePiece tokenizer in FluidAudio's dict format.

Usage (FluidAudio / Swift)

Place all five artifacts in one directory and construct AsrModels directly (the stock download path would fetch generic v2 weights instead):

import CoreML
import FluidAudio

let dir = URL(fileURLWithPath: "/path/to/omi-med-stt-v1-coreml")
let ane = AsrModels.defaultConfiguration()          // CPU + Neural Engine
let cpu = MLModelConfiguration(); cpu.computeUnits = .cpuOnly

let models = AsrModels(
    encoder: try await MLModel.load(contentsOf: dir.appending(path: "Encoder.mlmodelc"), configuration: ane),
    preprocessor: try await MLModel.load(contentsOf: dir.appending(path: "Preprocessor.mlmodelc"), configuration: cpu),
    decoder: try await MLModel.load(contentsOf: dir.appending(path: "Decoder.mlmodelc"), configuration: ane),
    joint: try await MLModel.load(contentsOf: dir.appending(path: "JointDecision.mlmodelc"), configuration: ane),
    configuration: ane,
    vocabulary: vocabulary,   // parse parakeet_vocab.json: [Int: String]
    version: .v2
)
let manager = AsrManager(config: .default)
try await manager.loadModels(models)
let result = try await manager.transcribe(audioURL: fileURL)

In MacParakeet, select the "Omi Med STT v1" Parakeet build (Settings, or macparakeet-cli models select parakeet-omi-med-v1) — the app downloads this repository automatically on first use, or pre-fetch with macparakeet-cli models download parakeet-omi-med-v1.

Upstream model

Per the upstream model card: trained for clinical dialogue (GP consultations, medication reviews, procedure discussions); 8.30 % WER and 97.95 % medical recall on a 7.18-hour clinical benchmark. English only. It formats numerics clinically (e.g. "500 mg", "130 over 85"). See the upstream card for details and limitations.

Disclaimer — use at your own risk

This model conversion is provided as-is, without warranty of any kind, including no warranty of transcription accuracy. Use it at your own risk.

• Not a medical device. Transcripts are informational and require human review; do not rely on them for diagnosis, treatment, or any clinical decision.
• Patient privacy / HIPAA. If you process patient speech or any protected health information with this model, you are solely responsible for compliance with HIPAA and any other applicable privacy law or regulation (GDPR, state law, or local equivalents). On-device processing helps minimize data exposure but does not by itself make a workflow compliant.
• Institutional policy. Check with your institution's compliance, privacy, and IT-security policies before using this model in any clinical, research, or educational setting.
• No liability. The publisher of this conversion accepts no responsibility or liability for privacy, regulatory, or policy violations arising from its use.

License and attribution

Weights are CC-BY-4.0, following the upstream release by Omi Health (omi-med-stt-v1), which is built from NVIDIA's parakeet-tdt-0.6b-v2 (also CC-BY-4.0). This repository redistributes a format conversion of those weights with attribution and no additional restrictions.