Add Pocket (300M, CPU) variant: int8/fp32 ONNX, PyTorch checkpoint, vocab, CPU inference script, results; README variants table

README.md

@@ -20,6 +20,7 @@ tags:
 - ctc
 - omnilingual-asr
 - streaming
+- onnx
 ---
 
 # Dynamic ASR
@@ -39,6 +40,66 @@ Dynamic ASR is Meta's [Omnilingual ASR](https://github.com/facebookresearch/omni
 
 Both are the same size (1B, ~1.95 GB bf16 state dict) and load the same way.
 
+## Variants
+
+| folder | model | parameters | files | runtime |
+|---|---|---|---|---|
+| `real/` (**default**) | omniASR_CTC_1B_v2 fine-tuned on licensed real speech | 1B | `model.pt` (1.95 GB, bf16) | PyTorch / fairseq2 (`transcribe.py`) |
+| `codeswitch-experimental/` | same, plus synthetic code-switched speech | 1B | `model.pt` (1.95 GB, bf16) | PyTorch / fairseq2 (`transcribe.py`) |
+| `pocket/` | omniASR_CTC_300M_v2 student distilled from a 1B Dynamic ASR teacher | 300M | `model_int8.onnx` (367 MB), `model_fp32.onnx` (1.30 GB), `model.pt` (1.30 GB), `vocab.json` | onnxruntime on CPU (`pocket/transcribe_pocket.py`) |
+
+The Pocket numbers below use a different scorer from the 1B tables in Results, so compare Pocket only with the teacher row in its own table.
+
+## Pocket (300M, CPU)
+
+`pocket/` is a smaller model for CPU and offline use. It is Meta's `omniASR_CTC_300M_v2` (300M-parameter wav2vec2 CTC, Apache-2.0, same tokenizer and 20 ms frame rate as the 1B models), distilled from a 1B Dynamic ASR teacher. The teacher is an internal checkpoint (`real/` further fine-tuned with real-voice spliced code-switch clips); it is not one of the files in this repository. Checkpoint: P2, step 2500.
+
+**Training.** Loss = CTC on the reference transcripts + KL divergence to the teacher's per-frame distribution; CNN feature encoder frozen. Data: the licensed real speech listed under Training data, code-switch clips spliced from that same licensed speech, and an oversample of 0.9 h of real code-switched training clips. No synthetic TTS audio and no African Voices data.
+
+**Files**
+
+| file | size | what it is |
+|---|---|---|
+| `pocket/model_int8.onnx` | 367 MB | int8-quantised ONNX (recommended for CPU) |
+| `pocket/model_fp32.onnx` | 1.30 GB | fp32 ONNX |
+| `pocket/model.pt` | 1.30 GB | PyTorch checkpoint: a dict with `model` (state dict for `omniASR_CTC_300M_v2`), `step` and the training `args` |
+| `pocket/vocab.json` | 127 KB | CTC vocabulary (index 0 = blank, 1–3 = special tokens) |
+| `pocket/transcribe_pocket.py` | | CPU inference: onnxruntime, greedy CTC decoding, 16 kHz mono input |
+| `pocket/results.json` | | the scores and CPU benchmark below |
+
+ONNX graph: input `wav` (float32, `[1, samples]`, 16 kHz mono), output `logprobs` (`[1, frames, 10288]`).
+
+**Usage**
+
+```bash
+pip install onnxruntime numpy soundfile scipy huggingface_hub
+huggingface-cli download Modularcomputing/dynamic-asr --include "pocket/model_int8.onnx" "pocket/vocab.json" "pocket/transcribe_pocket.py" --local-dir .
+python pocket/transcribe_pocket.py audio.wav                                  # int8 (default)
+python pocket/transcribe_pocket.py audio.wav --model pocket/model_fp32.onnx   # fp32
+```
+
+Any wav is mixed to mono and resampled to 16 kHz; clips over ~20 s are cut at quiet points and transcribed in pieces. Output format is the same as the 1B models.
+
+**Results** (CER, greedy CTC, no language model). *gdev* = the general dev set (WAXAL validation + OpenSLR-70, 187 clips); it was used for checkpoint selection, so it is a dev set, not a test set. *Real code-switch* = the held-out 94-clip real code-switched test set described above, never used for training or selection. Scoring: NFC, lowercase, punctuation and apostrophes removed, English number words to digits, Igbo tone marks removed from both sides, Yoruba tone marks kept.
+
+| model | English | Hausa | Igbo | Pidgin | Yoruba | gdev mean | real code-switch |
+|---|---|---|---|---|---|---|---|
+| Pocket int8 ONNX | 0.0571 | 0.0522 | 0.0595 | 0.0963 | 0.1359 | 0.0802 | 0.1558 |
+| Pocket fp32 ONNX | 0.0571 | 0.0514 | 0.0595 | 0.0925 | 0.1350 | 0.0791 | 0.1558 |
+| 1B teacher (greedy) | 0.0294 | 0.0489 | 0.0598 | 0.0823 | 0.1288 | 0.0698 | 0.1375 |
+
+- int8 adds about 0.001 to the gdev mean (mostly Pidgin) and nothing on real code-switch.
+- English is the largest gap to the teacher (0.057 vs 0.029).
+
+**CPU speed** (Intel Xeon Platinum 8480+, batch 1, 20 clips totalling 110 s, one fresh process per setting)
+
+| model | threads | RTF | median latency per clip | peak RAM |
+|---|---|---|---|---|
+| int8 ONNX | 4 | 0.071 | 358 ms | 939 MB |
+| fp32 ONNX | 4 | 0.186 | 952 ms | 2287 MB |
+
+Pocket shares the limitations listed below, with higher error than the 1B models.
+
 ## Results
 
 All test data is held out (never trained on). Scores are WER / CER after lower-casing and stripping punctuation; "no-diacritic" CER folds tone marks and dots. Decoding: greedy CTC with a Latin-script logit mask (the duplex setting).
@@ -180,7 +241,8 @@ Full fine-tune of all transformer layers (CNN feature encoder frozen), bf16 auto
 ## Licence and attribution
 
 - **Weights: CC-BY-SA-4.0.** Several training sources are CC-BY-SA-4.0 (WAXAL Media Trust subsets, Open.Bible, OpenSLR 70/86), so the fine-tuned weights are shared alike under CC-BY-SA-4.0. The base model omniASR_CTC_1B_v2 is Apache-2.0 (© Meta Platforms).
-- **Code (`transcribe.py`, `code/`): Apache-2.0** (`LICENSE-CODE`).
+- **Code (`transcribe.py`, `code/`, `pocket/transcribe_pocket.py`): Apache-2.0** (`LICENSE-CODE`).
+- **Pocket weights (`pocket/`): CC-BY-SA-4.0**, like the 1B weights. Base model omniASR_CTC_300M_v2 is Apache-2.0 (© Meta Platforms).
 - Attribution: Google WAXAL (Media Trust subsets), Google FLEURS (CC-BY-4.0), OpenSLR 70 Nigerian English and OpenSLR 86 Yoruba (Google, CC-BY-SA-4.0), Open.Bible / Biblica (CC-BY-SA-4.0), NaijaS2ST (CC-BY-4.0), IgboAPI (CC-BY-4.0), IgboSynCorp (CC0-1.0, Harvard Dataverse doi:10.7910/DVN/YB9FWK), Meta Omnilingual ASR (Apache-2.0), Qwen3 (Apache-2.0).
 
 ## Citation

config.json

@@ -23,6 +23,19 @@
    "path": "codeswitch-experimental/model.pt",
    "trained_on": "same real speech + synthetic code-switched audio (Flash-DPO TTS)",
    "status": "experimental"
+  },
+  "pocket": {
+   "path": "pocket/model_int8.onnx",
+   "alt_paths": [
+    "pocket/model_fp32.onnx",
+    "pocket/model.pt"
+   ],
+   "vocab": "pocket/vocab.json",
+   "architecture": "omniASR_CTC_300M_v2 student distilled from a 1B Dynamic ASR teacher",
+   "runtime": "onnxruntime CPU, greedy CTC, 16 kHz mono",
+   "script": "pocket/transcribe_pocket.py",
+   "results": "pocket/results.json",
+   "status": "CPU variant"
   }
  },
  "decoding": {

pocket/model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:315a5f44e5b47e6503f22cfaf696caf80048be5aee21ecbf2d04d0dfb2b8ec6b
+size 1304038762

pocket/model_fp32.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8fe64e7dd2b52931eaa6f957ce084f3af9392d36a561a38d5e2a6539a866ed66
+size 1304478206

pocket/model_int8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5431eba442f651351e69e1bf91a467253a39350b465595a9b04a6f5bfb7aa33f
+size 366630967

pocket/results.json

@@ -0,0 +1,121 @@
+{
+ "model": "Dynamic ASR Pocket",
+ "architecture": "omniASR_CTC_300M_v2 student distilled from a 1B Dynamic ASR teacher (omniASR_CTC_1B_v2 fine-tune)",
+ "checkpoint": "P2, step 2500",
+ "decoding": "greedy CTC, no language model",
+ "input": "16 kHz mono float32",
+ "scoring": "CER after: NFC, lowercase, punctuation and apostrophes to spaces, English number words to digits, dot-below variants unified, Igbo tone marks stripped from both sides (Yoruba tone marks kept)",
+ "eval_sets": {
+  "gdev": "general dev set (WAXAL validation + OpenSLR-70; 187 clips, 1269 s; eng/hau/ibo/pcm/yor). Used for checkpoint selection, so it is a dev set, not a test set.",
+  "real_codeswitch": "held-out real code-switched speech test set (94 clips, 758 s); never used for training or selection"
+ },
+ "results": {
+  "pocket_int8_onnx": {
+   "gdev_cer": {
+    "eng": 0.0571,
+    "hau": 0.0522,
+    "ibo": 0.0595,
+    "pcm": 0.0963,
+    "yor": 0.1359,
+    "mean": 0.0802
+   },
+   "real_codeswitch_cer": 0.1558,
+   "real_codeswitch_wer": 0.4311,
+   "real_codeswitch_english_word_recall": 0.3009
+  },
+  "pocket_fp32_onnx": {
+   "gdev_cer": {
+    "eng": 0.0571,
+    "hau": 0.0514,
+    "ibo": 0.0595,
+    "pcm": 0.0925,
+    "yor": 0.135,
+    "mean": 0.0791
+   },
+   "real_codeswitch_cer": 0.1558,
+   "real_codeswitch_wer": 0.4333,
+   "real_codeswitch_english_word_recall": 0.3009
+  },
+  "pocket_pytorch_fp32": {
+   "gdev_cer": {
+    "eng": 0.0571,
+    "hau": 0.0514,
+    "ibo": 0.0595,
+    "pcm": 0.0925,
+    "yor": 0.135,
+    "mean": 0.0791
+   },
+   "real_codeswitch_cer": 0.1558,
+   "real_codeswitch_wer": 0.4333,
+   "real_codeswitch_english_word_recall": 0.3009
+  },
+  "teacher_1b_greedy": {
+   "gdev_cer": {
+    "eng": 0.0294,
+    "hau": 0.0489,
+    "ibo": 0.0598,
+    "pcm": 0.0823,
+    "yor": 0.1288,
+    "mean": 0.0698
+   },
+   "real_codeswitch_cer": 0.1375,
+   "real_codeswitch_wer": 0.3968,
+   "real_codeswitch_english_word_recall": 0.4248
+  }
+ },
+ "sizes_bytes": {
+  "model_int8.onnx": 366630967,
+  "model_fp32.onnx": 1304478206,
+  "model.pt": 1304038762
+ },
+ "cpu_benchmark": {
+  "cpu": "Intel Xeon Platinum 8480+",
+  "batch": 1,
+  "clips": 20,
+  "audio_s": 110.1,
+  "runs": [
+   {
+    "model": "pocket_int8.onnx",
+    "threads": 4,
+    "rtf": 0.0706,
+    "ms_median": 357.7,
+    "ms_p90": 509.3,
+    "peak_rss_mb": 939.1,
+    "load_s": 1.02
+   },
+   {
+    "model": "pocket_int8.onnx",
+    "threads": 2,
+    "rtf": 0.133,
+    "ms_median": 693.8,
+    "ms_p90": 947.1,
+    "peak_rss_mb": 935.2,
+    "load_s": 1.0
+   },
+   {
+    "model": "pocket_int8.onnx",
+    "threads": 1,
+    "rtf": 0.139,
+    "ms_median": 708.1,
+    "ms_p90": 995.2,
+    "peak_rss_mb": 935.2,
+    "load_s": 0.92
+   },
+   {
+    "model": "pocket_fp32.onnx",
+    "threads": 4,
+    "rtf": 0.1857,
+    "ms_median": 952.1,
+    "ms_p90": 1294.9,
+    "peak_rss_mb": 2287.3,
+    "load_s": 3.21
+   }
+  ]
+ },
+ "notes": [
+  "The 1B teacher is an internal checkpoint used for distillation; it is not one of the model files in this repository.",
+  "English is the largest gap between Pocket and the teacher."
+ ],
+ "training_data": "licensed real speech (same sources as the 1B models), real-voice splice code-switch clips built from that licensed speech, and an oversample of 0.9 h of real code-switched training clips; no synthetic TTS audio; no African Voices data",
+ "distillation": "CTC loss on references + KL divergence to the 1B teacher's per-frame distribution; CNN feature encoder frozen"
+}

pocket/transcribe_pocket.py

@@ -0,0 +1,90 @@
+#!/usr/bin/env python3
+"""Dynamic ASR Pocket (300M, CPU): offline transcription with onnxruntime and greedy CTC decoding.
+
+  pip install onnxruntime numpy soundfile scipy
+  python transcribe_pocket.py clip.wav [more.wav ...]            # int8 model (default)
+  python transcribe_pocket.py clip.wav --model model_fp32.onnx   # fp32 model
+
+Input: any wav; it is mixed to mono and resampled to 16 kHz. Clips longer than ~20 s are cut at quiet points.
+Model files are looked up next to this script: model_int8.onnx (or model_fp32.onnx) and vocab.json.
+Code licence: Apache-2.0. Model weights: CC-BY-SA-4.0.
+"""
+import argparse, json, os, time
+import numpy as np
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+SR = 16000
+
+
+class PocketASR:
+    def __init__(self, model=None, vocab=None, threads=4):
+        import onnxruntime as ort
+        so = ort.SessionOptions()
+        so.intra_op_num_threads = threads
+        so.inter_op_num_threads = 1
+        so.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
+        self.sess = ort.InferenceSession(model or os.path.join(HERE, "model_int8.onnx"), so,
+                                         providers=["CPUExecutionProvider"])
+        v = json.load(open(vocab or os.path.join(HERE, "vocab.json"), encoding="utf-8"))
+        self.pieces = v["pieces"]          # index 0 = CTC blank; 1-3 = special tokens (never emitted)
+
+    def logprobs(self, wav):
+        # input "wav": float32 [1, T] at 16 kHz -> output "logprobs": [1, frames, vocab]
+        return self.sess.run(None, {"wav": np.asarray(wav, dtype=np.float32)[None]})[0][0]
+
+    def greedy(self, lp):
+        out, prev = [], -1
+        for p in lp.argmax(-1):
+            if p != prev and p > 3:
+                out.append(self.pieces[p])
+            prev = p
+        return " ".join("".join(out).split())
+
+    def transcribe(self, wav):
+        if len(wav) < SR // 10:
+            return ""
+        return self.greedy(self.logprobs(wav))
+
+
+def load_wav(path):
+    import soundfile as sf
+    y, sr = sf.read(path, dtype="float32", always_2d=True)
+    y = y.mean(1)
+    if sr != SR:
+        from math import gcd
+        from scipy.signal import resample_poly
+        g = gcd(sr, SR)
+        y = resample_poly(y, SR // g, sr // g).astype(np.float32)
+    return y
+
+
+def transcribe_long(asr, y, win=20.0):
+    """Audio over ~20 s: cut at the quietest 100 ms in the last 4 s of each window, transcribe pieces, join."""
+    if len(y) <= win * SR:
+        return asr.transcribe(y)
+    out, s = [], 0
+    while s < len(y):
+        e = min(len(y), s + int(win * SR))
+        if e < len(y):
+            lo = s + int((win - 4) * SR)
+            fr = y[lo:e][: (e - lo) // 1600 * 1600].reshape(-1, 1600)
+            e = lo + int(np.argmin((fr ** 2).mean(1))) * 1600 + 800
+        out.append(asr.transcribe(y[s:e]))
+        s = e
+    return " ".join(t for t in out if t)
+
+
+if __name__ == "__main__":
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("wav", nargs="+")
+    ap.add_argument("--model", default=None, help="path to model_int8.onnx (default) or model_fp32.onnx")
+    ap.add_argument("--vocab", default=None)
+    ap.add_argument("--threads", type=int, default=4)
+    a = ap.parse_args()
+    asr = PocketASR(a.model, a.vocab, a.threads)
+    for p in a.wav:
+        y = load_wav(p)
+        t = time.perf_counter()
+        txt = transcribe_long(asr, y)
+        dt = time.perf_counter() - t
+        print(f"{p}\t{txt}\t[{len(y) / SR:.1f} s audio, {dt * 1000:.0f} ms, RTF {dt / max(len(y) / SR, 1e-3):.3f}]")