Store the recognizers' and layout model's weights in fp16 (half the download)

Weights are stored as fp16 and cast to float32 at load; ONNX Runtime folds the casts. CPU text identical on 325/325 lines, layout pages identical. lines/ unchanged. Model card updated for netra-ocr 1.2.0.

README.md

@@ -12,7 +12,6 @@ tags:
 - khmer-ocr
 - document-layout
 - onnx
-- netra-runtime
 - netra-lab
 ---
 
@@ -24,56 +23,58 @@ tags:
 
 <p align="center">
   <a href="https://github.com/netra-ai-lab/Netra-OCR"><b>GitHub</b></a> |
-  <a href="https://github.com/netra-ai-lab/Netra-Runtime"><b>Netra Runtime</b></a> |
   <a href="https://pypi.org/project/netra-ocr/"><b>PyPI</b></a> |
   <a href="https://huggingface.co/Darayut/khmer-text-recognition"><b>PyTorch weights</b></a>
 </p>
 
 # Netra OCR
 
-The models behind the Netra OCR app, exported as [Netra Runtime](https://github.com/netra-ai-lab/Netra-Runtime) bundles: ONNX graphs plus a `netra-model.json` manifest that says how to pre-process, decode and post-process. The same bundles run on CPU and CUDA from Python and in the browser (WebGPU / WebAssembly), with no PyTorch.
+The models behind the Netra OCR app, exported to ONNX. Each folder holds the ONNX graphs plus a `netra-model.json` config with the settings the app's pre- and post-processing need (line height and chunk width, decoder shape, class names, thresholds). The [`netra-ocr`](https://pypi.org/project/netra-ocr/) package runs them on ONNX Runtime on CPU and CUDA, and its [`web/`](https://github.com/netra-ai-lab/Netra-OCR/tree/master/web) folder runs the same files in the browser (WebGPU / WebAssembly), with no PyTorch.
 
-Each folder is one bundle:
+Each folder is one model:
 
-| Folder | Task | What it is | Variants |
-|---|---|---|---|
-| `recognition/` | text recognition | Khmer + English line recognizer, 19M params, greedy decoding with a KV cache | – |
-| `recognition-blockwise/` | text recognition | the same recognizer plus a blockwise proposal head (same output, fewer decoder passes) | – |
-| `lines/` | object detection | YOLO text-line and logo detector | `fp16` (CUDA default) |
-| `layout/` | object detection | DocLayout-YOLO (DocStructBench) page-layout analyzer | `fp16` (CUDA default) |
+| Folder | Task | What it is | Download | Variants |
+|---|---|---|---|---|
+| `recognition/` | text recognition | Khmer + English line recognizer, 19M params, greedy decoding with a KV cache | 41 MB | – |
+| `recognition-blockwise/` | text recognition | the same recognizer plus a blockwise proposal head (same output, fewer decoder passes) | 44 MB | – |
+| `lines/` | object detection | YOLO text-line and logo detector | 38 MB (19 MB on CUDA) | `fp16` (CUDA default) |
+| `layout/` | object detection | DocLayout-YOLO (DocStructBench) page-layout analyzer | 39 MB | `fp16` (CUDA default) |
+
+A full page pipeline on the CPU downloads about 118 MB. The recognizers' and the layout model's weights are *stored* in half precision and cast back to float32 when the model loads, so they run in float32 at the same speed; only the weights' rounding changes. On 325 test lines, the text matches the float32 files on 325/325 lines on the CPU and 321/325 on CUDA (on the other 4, the number of errors against the labels is unchanged). The layout model gives identical pages. The line detector stays float32, because half-precision storage moved boxes on 2 of 10 test pages.
 
 ## Usage
 
 ```bash
-pip install "netra-runtime[cpu]"     # or [gpu] for CUDA
+pip install "netra-ocr[cpu]"     # or [gpu] for CUDA
 ```
 
 ```python
-import netra_runtime
+from netra_ocr.inference import Detector, Recognizer
 
-recognizer = netra_runtime.load("hf://Netra-Lab/Netra-OCR/recognition", device="auto")
+recognizer = Recognizer("hf://Netra-Lab/Netra-OCR/recognition")   # device: "auto" (default), "cpu", "cuda"
 print(recognizer("line.png").text)
 
-detector = netra_runtime.load("hf://Netra-Lab/Netra-OCR/lines")
+detector = Detector("hf://Netra-Lab/Netra-OCR/lines")
 for box in detector("page.png"):
     print(box.label, box.score, box.box)
 ```
 
 ```js
-import { load } from "@netra-ai-lab/runtime-web";
-const recognizer = await load("hf://Netra-Lab/Netra-OCR/recognition");
+// web/src/index.js in the Netra-OCR repo
+import { loadRecognizer } from "./web/src/index.js";
+const recognizer = await loadRecognizer("hf://Netra-Lab/Netra-OCR/recognition");
 const { text } = await recognizer.recognize(imageElement);
 ```
 
-`hf://` downloads only the folder you ask for, once, into the Hugging Face cache. For whole pages (detection, layout, reading order, Word/HTML/Markdown export) use the [`netra-ocr`](https://pypi.org/project/netra-ocr/) package or the Netra OCR Docker image, which load these bundles.
+`hf://` downloads only the folder you ask for, and only the files your device runs, once, into the Hugging Face cache. For whole pages (detection, layout, reading order, Word/HTML/Markdown export) use `netra_ocr.KhmerOCRPipeline`, the `netra_ocr serve` web app or the Netra OCR Docker image.
 
 ## Parity and speed
 
 Measured against the PyTorch models on `test_set/khob_eval` (325 lines) and 10 test pages, on an i7-13700KF and an RTX 3060.
 
 - **Recognizer:** text identical to PyTorch greedy decoding on 325/325 lines, on CPU and CUDA, single and batched. Per line: CPU 36.7 → about 13 ms, CUDA 23 → 12.8 ms.
-- **Blockwise:** identical text, 2.32 tokens per decoder pass. Not faster than the KV-cached greedy decoder on CPU or CUDA; faster on WebGPU (64 vs 86 ms per line).
-- **Full page pipeline:** CPU 2.18 → 1.14 s/page, GPU 1.24 → 0.70 s/page. 7 of 10 pages identical to the PyTorch pipeline, 0.43% characters different overall (letterbox resize is defined by the bundle spec rather than OpenCV, which moves about 1% of borderline boxes).
+- **Blockwise:** identical text, 2.32 tokens per decoder pass. Not faster than the KV-cached greedy decoder on CPU, CUDA or WebGPU (82 vs 85 ms per line in Chrome).
+- **Full page pipeline:** CPU 2.18 → 1.14 s/page, GPU 1.24 → 0.70 s/page. 7 of 10 pages identical to the PyTorch pipeline, 0.43% characters different overall (the letterbox resize is Netra OCR's own float64 bilinear resize rather than OpenCV's, which moves about 1% of borderline boxes).
 - **fp16:** default on CUDA for the detectors (DocLayout 65 → 39 ms). Not used on WebGPU, where it is broken in onnxruntime-web 1.31.
 - **Browser:** WebAssembly gives the same text on 325/325 lines and detector boxes at IoU 1.0; WebGPU in Chrome gave the same text on 100/100 lines.
 
@@ -83,11 +84,11 @@ The recognizer's CER is the PyTorch model's: see [Darayut/khmer-text-recognition
 
 | File | Contents |
 |---|---|
-| `*/netra-model.json` | manifest: task, components, graphs, variants, sha256 of every file |
+| `*/netra-model.json` | config: pre/post-processing settings, graphs, precision variants, sha256 of every file |
 | `*/*.onnx` | opset-18 graphs (`encoder`/`decoder_step` for recognition, `model` for detection) |
-| `recognition*/vocab.json` | tokenizer vocabulary (class names of the detectors are in the manifest) |
+| `recognition*/vocab.json` | tokenizer vocabulary (class names of the detectors are in the config) |
 
-The bundle format is specified in [Netra-Runtime/spec/bundle.md](https://github.com/netra-ai-lab/Netra-Runtime/blob/main/spec/bundle.md).
+The Python reader is [`netra_ocr/inference/`](https://github.com/netra-ai-lab/Netra-OCR/tree/master/netra_ocr/inference); the exporters are in [`training/netra_ocr_train/scripts/`](https://github.com/netra-ai-lab/Netra-OCR/tree/master/training/netra_ocr_train/scripts).
 
 ## License and attribution
 

layout/model.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e15787e894a211f64602e4efef59e1179e6d79f12f5b4f0734a00b32daf94e2d
-size 76930373
+oid sha256:87b1b6b983f7120547342ab23d0564fb2b3766f23048a297590d73919aee8383
+size 39405559

layout/netra-model.json

@@ -48,6 +48,6 @@
   },
   "files": {
     "model.fp16.onnx": "6fbbe1a2d71b09f937e50b46b81adad0a54f06fcbae406a0fa3c2aafaa1585f2",
-    "model.onnx": "e15787e894a211f64602e4efef59e1179e6d79f12f5b4f0734a00b32daf94e2d"
+    "model.onnx": "87b1b6b983f7120547342ab23d0564fb2b3766f23048a297590d73919aee8383"
   }
 }

recognition-blockwise/decoder_step.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:73b934b9817f25e03cbd14bdf4260c3145dc9e4347ab77f4d9fd883b1d72279a
-size 33412682
+oid sha256:9333350eb3398dfd41cb9734755493e562d10c4dd221234df06e0ae2a3d3d55c
+size 16785190

recognition-blockwise/encoder.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f58c6a10b9f5631492ded33641ae337c70300ec15f96b74b8b89adbb5c0c3bfb
-size 54400775
+oid sha256:a2b599925350e191c8dbc86126842cef7eb4b621a8e5ef276ce4b5d730beb9c6
+size 27335904

recognition-blockwise/netra-model.json

@@ -44,8 +44,8 @@
     "license": "MIT"
   },
   "files": {
-    "decoder_step.onnx": "73b934b9817f25e03cbd14bdf4260c3145dc9e4347ab77f4d9fd883b1d72279a",
-    "encoder.onnx": "f58c6a10b9f5631492ded33641ae337c70300ec15f96b74b8b89adbb5c0c3bfb",
+    "decoder_step.onnx": "9333350eb3398dfd41cb9734755493e562d10c4dd221234df06e0ae2a3d3d55c",
+    "encoder.onnx": "a2b599925350e191c8dbc86126842cef7eb4b621a8e5ef276ce4b5d730beb9c6",
     "vocab.json": "d55b8578e66dd4069f3abdaf01d0993f8526587dbebec78dcbcd15c514a6268b"
   }
 }

recognition/decoder_step.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c24deade583d3a0085de6ae18ce99b5e2a6ad2f874ef7670bdc57c02c6f8f63e
-size 26311577
+oid sha256:adf53d37dc558f562d2a1eddcde723d36eda2c2d922a6fe3c9864bbe51e12647
+size 13227369

recognition/encoder.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f1c11222ae3e7363618b3e0ce14ad49de6b8332b70502228bf6eb03656e031fd
-size 54400037
+oid sha256:3e6c1b4984cdaf7578d9f591e1c3117df3b90c2a04fb2a18f274b19e4888c475
+size 27335166

recognition/netra-model.json

@@ -43,8 +43,8 @@
     "license": "MIT"
   },
   "files": {
-    "decoder_step.onnx": "c24deade583d3a0085de6ae18ce99b5e2a6ad2f874ef7670bdc57c02c6f8f63e",
-    "encoder.onnx": "f1c11222ae3e7363618b3e0ce14ad49de6b8332b70502228bf6eb03656e031fd",
+    "decoder_step.onnx": "adf53d37dc558f562d2a1eddcde723d36eda2c2d922a6fe3c9864bbe51e12647",
+    "encoder.onnx": "3e6c1b4984cdaf7578d9f591e1c3117df3b90c2a04fb2a18f274b19e4888c475",
     "vocab.json": "d55b8578e66dd4069f3abdaf01d0993f8526587dbebec78dcbcd15c514a6268b"
   }
 }