Final Rizeh reader INT4 conversions; source 51b8c314b055; full-page qualification limits documented

BENCHMARK.md

@@ -0,0 +1,26 @@
+# Reader throughput measurements
+
+Crops/second on the same 300 frozen development crops per reader. Exact-result caches were disabled. This measures preprocessed input to compact CTC outputs, excluding loading, page segmentation, cropping, text assembly and table extraction. The source and ONNX/LiteRT measurements are batch one; tract reports warmed native model execution. Implementation timing boundaries differ slightly; these are indicative measurements, not a controlled cross-runtime ranking.
+
+**Mac conversions and browser validation ran concurrently, so Mac timing numbers are contended.** WASM body/short-label tests used two threads and the other three used one. Browser background scheduling may affect these numbers. Linux CPU and GPU measurements must not be treated as the same device. Native Windows, iOS and Android hardware remain untested.
+
+| Runtime/device | Body | Short labels | Printed | Numeric/math | Degraded |
+|---|---:|---:|---:|---:|---:|
+| Source FP32 / PyTorch2.10 CUDA / RTX5080 Laptop | 210.08 | 210.76 | 211.76 | 211.45 | 210.52 |
+| Linux / RTX5080 Laptop / ORT CUDA1.23.2 | 347.08 | 342.76 | 315.42 | 330.48 | 323.72 |
+| Linux / RTX5080 Laptop / tract CUDA | 230.45 | 242.53 | 225.46 | 222.40 | 216.53 |
+| Linux / CPU / ORT1.31,2 threads | 14.42 | 14.50 | 10.16 | 13.45 | 13.47 |
+| Linux / CPU / LiteRT2.3,2 threads | 43.50 | 45.07 | 42.84 | 42.01 | 43.30 |
+| M2 Mac / Core ML CPU_AND_GPU | 51.39 | 37.93 | 37.90 | 47.45 | 78.94 |
+| M2 Mac / tract CPU | 21.78 | 21.85 | 17.85 | 17.48 | 27.58 |
+| M2 Mac / tract Metal | 27.30 | 27.76 | 14.09 | 20.46 | 43.57 |
+| M2 Mac / ORT1.31 CPU | 8.34 | 8.34 | 5.11 | 5.23 | 7.32 |
+| M2 Mac / ORT1.31 CPU HQQ | 7.69 | 7.67 | 5.34 | 5.27 | 9.59 |
+| M2 Mac / Chrome WebGPU | 8.43 | 8.82 | 8.62 | 7.43 | 5.81 |
+| M2 Mac / Chrome WASM | 0.91 | 0.99 | 0.55 | 0.45 | 0.47 |
+
+All 55 reader/runtime pairs passed the >=98% character-parity gate in every development bucket. The lowest bucket across all pairs was 98.2586%. This is character agreement with the uncompressed model, not exact-line agreement or OCR accuracy.
+
+The pure-W4 full-page handwritten-table diagnostic failed (~86.51% parity on the CPU block16 route). An optional image-only route retained FP32 readers for color-separated handwriting and reached 100% on that example. The three public page diagnostics with that mixed-precision policy reached 100%,99.8895%,100%. No table-grid extraction was exercised (table_grids=0); full document qualification remains open.
+
+See VALIDATION.json for the complete per-reader bucket metrics and diagnostic scope.

MANIFEST.json

@@ -0,0 +1,221 @@
+{
+  "base_revision": "51b8c314b0552b1884c1f0e28016cb12a143f637",
+  "files": {
+    "requirements.txt": {
+      "bytes": 69,
+      "sha256": "9cd44c4193ee2ecb75fa477e10dd43690492120201bf43820151f38030e45df3"
+    },
+    "polarity.py": {
+      "bytes": 1442,
+      "sha256": "f006a8a94626496af91f604489efba22d29cca75621c3766a912e271a5139e9c"
+    },
+    "RECIPE.md": {
+      "bytes": 4075,
+      "sha256": "34dee9f78a3e9639def9c64926b088eb31f5cb6dca22ee54c92b811f10feb80f"
+    },
+    "BENCHMARK.md": {
+      "bytes": 2584,
+      "sha256": "adaab95a33f2e857a2f295a52e1c6fc61c7926c60b28e1e011dcc7865b0c618f"
+    },
+    "README.md": {
+      "bytes": 5015,
+      "sha256": "5f1b76bc9a908613348ea5a004374c659b5d27bf96037a8c2b49a61267929c42"
+    },
+    "quantization_config.json": {
+      "bytes": 2590,
+      "sha256": "da732a8e71b618ca018705b6e0aec17590fc067f7a8f5a801377d07944c5f18a"
+    },
+    "geometry.py": {
+      "bytes": 1922,
+      "sha256": "e8b6e0d5cabeeb5a6eeff6aa2ef9b317f8f34c83e88f0a5f82e17a99f7d0a6cc"
+    },
+    "VALIDATION.json": {
+      "bytes": 140369,
+      "sha256": "245ed6184048bfecefa67b188c0eeb04ea45943713cc2fbb5ddda0832f01c749"
+    },
+    "inference.py": {
+      "bytes": 4595,
+      "sha256": "2fab88114deb001eba8e0cb4cd4f9af7d190867fea5c84245d9ae78981ff712b"
+    },
+    "components.json": {
+      "bytes": 2016,
+      "sha256": "5d49aa0d30d5a7e0eebde35cdf4f149b07e6903d3e1461cf30b22e565ee15be5"
+    },
+    "vocab.json": {
+      "bytes": 20007,
+      "sha256": "d4330d46b65dca6f328fa6191bf02240c819ad83d9216bd9c78c13de895048dc"
+    },
+    "rtl.py": {
+      "bytes": 6097,
+      "sha256": "d5e90e33913d66d1204a2966cc54f42b7b8fd3f37ab476c493c564fe4d020b8b"
+    },
+    "experimental/portable_pipeline.py": {
+      "bytes": 11120,
+      "sha256": "c2d0b4a620b7e04b0ad46766950ead312a59f602a41293e99f7bc85bb174ed97"
+    },
+    "checks/package-smoke.json": {
+      "bytes": 311,
+      "sha256": "f4acb4534946a3ac7f8e6bff6cc25e92c62beb6dce3de7337734c3e82729138d"
+    },
+    "checks/browser-loader-check.json": {
+      "bytes": 338,
+      "sha256": "6681534f6712a591fb71c09e6a32fe49d4ab61d669036400106475ef94696d5e"
+    },
+    "checks/package-litert-smoke.json": {
+      "bytes": 140,
+      "sha256": "f1be9492777c493cf1beeacf99efa53815e0f96718830ebf872787c4f40eeae7"
+    },
+    "checks/browser-decoder-check.json": {
+      "bytes": 84,
+      "sha256": "7aa2b6c50334737c5f22721a7d689f7c45fc032e7dc9d94cbc52baabae4105a0"
+    },
+    "recipe/requirements-macos.txt": {
+      "bytes": 93,
+      "sha256": "8eb76eb39c4a004abd208f8ebe994a72a2669d07ff04fc3214b68ebf57b0d9cd"
+    },
+    "recipe/portable_model.py": {
+      "bytes": 2129,
+      "sha256": "f68eae9b098cd23e38560797f33e986ea922e56c886282b08631fe0d422ee283"
+    },
+    "recipe/export_litert.py": {
+      "bytes": 2807,
+      "sha256": "be7054e54d4d491074b49b294e31a9eab8a854756a96334e5eed8c1b30f4fe13"
+    },
+    "recipe/build_recognizer.py": {
+      "bytes": 5638,
+      "sha256": "1f0229995217d7265bc9cfba67948c2c9fce84922e86b4ceccc41d53e3da2431"
+    },
+    "recipe/requirements-litert.txt": {
+      "bytes": 97,
+      "sha256": "3def04424ae7d7f8023bf6164a16361744a36a6cae79b6ad385a5b0ddfbb017e"
+    },
+    "recipe/source/recognizer.py": {
+      "bytes": 3455,
+      "sha256": "03ffcc439ba4ada082c71b1e08e9bdcc8589332ae1e882557d44f004e3515dd2"
+    },
+    "__pycache__/inference.cpython-311.pyc": {
+      "bytes": 11159,
+      "sha256": "4daf2b018aa95dc40d25ce5db54ce592d1e96de8d95714fd358af32b2053c0e2"
+    },
+    "components/degraded/models/recognizer-w4-block16.mlpackage/Manifest.json": {
+      "bytes": 617,
+      "sha256": "675633649969487f825f5e61559d8fdeec20b22639b384f46f8713d55a3c5fa3"
+    },
+    "components/degraded/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/model.mlmodel": {
+      "bytes": 136655,
+      "sha256": "fbb59f5a96148ffd4cbcda007e715e76bc7b9beed799b9700a4b6dd6f087280a"
+    },
+    "components/degraded/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/weights/weight.bin": {
+      "bytes": 26263424,
+      "sha256": "b300ceb473e7c9e3d97220cb34be29f7b49b030fa8693e58ae1b45710da8f55d"
+    },
+    "components/body/models/recognizer-w4-block16.mlpackage/Manifest.json": {
+      "bytes": 617,
+      "sha256": "2c7c591fa00b408a154d4623215337654358c5b2191ef35934c614f3889ffc4a"
+    },
+    "components/body/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/model.mlmodel": {
+      "bytes": 136655,
+      "sha256": "068b9dd28a4464a544732ae3972b7da23629eb2fdd0434d855cdcec17be71fa9"
+    },
+    "components/body/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/weights/weight.bin": {
+      "bytes": 26263424,
+      "sha256": "45e4f1c7d1d8e0689dbe458d99014bef4ecf1e3f3e308a1399a94dfda1056e17"
+    },
+    "components/short_labels/models/recognizer-w4-block16.mlpackage/Manifest.json": {
+      "bytes": 617,
+      "sha256": "ce7c7bbce2d2f841baf7af009dd827cf0be193bbac12fab75ebbc62eb146f670"
+    },
+    "components/short_labels/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/model.mlmodel": {
+      "bytes": 136655,
+      "sha256": "236f3f7de9ce5e2b2356333e653d996ff335d0bb6acde2c612c7f3190f57631a"
+    },
+    "components/short_labels/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/weights/weight.bin": {
+      "bytes": 26263424,
+      "sha256": "3d5c895f49c743122580765a36e37ab4a5ed6ae88526e940296e949cdd27cdd8"
+    },
+    "components/numeric_math/models/recognizer-w4-block16.mlpackage/Manifest.json": {
+      "bytes": 617,
+      "sha256": "6befae03e35e7c8b0214292bb2c9cfd56946157d4950795c637461cefefaf867"
+    },
+    "components/numeric_math/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/model.mlmodel": {
+      "bytes": 136655,
+      "sha256": "c29f36bd45b9469a594b67e420efde84dfd8d663c138e0f1bee325c7d059debe"
+    },
+    "components/numeric_math/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/weights/weight.bin": {
+      "bytes": 26263424,
+      "sha256": "d15e99ea723d66fedbe3ab66319266a156273f84a352081b69f9650af4802ba1"
+    },
+    "components/printed/models/recognizer-w4-block16.mlpackage/Manifest.json": {
+      "bytes": 617,
+      "sha256": "81016c05409bafc0b082d3b4d3b4a60827adec441c383b41761648f2834b429b"
+    },
+    "components/printed/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/model.mlmodel": {
+      "bytes": 136655,
+      "sha256": "5bfd191c319ff8ede42f41f35ac4ed609d1373c02f97f14b012649d705d08374"
+    },
+    "components/printed/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/weights/weight.bin": {
+      "bytes": 26263424,
+      "sha256": "b7c7da642b5bb0f429cdde70b0ea8815cce86175a1eb8b2a98a1d04c88c40f79"
+    },
+    "experimental-detectors/line_guard/config.json": {
+      "bytes": 858,
+      "sha256": "0f99d8176e3fc194966659cf6f1c52ab9ca1d573f47ba57aca55036ff5267d69"
+    },
+    "experimental-detectors/line_guard/preprocessor_config.json": {
+      "bytes": 373,
+      "sha256": "594344843fcd54db03240194c7677b5c459e78d349e68d4a89d7d91e4fa90587"
+    },
+    "experimental-detectors/line_guard/detector-w4-block16.mlpackage/Manifest.json": {
+      "bytes": 617,
+      "sha256": "893c116ab8685122991388c4ecf30ee0f9c7b05eb290c94f9e29da429585d26f"
+    },
+    "experimental-detectors/line_guard/detector-w4-block16.mlpackage/Data/com.apple.CoreML/model.mlmodel": {
+      "bytes": 183292,
+      "sha256": "3bbc26f64bd5168c8df476ce207d70977a29391ac85fc8745d967f2b573f6064"
+    },
+    "experimental-detectors/line_guard/detector-w4-block16.mlpackage/Data/com.apple.CoreML/weights/weight.bin": {
+      "bytes": 30454400,
+      "sha256": "e9614d48774aae3a1e28ff645664d9b8f654f71c80bad4585432be258875b58c"
+    },
+    "experimental-detectors/printed_detector/config.json": {
+      "bytes": 858,
+      "sha256": "0f99d8176e3fc194966659cf6f1c52ab9ca1d573f47ba57aca55036ff5267d69"
+    },
+    "experimental-detectors/printed_detector/preprocessor_config.json": {
+      "bytes": 373,
+      "sha256": "594344843fcd54db03240194c7677b5c459e78d349e68d4a89d7d91e4fa90587"
+    },
+    "experimental-detectors/printed_detector/detector-w4-block16.mlpackage/Manifest.json": {
+      "bytes": 617,
+      "sha256": "55e73dae6485700c908feedaaafd055315a5b2fa40e2e2bf1ea741984518f173"
+    },
+    "experimental-detectors/printed_detector/detector-w4-block16.mlpackage/Data/com.apple.CoreML/model.mlmodel": {
+      "bytes": 183292,
+      "sha256": "b65e5fc8008ef59de2d91e4c7ec3e44a3d7e037367678564bfd5fb5810b69021"
+    },
+    "experimental-detectors/printed_detector/detector-w4-block16.mlpackage/Data/com.apple.CoreML/weights/weight.bin": {
+      "bytes": 30454400,
+      "sha256": "02ba967fc37e29fe11eca73bdc77334272af1d2533cb9c7004f5eb0eb6d7591d"
+    },
+    "experimental-detectors/main_detector/config.json": {
+      "bytes": 858,
+      "sha256": "0f99d8176e3fc194966659cf6f1c52ab9ca1d573f47ba57aca55036ff5267d69"
+    },
+    "experimental-detectors/main_detector/preprocessor_config.json": {
+      "bytes": 373,
+      "sha256": "594344843fcd54db03240194c7677b5c459e78d349e68d4a89d7d91e4fa90587"
+    },
+    "experimental-detectors/main_detector/detector-w4-block16.mlpackage/Manifest.json": {
+      "bytes": 617,
+      "sha256": "ba24569c2754a72a8d28dd3eeccd666812aca11b6e98bce29b4afcefa820241e"
+    },
+    "experimental-detectors/main_detector/detector-w4-block16.mlpackage/Data/com.apple.CoreML/model.mlmodel": {
+      "bytes": 183292,
+      "sha256": "a7df9637d121d036e2bb2e251b9004240f441d83433cde53226e56a36f6c27a8"
+    },
+    "experimental-detectors/main_detector/detector-w4-block16.mlpackage/Data/com.apple.CoreML/weights/weight.bin": {
+      "bytes": 30454400,
+      "sha256": "0640641e7f2df4053feb5f831e4974739f7839deee0b7c146229db7f10000267"
+    }
+  }
+}

README.md

@@ -0,0 +1,55 @@
+---
+language:
+- fa
+license: other
+license_name: bina-0.4-rizeh
+license_link: https://huggingface.co/Reza2kn/Bina-0.4-Rizeh/blob/51b8c314b0552b1884c1f0e28016cb12a143f637/README.md
+base_model: Reza2kn/Bina-0.4-Rizeh
+base_model_relation: quantized
+pipeline_tag: image-to-text
+tags:
+- bina
+- ocr
+- int4
+- coreml
+---
+
+# Bina0.4 Rizeh — CoreML-INT4
+
+Quantized CTC reader components and experimental line-detector conversions of [the final release](https://huggingface.co/Reza2kn/Bina-0.4-Rizeh/tree/51b8c314b0552b1884c1f0e28016cb12a143f637), pinned at `51b8c314b0552b1884c1f0e28016cb12a143f637`. All five readers are included: body95k, short labels, printed, numeric/math and degraded scans. Original pipeline code and nonconverted dependencies remain in the source repository.
+
+These are mixed-weight4-bit artifacts. They are not an all-INT4 document pipeline. The reference Python pipeline also uses Surya layout/order, native detector paths, table machinery and small admission/ordering models. No Sherpa-ONNX OCR support is claimed.
+
+## Measured reader parity
+
+Fresh uncompressed PyTorch2.10 CUDA baselines were generated for each published checkpoint. Each target uses the same300 frozen internal development crops (50 each: English, handwriting, mixed, newspaper, printed, synthetic). No text normalization is applied to parity. At least98% character parity is required in every bucket; exact-line agreement is reported separately in VALIDATION.json. These are development canaries, not held-out accuracy or full-page certification.
+
+| Runtime | Reader components | Lowest bucket character parity |
+|---|---:|---:|
+| coreml-w4-block16 | 5/5 | 98.466% |
+
+Mac timings were collected during concurrent conversion/browser work and are not controlled speed rankings. Linux CPU/CUDA results are host-specific; native Windows is untested. LiteRT was tested on Linux CPU, without an Android test device. Apple validation used an M2 Mac; iOS hardware was not tested.
+
+## Page-pipeline limitation
+
+The public release demo matched the original text in a small ONNX page test, and exact reader-cache reuse avoided recognition calls on its repeat. A broader public handwritten-table sample exposed a failure: the pure-W4 pipeline changed the output substantially even though line canaries passed. Therefore this bundle does **not** pass the full-page98% transcription gate and is not a drop-in fully qualified4-bit replacement for the production pipeline. Detector mask IoU is not transcription parity.
+
+The experimental Python adapter preserves upstream crop geometry, routing and decoding. It also partitions an upstream geometry-reread cache by actual backend/weight hash, preventing cross-model cache reuse. Its optional image-only safety route uses original FP32 readers on color-separated handwritten pages, before OCR, avoiding a trial quantized pass. That route is mixed-precision and requires separate full-pipeline qualification. It does not make the pure-W4 model pass.
+
+## Line-level use
+
+Install requirements.txt, then run `python inference.py line.png --role body`. This expects one text line or an explicit line box through `Recognizer.read_line`; it does not detect or assemble a whole document. Persistent sessions and bounded exact-input caching are included. Widths are never squashed or truncated.
+
+Inputs are64px-high float32 ink maps with aspect preserved, black=1/paper=0,20% vertical and30% horizontal margins measured in original line height. True widths retain masking semantics. CTC blank is0 and nonblank indices map to vocab.json. Persian visual-to-logical conversion follows the original rtl.py.
+
+Core ML ML Program packages require iOS18/macOS15 or later. Inputs: pixels float32 `[B,1,64,W]`, widths int32 `[B]`; batch1..8 and width8..8192. Use CPU_AND_GPU. W4/block16 applies to compatible large weights, retaining floating operations where required. GPU-only/ANE placement is not claimed.
+
+## Experimental detectors
+
+Three detector roles are included under experimental-detectors/: combined main detector1000, rule-C line guard500 and the printed detector/head. Their procedural-dev heatmaps were compared against their original FP32 models. Their full geometry/table/transcription impact is unqualified. Do not replace the production detectors based on mask IoU alone.
+
+## Provenance and licensing
+
+Quantization did not alter the source repository. The aggregate model.safetensors was verified tensor-for-tensor against the published native core checkpoints. Each component records its exact source checkpoint SHA256 in quantization_config.json. MANIFEST.json hashes the shipped artifacts. No private calibration images, benchmark fixtures, or private prediction texts are shipped.
+
+Original recognizer weights/code remain under the source release CC BY-NC4.0 terms; Surya-derived detector/layout components retain their upstream Surya model terms. These conversions grant no additional rights. Retain the original Bina/Indra/Surya attribution. See the source release model card for details.

RECIPE.md

@@ -0,0 +1,31 @@
+# Conversion recipe
+
+Source: Reza2kn/Bina-0.4-Rizeh, revision51b8c314b0552b1884c1f0e28016cb12a143f637. components.json records the five exact native checkpoint paths and SHA256 values. Verify the checkpoint hash before converting. All five readers have the same architecture and unchanged vocabulary IDs. Body weights matched the previously exported body graph and were reused only after identity verification; every runtime was retested with fresh source baselines.
+
+The recipe/source/recognizer.py file comes from that release. portable_model.py algebraically replaces pointwise Conv1d with Linear, preserving masked squeeze-excitation, true widths and CTC lengths. build_recognizer.py checks FP32 logit/argmax equivalence on varied synthetic shapes before exporting. Compact outputs contain tokens, frame confidence and lengths; callers do not need to transfer full logits.
+
+Use separate environments for Mac and Linux LiteRT. The requirements files list the measured key converter versions, not complete dependency locks. Python3.11 was used on Mac; LiteRT was built in its existing Linux Python3.12 environment. No upstream training environment was changed.
+
+```sh
+# Mac: repeat for each source checkpoint listed in components.json.
+python -m pip install -r recipe/requirements-macos.txt
+python recipe/build_recognizer.py --checkpoint /absolute/path/to/checkpoint.pt --onnx --hqq --coreml --blocks 16
+
+# Separate Linux environment; same source checkpoint.
+python -m pip install -r recipe/requirements-litert.txt
+python recipe/export_litert.py --checkpoint /absolute/path/to/checkpoint.pt
+```
+
+The LiteRT script's reader branch is portable; its legacy --detector branch requires additional detector helpers and the original Stallion checkout, and is not a standalone detector recipe.
+
+ONNX: opset18, dynamic batch/width, ORT1.31 MatMulNBits RTN symmetric4-bit group16, accuracy_level4; HQQ4 group32 is an additional CPU alternative. Eighty-one pointwise linear layers are packed. The stem, depthwise convolutions, biases and other unsupported values stay floating. Graph inputs are float32; accuracy_level4 permits lower internal precision in runtime kernels. HQQ does not require image calibration. The tract importer tested here supports RTN group16 but rejects HQQ floating zero points.
+
+Core ML: coremltools9 ML Program, FLOAT32 compute request, iOS18/macOS15 minimum, batch1..8, width8..8192. Apply linear_symmetric int4 per_block16 with weight_threshold512. Validate CPU_AND_GPU on Mac. Quantization storage does not guarantee a particular GPU/ANE placement.
+
+LiteRT: litert-torch0.10 converts five static batch-one signatures, widths512/1024/2048/4096/8192. True-width masks remain dynamic inputs. ai-edge-quantizer0.10 recipe.dynamic_wi4b32_afp32 is the accepted reader candidate. recipe.weight_only_wi4_afp32 is produced as an experiment, not the qualified reader artifact shipped here. Keep the dynamic4-block32 file and audit real INT4 buffers; about95.5% of stored constant elements are INT4. This denominator is not a trainable-parameter fraction.
+
+Qualification uses the same frozen300 development crops per reader,50 in each of six buckets. Compare every target to a freshly run uncompressed checkpoint without text normalization, requiring character parity>=0.98 in every bucket; report exact-line agreement separately. The frozen private manifest SHA256 is7e7049a3b56323588ae1f6bd38b1753afc2b1149d275c6f0f07c6993be991978. Private images and predictions are deliberately not included. Reproducing this exact canary requires authorized access to those frozen inputs; new public tests do not replace it.
+
+The successful recipes above do not use image calibration. A separate GPTQ experiment used144 frozen training-only calibration crops, did not recover the full-page target, and was not promoted or shipped. No public page was used to fit weights. Full-page pure-W4 qualification failed on a handwritten-table diagnostic; see the model card and VALIDATION.json.
+
+All original recognizer/code and Surya-derived model licensing and attribution still apply.

checks/browser-decoder-check.json

@@ -0,0 +1,6 @@
+{
+  "cases": 1500,
+  "failed": 0,
+  "passed": true,
+  "unicode_version": "13.0.0"
+}

checks/browser-loader-check.json

@@ -0,0 +1,12 @@
+{
+  "backend": "webgpu",
+  "text_matches_measured_candidate": true,
+  "concurrent_duplicate_identical": true,
+  "repeat_identical": true,
+  "repeat_cached": true,
+  "model_calls": 1,
+  "session_creates": 1,
+  "session_reused": true,
+  "model_sha256": "2cb6c60a23a2e9948b9df9679de0f8b2c849dd81bc54abbf4b23f64148edf574",
+  "passed": true
+}

checks/package-litert-smoke.json

@@ -0,0 +1,7 @@
+{
+  "format": "LiteRT-INT4",
+  "text_matches_measured_candidate": true,
+  "repeat_identical": true,
+  "model_calls": 1,
+  "cache_hits": 1
+}

checks/package-smoke.json

@@ -0,0 +1,16 @@
+[
+  {
+    "format": "ONNX-INT4",
+    "text_matches_measured_candidate": true,
+    "repeat_identical": true,
+    "model_calls": 1,
+    "cache_hits": 1
+  },
+  {
+    "format": "CoreML-INT4",
+    "text_matches_measured_candidate": true,
+    "repeat_identical": true,
+    "model_calls": 1,
+    "cache_hits": 1
+  }
+]

components.json

@@ -0,0 +1,67 @@
+[
+  {
+    "role": "short_labels",
+    "path": "stallion/bina04-order-form-batch-20261101/recognizer/weights-001000.pt",
+    "sha256": "367084525dc644ef6d69ad981819af07e3c6a41421359594a54ccc989c2fb90a",
+    "tensor_sha256": "c6c4db8e963a4a7d0c2ee4108972acf6980b688a464b5a184f2f4fa838b0145b",
+    "head": [
+      1625,
+      512,
+      1
+    ],
+    "parameters": 32795225,
+    "state_elements": 32796250
+  },
+  {
+    "role": "body",
+    "path": "stallion/bina04-pagepath-v3-round3-clean-20261031/run/weights-095000.pt",
+    "sha256": "036be44e87c05759c18297426d2d531b0c3ac04af172886071de2ecdd6703981",
+    "tensor_sha256": "f9d6e2479d18deee48d563e46121188bb5b7353ab9725948930ad12261fba51e",
+    "head": [
+      1625,
+      512,
+      1
+    ],
+    "parameters": 32795225,
+    "state_elements": 32796250
+  },
+  {
+    "role": "numeric_math",
+    "path": "stallion/bina04-numeric-math-recognizer-95k-20261102/recognizer/selected.pt",
+    "sha256": "6e3869a5cdebff89ce6cfbbcde8ce2d5a1aa9cfe342679d469d5da08bb82b443",
+    "tensor_sha256": "b77ac34fa6ba5478002c01b0ad702a9a7ad0f8ed91ce11d65b66fd7fb4471e03",
+    "head": [
+      1625,
+      512,
+      1
+    ],
+    "parameters": 32795225,
+    "state_elements": 32796250
+  },
+  {
+    "role": "printed",
+    "path": "stallion/bina04-print-textgate-20261102/run/selected-printed-recognizer.pt",
+    "sha256": "28c4ca8f9fbbc8c578bb130d997598d2e9c2c4a2dad0b22cb5cc06818ecb988a",
+    "tensor_sha256": "8302658e43d391299491dd0d4921dde0cefb35d15c707bc88ae04acf6c608d09",
+    "head": [
+      1625,
+      512,
+      1
+    ],
+    "parameters": 32795225,
+    "state_elements": 32796250
+  },
+  {
+    "role": "degraded",
+    "path": "stallion/bina04-real-degraded-95k-20261103/recognizer/selected.pt",
+    "sha256": "8867b309e881b698685e5b6e06a37c9f495e04dcf7027a044ad44c13cf8abd64",
+    "tensor_sha256": "6cb20a3e2d84233758efaabb90994ac3074d9cc4557deb531c859f0bbb7fe5dc",
+    "head": [
+      1625,
+      512,
+      1
+    ],
+    "parameters": 32795225,
+    "state_elements": 32796250
+  }
+]

components/body/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:068b9dd28a4464a544732ae3972b7da23629eb2fdd0434d855cdcec17be71fa9
+size 136655

components/body/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45e4f1c7d1d8e0689dbe458d99014bef4ecf1e3f3e308a1399a94dfda1056e17
+size 26263424

components/body/models/recognizer-w4-block16.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "7F9E401C-5F1D-40F2-BAB4-296CF4EE443D": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "C8427A65-5938-46B4-A568-974F293FFEB6": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "C8427A65-5938-46B4-A568-974F293FFEB6"
+}

components/degraded/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbb59f5a96148ffd4cbcda007e715e76bc7b9beed799b9700a4b6dd6f087280a
+size 136655

components/degraded/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b300ceb473e7c9e3d97220cb34be29f7b49b030fa8693e58ae1b45710da8f55d
+size 26263424

components/degraded/models/recognizer-w4-block16.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "B81BD9C2-F516-4993-913D-48720F04B6EB": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        },
+        "E39C1CFD-C8EA-42D0-A2BE-458AAA9087ED": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        }
+    },
+    "rootModelIdentifier": "B81BD9C2-F516-4993-913D-48720F04B6EB"
+}

components/numeric_math/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c29f36bd45b9469a594b67e420efde84dfd8d663c138e0f1bee325c7d059debe
+size 136655

components/numeric_math/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d15e99ea723d66fedbe3ab66319266a156273f84a352081b69f9650af4802ba1
+size 26263424

components/numeric_math/models/recognizer-w4-block16.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "D43BAB18-2A03-4550-8D6B-2C5382EB5516": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "F80FD108-F2CF-476D-A813-0F126C5BA6F7": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "F80FD108-F2CF-476D-A813-0F126C5BA6F7"
+}

components/printed/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5bfd191c319ff8ede42f41f35ac4ed609d1373c02f97f14b012649d705d08374
+size 136655

components/printed/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7c7da642b5bb0f429cdde70b0ea8815cce86175a1eb8b2a98a1d04c88c40f79
+size 26263424

components/printed/models/recognizer-w4-block16.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "129E53A2-6ECD-453B-A355-413C35B490C2": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        },
+        "9A7DDB3E-8AEE-49FD-8DB8-5C5D7D37CE2B": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        }
+    },
+    "rootModelIdentifier": "129E53A2-6ECD-453B-A355-413C35B490C2"
+}

components/short_labels/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:236f3f7de9ce5e2b2356333e653d996ff335d0bb6acde2c612c7f3190f57631a
+size 136655

components/short_labels/models/recognizer-w4-block16.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d5c895f49c743122580765a36e37ab4a5ed6ae88526e940296e949cdd27cdd8
+size 26263424

components/short_labels/models/recognizer-w4-block16.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "8B16FAA1-CC03-401E-A352-4E583220EE42": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        },
+        "92B0EED9-262B-44CF-8D73-69D65C318DD5": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        }
+    },
+    "rootModelIdentifier": "8B16FAA1-CC03-401E-A352-4E583220EE42"
+}

experimental-detectors/line_guard/config.json

@@ -0,0 +1,54 @@
+{
+  "_name_or_path": "datalab-to/line_det_2.20",
+  "architectures": [
+    "EfficientViTForSemanticSegmentation"
+  ],
+  "attention_probs_dropout_prob": 0.0,
+  "classifier_dropout_prob": 0.0,
+  "decoder_hidden_size": 512,
+  "decoder_layer_hidden_size": 128,
+  "depths": [
+    1,
+    1,
+    1,
+    6,
+    6
+  ],
+  "head_dim": 32,
+  "hidden_dropout_prob": 0.0,
+  "hidden_sizes": [
+    32,
+    64,
+    160,
+    256
+  ],
+  "initializer_range": 0.02,
+  "layer_norm_eps": 1e-06,
+  "model_type": "efficientvit",
+  "num_channels": 3,
+  "num_classes": 2,
+  "num_stages": 4,
+  "patch_size": [
+    7,
+    7
+  ],
+  "pos_weight": 1.0,
+  "semantic_loss_ignore_index": -1,
+  "strides": [
+    2,
+    2,
+    2,
+    2,
+    2
+  ],
+  "torch_dtype": "float16",
+  "transformers_version": "4.49.0",
+  "use_focal": false,
+  "widths": [
+    32,
+    64,
+    128,
+    256,
+    512
+  ]
+}

experimental-detectors/line_guard/detector-w4-block16.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3bbc26f64bd5168c8df476ce207d70977a29391ac85fc8745d967f2b573f6064
+size 183292

experimental-detectors/line_guard/detector-w4-block16.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9614d48774aae3a1e28ff645664d9b8f654f71c80bad4585432be258875b58c
+size 30454400

experimental-detectors/line_guard/detector-w4-block16.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "B9835E68-04AC-48C8-B053-470F990C6089": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "CAB21B1A-9480-4150-8DBC-1AB7DF83912B": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "CAB21B1A-9480-4150-8DBC-1AB7DF83912B"
+}

experimental-detectors/line_guard/preprocessor_config.json

@@ -0,0 +1,23 @@
+{
+  "do_normalize": true,
+  "do_reduce_labels": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.485,
+    0.456,
+    0.406
+  ],
+  "image_processor_type": "SegformerImageProcessor",
+  "image_std": [
+    0.229,
+    0.224,
+    0.225
+  ],
+  "resample": 2,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 1200,
+    "width": 1200
+  }
+}

experimental-detectors/main_detector/config.json

@@ -0,0 +1,54 @@
+{
+  "_name_or_path": "datalab-to/line_det_2.20",
+  "architectures": [
+    "EfficientViTForSemanticSegmentation"
+  ],
+  "attention_probs_dropout_prob": 0.0,
+  "classifier_dropout_prob": 0.0,
+  "decoder_hidden_size": 512,
+  "decoder_layer_hidden_size": 128,
+  "depths": [
+    1,
+    1,
+    1,
+    6,
+    6
+  ],
+  "head_dim": 32,
+  "hidden_dropout_prob": 0.0,
+  "hidden_sizes": [
+    32,
+    64,
+    160,
+    256
+  ],
+  "initializer_range": 0.02,
+  "layer_norm_eps": 1e-06,
+  "model_type": "efficientvit",
+  "num_channels": 3,
+  "num_classes": 2,
+  "num_stages": 4,
+  "patch_size": [
+    7,
+    7
+  ],
+  "pos_weight": 1.0,
+  "semantic_loss_ignore_index": -1,
+  "strides": [
+    2,
+    2,
+    2,
+    2,
+    2
+  ],
+  "torch_dtype": "float16",
+  "transformers_version": "4.49.0",
+  "use_focal": false,
+  "widths": [
+    32,
+    64,
+    128,
+    256,
+    512
+  ]
+}

experimental-detectors/main_detector/detector-w4-block16.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a7df9637d121d036e2bb2e251b9004240f441d83433cde53226e56a36f6c27a8
+size 183292

experimental-detectors/main_detector/detector-w4-block16.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0640641e7f2df4053feb5f831e4974739f7839deee0b7c146229db7f10000267
+size 30454400

experimental-detectors/main_detector/detector-w4-block16.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "6AD650CC-ED3F-40BF-8974-8446E47C0705": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "78907290-76FC-41EB-B44C-49D23AB10120": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "78907290-76FC-41EB-B44C-49D23AB10120"
+}

experimental-detectors/main_detector/preprocessor_config.json

@@ -0,0 +1,23 @@
+{
+  "do_normalize": true,
+  "do_reduce_labels": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.485,
+    0.456,
+    0.406
+  ],
+  "image_processor_type": "SegformerImageProcessor",
+  "image_std": [
+    0.229,
+    0.224,
+    0.225
+  ],
+  "resample": 2,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 1200,
+    "width": 1200
+  }
+}

experimental-detectors/printed_detector/config.json

@@ -0,0 +1,54 @@
+{
+  "_name_or_path": "datalab-to/line_det_2.20",
+  "architectures": [
+    "EfficientViTForSemanticSegmentation"
+  ],
+  "attention_probs_dropout_prob": 0.0,
+  "classifier_dropout_prob": 0.0,
+  "decoder_hidden_size": 512,
+  "decoder_layer_hidden_size": 128,
+  "depths": [
+    1,
+    1,
+    1,
+    6,
+    6
+  ],
+  "head_dim": 32,
+  "hidden_dropout_prob": 0.0,
+  "hidden_sizes": [
+    32,
+    64,
+    160,
+    256
+  ],
+  "initializer_range": 0.02,
+  "layer_norm_eps": 1e-06,
+  "model_type": "efficientvit",
+  "num_channels": 3,
+  "num_classes": 2,
+  "num_stages": 4,
+  "patch_size": [
+    7,
+    7
+  ],
+  "pos_weight": 1.0,
+  "semantic_loss_ignore_index": -1,
+  "strides": [
+    2,
+    2,
+    2,
+    2,
+    2
+  ],
+  "torch_dtype": "float16",
+  "transformers_version": "4.49.0",
+  "use_focal": false,
+  "widths": [
+    32,
+    64,
+    128,
+    256,
+    512
+  ]
+}

experimental-detectors/printed_detector/detector-w4-block16.mlpackage/Data/com.apple.CoreML/model.mlmodel

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b65e5fc8008ef59de2d91e4c7ec3e44a3d7e037367678564bfd5fb5810b69021
+size 183292

experimental-detectors/printed_detector/detector-w4-block16.mlpackage/Data/com.apple.CoreML/weights/weight.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:02ba967fc37e29fe11eca73bdc77334272af1d2533cb9c7004f5eb0eb6d7591d
+size 30454400

experimental-detectors/printed_detector/detector-w4-block16.mlpackage/Manifest.json

@@ -0,0 +1,18 @@
+{
+    "fileFormatVersion": "1.0.0",
+    "itemInfoEntries": {
+        "73141E41-7FD9-48F3-9AB5-9EB5BB522A59": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Weights",
+            "name": "weights",
+            "path": "com.apple.CoreML/weights"
+        },
+        "CBF54C80-86F8-4AA1-90FB-5CE3AE0B0F28": {
+            "author": "com.apple.CoreML",
+            "description": "CoreML Model Specification",
+            "name": "model.mlmodel",
+            "path": "com.apple.CoreML/model.mlmodel"
+        }
+    },
+    "rootModelIdentifier": "CBF54C80-86F8-4AA1-90FB-5CE3AE0B0F28"
+}

experimental-detectors/printed_detector/preprocessor_config.json

@@ -0,0 +1,23 @@
+{
+  "do_normalize": true,
+  "do_reduce_labels": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.485,
+    0.456,
+    0.406
+  ],
+  "image_processor_type": "SegformerImageProcessor",
+  "image_std": [
+    0.229,
+    0.224,
+    0.225
+  ],
+  "resample": 2,
+  "rescale_factor": 0.00392156862745098,
+  "size": {
+    "height": 1200,
+    "width": 1200
+  }
+}

experimental/portable_pipeline.py

@@ -0,0 +1,176 @@
+"""Use quantized CTC readers inside the pinned, otherwise unchanged Rizeh pipeline.
+
+Install before constructing Rizeh. Existing crop geometry, routing, decoding,
+confidence thresholds and table assembly remain owned by the upstream code.
+"""
+import collections,hashlib,json,math,sys
+from pathlib import Path
+import numpy as np
+
+class CompactCTCOutput:
+    """Expose only CTC consumers used upstream, from already computed outputs.
+
+    This object does not invent logits. Full probability distributions and
+    arbitrary tensor operations are deliberately unavailable.
+    """
+    def __init__(self,tokens,confidence):
+        import torch
+        self.tokens=torch.from_numpy(tokens.astype(np.int64));self.confidence=torch.from_numpy(confidence.astype(np.float32))
+    def float(self):return self
+    def softmax(self,dim):
+        if dim!=-1:raise ValueError('Only original CTC class softmax is available')
+        return self
+    def max(self,dim):
+        if dim!=-1:raise ValueError('Only original CTC class maximum is available')
+        return self.confidence,self.tokens
+    def argmax(self,dim):
+        if dim!=-1:raise ValueError('Only original CTC class argmax is available')
+        return self.tokens
+
+class ReaderModel:
+    def __init__(self,root,role,backend,provider,cache_bytes,variant='block16'):
+        self.root,self.role,self.backend,self.provider=root,role,backend,provider
+        self.cache=collections.OrderedDict();self.used=0;self.capacity=cache_bytes;self.calls=0;self.hits=0
+        self.session=None;self.variant=variant
+        self.high_precision=False;self.precise_session=None;self.source_checkpoint=None
+    def to(self,*args,**kwargs):return self
+    def eval(self):return self
+    def parameters(self):return iter(())
+    def _load(self):
+        if self.session is not None:return
+        d=self.root/'components'/self.role
+        if self.backend=='onnx':
+            import onnxruntime as ort
+            if self.provider not in ort.get_available_providers():raise RuntimeError('Unavailable provider: '+self.provider)
+            name={'hqq32':'recognizer-hqq4-block32.onnx','stable-head':'recognizer-w4-stable-head.onnx','stable-se':'recognizer-w4-stable-se.onnx','accurate':'recognizer-w4-accurate.onnx','fp32':'recognizer-fp32.onnx'}.get(self.variant,'recognizer-w4-block16.onnx')
+            options=ort.SessionOptions();options.intra_op_num_threads=2;options.inter_op_num_threads=1
+            self.session=ort.InferenceSession(str(d/'models'/name),sess_options=options,providers=[self.provider])
+            self.invoke=lambda x,w:self.session.run(None,dict(pixels=x,widths=w))
+        elif self.backend=='coreml':
+            import coremltools as ct
+            self.session=ct.models.MLModel(str(d/'models/recognizer-w4-block16.mlpackage'),compute_units=ct.ComputeUnit.CPU_AND_GPU)
+            def invoke(x,w):
+                o=self.session.predict(dict(pixels=x,widths=w.astype(np.int32)));return o['tokens'],o['confidence'],o['lengths']
+            self.invoke=invoke
+        elif self.backend=='litert':
+            from ai_edge_litert.interpreter import Interpreter
+            self.session=Interpreter(model_path=str(d/'litert/recognizer-dynamic4-block32.tflite'),num_threads=2)
+            runners={}
+            def invoke(x,w):
+                width=x.shape[-1]
+                if width not in runners:runners[width]=self.session.get_signature_runner('width'+str(width))
+                o=runners[width](args_0=x,args_1=w.astype(np.int32));return o['output_0'],o['output_1'],o['output_2']
+            self.invoke=invoke
+        elif self.backend=='source':
+            import torch
+            from recognizer import Recognizer
+            state=torch.load(self.source_checkpoint,map_location='cpu',weights_only=True)['model']
+            device='cuda'if self.provider=='CUDAExecutionProvider'else'cpu'
+            self.session=Recognizer(classes=state['head.weight'].shape[0],gradient_checkpointing=False).eval()
+            self.session.load_state_dict(state);self.session.to(device)
+            def invoke(x,w):
+                with torch.inference_mode():
+                    logits,lengths=self.session(torch.from_numpy(x).to(device),torch.from_numpy(w).to(device));confidence,tokens=logits.float().softmax(-1).max(-1)
+                    return tokens.cpu().numpy(),confidence.cpu().numpy(),lengths.cpu().numpy()
+            self.invoke=invoke
+        else:raise ValueError(self.backend)
+    def __call__(self,pixels,widths):
+        import torch
+        x=pixels.detach().cpu().numpy();w=widths.detach().cpu().numpy().astype(np.int64)
+        if x.ndim!=4 or x.shape[1:3]!=(1,64)or len(w)!=len(x):raise ValueError('Original Bx1x64xW input required')
+        if self.high_precision:
+            if self.precise_session is None:
+                self.precise_session=ReaderModel(self.root,self.role,'source',self.provider,self.capacity,'fp32')
+                self.precise_session.source_checkpoint=self.source_checkpoint
+            return self.precise_session(pixels,widths)
+        keys=[];values={};pending={}
+        for i,n in enumerate(w):
+            if n<1 or n>x.shape[-1]:raise ValueError('Invalid original width')
+            crop=np.ascontiguousarray(x[i,0,:,:n]);key=hashlib.sha256(str(crop.shape).encode()+crop.tobytes()).hexdigest();keys.append(key)
+            if key in self.cache:self.cache.move_to_end(key);values[key]=self.cache[key][0];self.hits+=1
+            elif key not in pending:pending[key]=i
+        self._load()
+        order=sorted(pending,key=lambda k:w[pending[k]])
+        # Core ML is bounded to eight; LiteRT has batch-one signatures.
+        batch=1 if self.backend=='litert'else 8
+        for start in range(0,len(order),batch):
+            group=order[start:start+batch];ids=[pending[k]for k in group];true=w[ids]
+            bucket=max(512,2**int(np.ceil(np.log2(int(max(true))))))
+            if self.backend in ['coreml','litert']and bucket>8192:raise ValueError('Input exceeds validated8192 width; never truncated')
+            canvas=np.zeros((len(ids),1,64,bucket),np.float32)
+            for j,i in enumerate(ids):canvas[j,0,:,:w[i]]=x[i,0,:,:w[i]]
+            tokens,confidence,lengths=self.invoke(canvas,true);self.calls+=1
+            for j,key in enumerate(group):
+                n=int(lengths[j]);value=(np.array(tokens[j,:n],copy=True),np.array(confidence[j,:n],copy=True),n);values[key]=value
+                size=value[0].nbytes+value[1].nbytes+128
+                if size<=self.capacity:
+                    while self.used+size>self.capacity:
+                        _,(_,old)=self.cache.popitem(last=False);self.used-=old
+                    self.cache[key]=(value,size);self.used+=size
+        frames=math.ceil(x.shape[-1]/4);tokens=np.zeros((len(x),frames),np.int32);confidence=np.zeros((len(x),frames),np.float32);lengths=[]
+        for i,key in enumerate(keys):
+            a,b,n=values[key];tokens[i,:n]=a;confidence[i,:n]=b;lengths.append(n)
+        return CompactCTCOutput(tokens,confidence),torch.tensor(lengths,dtype=torch.int64)
+
+def install(root,runtime,backend='onnx',provider='CPUExecutionProvider',cache_bytes=16*1024*1024,variant='block16'):
+    """Return shared model sessions, keyed by exact published checkpoint hash."""
+    root=Path(root);runtime=Path(runtime)
+    sys.path.insert(0,str(runtime/'stallion/indra-scale'))
+    import rtl_reader,torch
+    entries=json.loads((root/'components.json').read_text());lookup={r['sha256']:r['role']for r in entries};sessions={}
+    def initialize(reader,checkpoint,vocabulary,device='cpu',frame_budget=8192):
+        digest=hashlib.sha256(Path(checkpoint).read_bytes()).hexdigest()
+        if digest not in lookup:raise ValueError('Unrecognized published checkpoint: '+digest)
+        role=lookup[digest]
+        chosen='block16'if variant=='stable-se'and role not in ['body','printed']else variant
+        if role not in sessions:sessions[role]=ReaderModel(root,role,backend,provider,cache_bytes,chosen)
+        sessions[role].source_checkpoint=Path(checkpoint)
+        reader.alphabet=json.loads(Path(vocabulary).read_text())['chars'];reader.model=sessions[role];reader.frame_budget=frame_budget;reader.device=torch.device('cpu')
+    rtl_reader.RTLReader.__init__=initialize
+    # Upstream .to('cuda') calls are harmless; inputs stay on the CPU until ORT
+    # transfers them directly, avoiding a redundant Torch CPU->GPU->CPU copy.
+    rtl_reader.RTLReader.device=property(lambda self:torch.device('cpu'),lambda self,value:None)
+    return sessions
+
+def namespace_intermediate_cache(root,sessions=None,policy='quantized-only'):
+    """Partition upstream geometry reread caches by actual backend/weight identity.
+
+    The upstream key includes pixels and geometry but omits reader identity.
+    Sharing it between original/quantized models can silently reuse old OCR.
+    Call after Rizeh construction, before its first image.
+    """
+    root=Path(root);identities=[]
+    for role,model in sorted((sessions or {}).items()):
+        name={'hqq32':'recognizer-hqq4-block32.onnx','stable-head':'recognizer-w4-stable-head.onnx','stable-se':'recognizer-w4-stable-se.onnx','accurate':'recognizer-w4-accurate.onnx','fp32':'recognizer-fp32.onnx'}.get(model.variant,'recognizer-w4-block16.onnx')
+        if model.backend=='onnx':
+            digest=hashlib.sha256((root/'components'/role/'models'/name).read_bytes()).hexdigest()
+        else:digest=model.backend+':'+role+':'+model.variant
+        identities.append([role,model.backend,model.provider,digest])
+    if not identities:identities=['original-pytorch',json.loads((root/'components.json').read_text())]
+    if policy!='quantized-only':
+        for role in sorted(sessions or {}):
+            precise=(sessions or {})[role].source_checkpoint
+            identities.append([role,'safety-original-pytorch',hashlib.sha256(precise.read_bytes()).hexdigest()])
+    key=hashlib.sha256(json.dumps([policy,identities],sort_keys=True).encode()).hexdigest()[:24]
+    module=sys.modules.get('geometry_fix')
+    if module is None:return key
+    original=module.apply_geometry
+    if getattr(original,'_cache_partitioned',False):return key
+    def apply(old,image,reader,expert,cache):return original(old,image,reader,expert,Path(cache)/key)
+    apply._cache_partitioned=True
+    for loaded in list(sys.modules.values()):
+        if loaded is not None and getattr(loaded,'apply_geometry',None)is original:loaded.apply_geometry=apply
+    return key
+
+def select_page_precision(sessions,image):
+    """Image-only safety route, chosen before OCR; no trial quantized pass.
+
+    Color-separated handwritten form pages exposed a pure-W4 parity failure.
+    This optional hybrid policy uses original FP32 readers for that scope.
+    Quantized reader artifacts remain available independently of this policy.
+    """
+    from print_scope import handwriting_guard
+    scope=handwriting_guard(image)
+    fallback=bool(scope.get('protected'))
+    for model in sessions.values():model.high_precision=fallback
+    return dict(policy='color-separated-handwriting-fp32-v1',original_precision=fallback,scope=scope)

geometry.py

@@ -0,0 +1,27 @@
+"""One 64px crop contract for training/inference; width is never clamped or squashed.
+
+Margins and polarity follow the Apache-2.0 Indra rec_preproc implementation.
+Only right-padding batches changes the canvas; widths retain the unpadded size.
+"""
+import math
+import numpy as np
+def crop_line64(gray,box,*,pad_y=.2,pad_x=.3,dark=None):
+ import cv2
+ from polarity import is_dark_line
+ if gray.ndim!=2 or gray.dtype!=np.uint8:raise ValueError('Original uint8 grayscale required')
+ x0,y0,x1,y1=map(float,box[:4])
+ if not all(math.isfinite(v)for v in [x0,y0,x1,y1])or x1<=x0 or y1<=y0:raise ValueError('Invalid line geometry')
+ height=y1-y0;ix0=math.floor(x0-pad_x*height);ix1=math.ceil(x1+pad_x*height);iy0=math.floor(y0-pad_y*height);iy1=math.ceil(y1+pad_y*height);cw,ch=ix1-ix0,iy1-iy0
+ H,W=gray.shape;sx0,sy0,sx1,sy1=max(0,ix0),max(0,iy0),min(W,ix1),min(H,iy1)
+ if dark is None:dark=is_dark_line(gray,box)
+ canvas=np.full((ch,cw),0 if dark else 255,dtype=np.uint8)
+ if sx1>sx0 and sy1>sy0:canvas[sy0-iy0:sy1-iy0,sx0-ix0:sx1-ix0]=gray[sy0:sy1,sx0:sx1]
+ width=max(1,round(cw*64/ch));resized=cv2.resize(canvas,(width,64),interpolation=cv2.INTER_AREA if ch>64 else cv2.INTER_LINEAR).astype(np.float32)/255.
+ ink=resized if dark else 1.-resized
+ receipt=dict(original_WH=[W,H],ink_box=[x0,y0,x1,y1],padded_source_extent=[ix0,iy0,ix1,iy1],padded_source_WH=[cw,ch],resized_WH=[width,64],scale_XY=[width/cw,64/ch],width_clamped=False,aspect_rounding_error_pixels=abs(width-cw*64/ch),dark_polarity=bool(dark))
+ return ink,receipt
+def pad_batch(crops,multiple=8):
+ if not crops or any(c.ndim!=2 or c.shape[0]!=64 for c in crops):raise ValueError('Every recognizer crop must be 64px high')
+ widths=np.array([c.shape[1]for c in crops],dtype=np.int64);padded=math.ceil(int(widths.max())/multiple)*multiple;out=np.zeros((len(crops),1,64,padded),dtype=np.float32)
+ for i,c in enumerate(crops):out[i,0,:,:c.shape[1]]=c
+ return out,widths

inference.py

@@ -0,0 +1,60 @@
+"""Standalone quantized line recognition; the page pipeline is a separate layer."""
+import argparse,collections,hashlib,json
+from pathlib import Path
+import numpy as np
+from rtl import flip
+
+class Recognizer:
+    def __init__(self,root=None,role='body',provider='CPUExecutionProvider',cache_entries=512,variant='block16'):
+        self.root=Path(root or Path(__file__).resolve().parent);self.role=role
+        config=json.loads((self.root/'quantization_config.json').read_text());self.backend=config['format'];self.variant=config.get('default_variant','block16')
+        self.alphabet=json.loads((self.root/'vocab.json').read_text())['chars'];d=self.root/'components'/role
+        self.cache=collections.OrderedDict();self.cache_entries=cache_entries;self.calls=0;self.cache_hits=0
+        if self.backend=='onnx':
+            import onnxruntime as ort
+            if provider not in ort.get_available_providers():raise RuntimeError('Unavailable provider: '+provider)
+            options=ort.SessionOptions();options.intra_op_num_threads=2;options.inter_op_num_threads=1
+            if variant=='hqq32'and provider!='CPUExecutionProvider':raise ValueError('HQQ32 helper is qualified for ORT CPU only')
+            name='recognizer-hqq4-block32.onnx'if variant=='hqq32'else'recognizer-w4-block16.onnx'
+            self.session=ort.InferenceSession(str(d/'models'/name),sess_options=options,providers=[provider]);self.run=lambda x,w:self.session.run(None,dict(pixels=x,widths=w))
+        elif self.backend=='coreml':
+            import coremltools as ct
+            self.session=ct.models.MLModel(str(d/'models/recognizer-w4-block16.mlpackage'),compute_units=ct.ComputeUnit.CPU_AND_GPU)
+            def run(x,w):
+                o=self.session.predict(dict(pixels=x,widths=w.astype(np.int32)));return o['tokens'],o['confidence'],o['lengths']
+            self.run=run
+        elif self.backend=='litert':
+            from ai_edge_litert.interpreter import Interpreter
+            self.session=Interpreter(model_path=str(d/'litert/recognizer-dynamic4-block32.tflite'),num_threads=2);self.runners={}
+            def run(x,w):
+                width=x.shape[-1]
+                if width not in self.runners:self.runners[width]=self.session.get_signature_runner('width'+str(width))
+                o=self.runners[width](args_0=x,args_1=w.astype(np.int32));return o['output_0'],o['output_1'],o['output_2']
+            self.run=run
+        else:raise ValueError(self.backend)
+    def predict_ink(self,ink):
+        ink=np.asarray(ink,dtype=np.float32)
+        if ink.ndim!=2 or ink.shape[0]!=64 or ink.shape[1]<1 or not np.isfinite(ink).all():raise ValueError('Finite float32 ink map [64,true_width] required')
+        key=hashlib.sha256(str(ink.shape).encode()+ink.tobytes()).hexdigest()
+        if key in self.cache:
+            self.cache_hits+=1;self.cache.move_to_end(key);return dict(self.cache[key])
+        width=ink.shape[1];bucket=max(512,2**int(np.ceil(np.log2(width))))
+        if bucket>8192:raise ValueError('Width exceeds the validated8192 bound; never truncate a line')
+        x=np.zeros((1,1,64,bucket),np.float32);x[0,0,:,:width]=ink
+        tokens,confidence,lengths=self.run(x,np.array([width],np.int64));self.calls+=1
+        n=int(lengths[0]);seq=np.asarray(tokens[0,:n]);keep=(seq!=0)&np.r_[True,seq[1:]!=seq[:-1]];ids=seq[keep].tolist()
+        visual=''.join(self.alphabet[int(i)-1]for i in ids)
+        result=dict(text=flip(visual,base_rtl=True),visual_text=visual,confidence=float(np.asarray(confidence[0,:n])[keep].mean())if keep.any()else 0.,role=self.role)
+        if self.cache_entries>0:
+            self.cache[key]=result
+            while len(self.cache)>self.cache_entries:self.cache.popitem(last=False)
+        return dict(result)
+    def read_line(self,image,box=None):
+        from geometry import crop_line64
+        gray=np.asarray(image.convert('L'));box=box or [0,0,gray.shape[1],gray.shape[0]]
+        ink,receipt=crop_line64(gray,box);result=self.predict_ink(ink);result['geometry']=receipt;return result
+
+if __name__=='__main__':
+    p=argparse.ArgumentParser(description=__doc__);p.add_argument('line_image');p.add_argument('--role',default='body',choices=['body','short_labels','printed','numeric_math','degraded']);p.add_argument('--provider',default='CPUExecutionProvider');p.add_argument('--variant',default='block16',choices=['block16','hqq32']);a=p.parse_args()
+    from PIL import Image
+    reader=Recognizer(role=a.role,provider=a.provider,variant=a.variant);print(json.dumps(reader.read_line(Image.open(a.line_image)),ensure_ascii=False,indent=2))

polarity.py

@@ -0,0 +1,37 @@
+import math
+import numpy as np
+
+def _percentiles(b: np.ndarray, qs=(0.02, 0.5, 0.98)) -> list[float]:
+    """np.percentile(b, 100 * qs) (linear) of a uint8 array, from its histogram:
+    no sort of the pixels."""
+    n = b.size
+    cum = np.cumsum(np.bincount(b.ravel(), minlength=256))
+    out = []
+    for q in qs:
+        x = q * (n - 1)
+        i = math.floor(x)
+        g = x - i
+        lo = float(np.searchsorted(cum, i, side="right"))
+        hi = float(np.searchsorted(cum, min(i + 1, n - 1), side="right"))
+        d = hi - lo
+        # numpy's lerp: from the nearer end
+        out.append(hi - d * (1 - g) if g >= 0.5 else lo + d * g)
+    return out
+
+def is_dark_line(gray: np.ndarray, box) -> bool:
+    """Light text on a dark ground (a banner, a dark table head, a slide): the
+    ground (the median of the line box) lies nearer the dark end than the text
+    does, so the bright tail is the longer one.  A shadowed line keeps dark text
+    on a dimmer ground.  The recognizers read dark text on light paper, so a
+    light-on-dark line is read inverted."""
+    H, W = gray.shape
+    x0, y0, x1, y1 = (float(v) for v in box[:4])
+    b = gray[
+        max(0, int(y0)) : min(H, int(y1) + 1), max(0, int(x0)) : min(W, int(x1) + 1)
+    ]
+    if not b.size:
+        return False
+    lo, med, hi = (
+        _percentiles(b) if b.dtype == np.uint8 else np.percentile(b, (2, 50, 98))
+    )
+    return bool(hi - med > 2 * (med - lo) + 8)

quantization_config.json

@@ -0,0 +1,79 @@
+{
+  "format": "coreml",
+  "base_model": "Reza2kn/Bina-0.4-Rizeh",
+  "base_revision": "51b8c314b0552b1884c1f0e28016cb12a143f637",
+  "weight_bits": 4,
+  "mixed_precision": true,
+  "reader_components": [
+    {
+      "role": "short_labels",
+      "path": "stallion/bina04-order-form-batch-20261101/recognizer/weights-001000.pt",
+      "sha256": "367084525dc644ef6d69ad981819af07e3c6a41421359594a54ccc989c2fb90a",
+      "tensor_sha256": "c6c4db8e963a4a7d0c2ee4108972acf6980b688a464b5a184f2f4fa838b0145b",
+      "head": [
+        1625,
+        512,
+        1
+      ],
+      "parameters": 32795225,
+      "state_elements": 32796250
+    },
+    {
+      "role": "body",
+      "path": "stallion/bina04-pagepath-v3-round3-clean-20261031/run/weights-095000.pt",
+      "sha256": "036be44e87c05759c18297426d2d531b0c3ac04af172886071de2ecdd6703981",
+      "tensor_sha256": "f9d6e2479d18deee48d563e46121188bb5b7353ab9725948930ad12261fba51e",
+      "head": [
+        1625,
+        512,
+        1
+      ],
+      "parameters": 32795225,
+      "state_elements": 32796250
+    },
+    {
+      "role": "numeric_math",
+      "path": "stallion/bina04-numeric-math-recognizer-95k-20261102/recognizer/selected.pt",
+      "sha256": "6e3869a5cdebff89ce6cfbbcde8ce2d5a1aa9cfe342679d469d5da08bb82b443",
+      "tensor_sha256": "b77ac34fa6ba5478002c01b0ad702a9a7ad0f8ed91ce11d65b66fd7fb4471e03",
+      "head": [
+        1625,
+        512,
+        1
+      ],
+      "parameters": 32795225,
+      "state_elements": 32796250
+    },
+    {
+      "role": "printed",
+      "path": "stallion/bina04-print-textgate-20261102/run/selected-printed-recognizer.pt",
+      "sha256": "28c4ca8f9fbbc8c578bb130d997598d2e9c2c4a2dad0b22cb5cc06818ecb988a",
+      "tensor_sha256": "8302658e43d391299491dd0d4921dde0cefb35d15c707bc88ae04acf6c608d09",
+      "head": [
+        1625,
+        512,
+        1
+      ],
+      "parameters": 32795225,
+      "state_elements": 32796250
+    },
+    {
+      "role": "degraded",
+      "path": "stallion/bina04-real-degraded-95k-20261103/recognizer/selected.pt",
+      "sha256": "8867b309e881b698685e5b6e06a37c9f495e04dcf7027a044ad44c13cf8abd64",
+      "tensor_sha256": "6cb20a3e2d84233758efaabb90994ac3074d9cc4557deb531c859f0bbb7fe5dc",
+      "head": [
+        1625,
+        512,
+        1
+      ],
+      "parameters": 32795225,
+      "state_elements": 32796250
+    }
+  ],
+  "default_variant": "block16",
+  "coreml_minimum": "iOS18/macOS15",
+  "qualified_widths": "true-width mask; canvas powers of two512..8192; no truncation",
+  "full_pipeline_all_int4": false,
+  "full_pipeline_transcription_qualified": false
+}

recipe/build_recognizer.py

@@ -0,0 +1,87 @@
+import argparse, hashlib, json, sys, time
+from pathlib import Path
+import numpy as np
+import torch
+from portable_model import PortableRecognizer, Compact
+
+R = Path(__file__).resolve().parent
+sys.path.insert(0, str(R/'source'))
+from recognizer import Recognizer
+
+def load(path):
+    state = torch.load(path, map_location='cpu', weights_only=True)['model']
+    source = Recognizer(classes=state['head.weight'].shape[0], gradient_checkpointing=False).eval()
+    source.load_state_dict(state)
+    return source
+
+def main():
+    p=argparse.ArgumentParser()
+    p.add_argument('--checkpoint',type=Path,default=R/'source/recognizer-095000.pt')
+    p.add_argument('--out',type=Path,default=R/'models')
+    p.add_argument('--coreml',action='store_true')
+    p.add_argument('--onnx',action='store_true')
+    p.add_argument('--hqq',action='store_true')
+    p.add_argument('--blocks',default='16,32,64')
+    a=p.parse_args();a.out.mkdir(parents=True,exist_ok=True)
+    torch.set_num_threads(2);torch.manual_seed(18404)
+    source=load(a.checkpoint);model=PortableRecognizer(source).eval()
+    controls=[]
+    with torch.inference_mode():
+        for widths in [[1,7,64],[63,257,511],[769,1025,1537]]:
+            w=torch.tensor(widths,dtype=torch.int64);x=torch.rand(len(w),1,64,((max(widths)+7)//8)*8)
+            for i,n in enumerate(widths):x[i,:,:,n:]=0
+            y,n=source(x,w);z,m=model(x,w)
+            error=float((y-z).abs().max()); agreement=float((y.argmax(-1)==z.argmax(-1)).float().mean())
+            # Near-empty synthetic crops amplify fp32 summation-order noise in SE.
+            tolerance = .02 if min(widths)<8 else 2e-4
+            assert error < tolerance and agreement == 1.0 and torch.equal(n,m), (widths,error,agreement)
+            controls.append(dict(widths=widths,max_abs_logit_error=error,frame_argmax_agreement=agreement))
+    identity=dict(checkpoint=str(a.checkpoint.resolve()),sha256=hashlib.sha256(a.checkpoint.read_bytes()).hexdigest(),
+        parameters=sum(t.numel()for t in source.parameters()),lowering_controls=controls,
+        torch=torch.__version__,output_contract=['tokens:int32','confidence:float32','lengths:int32'])
+    (a.out/'source.json').write_text(json.dumps(identity,indent=2)+'\n')
+    compact=Compact(model).eval();x=torch.zeros(1,1,64,512);w=torch.tensor([512])
+    if a.onnx:
+        import onnx
+        from onnxruntime.quantization.matmul_nbits_quantizer import MatMulNBitsQuantizer,DefaultWeightOnlyQuantConfig
+        from onnxruntime.quantization.quant_utils import load_model_with_shape_infer
+        fp=a.out/'recognizer-fp32.onnx'
+        torch.onnx.export(compact,(x,w),str(fp),opset_version=18,dynamo=False,
+            input_names=['pixels','widths'],output_names=['tokens','confidence','lengths'],
+            dynamic_axes={'pixels':{0:'batch',3:'width'},'widths':{0:'batch'},
+                          'tokens':{0:'batch',1:'frames'},'confidence':{0:'batch',1:'frames'},'lengths':{0:'batch'}})
+        graph=onnx.load(fp);onnx.checker.check_model(graph)
+        for block in map(int,a.blocks.split(',')):
+            q=MatMulNBitsQuantizer(load_model_with_shape_infer(fp),
+                algo_config=DefaultWeightOnlyQuantConfig(block_size=block,is_symmetric=True,accuracy_level=4))
+            q.process();target=a.out/f'recognizer-w4-block{block}.onnx'
+            q.model.save_model_to_file(str(target),use_external_data_format=False)
+            graph=onnx.load(target);onnx.checker.check_model(graph)
+            counts={}
+            for node in graph.graph.node:counts[node.op_type]=counts.get(node.op_type,0)+1
+            (target.with_suffix('.audit.json')).write_text(json.dumps(dict(bytes=target.stat().st_size,
+                ops=counts,weight_bits=4,graph_input_bits=32,accuracy_level=4,retained_float_ops=['stem','depthwise convolutions','biases'],
+                format='ORT MatMulNBits W4 with runtime-dependent internal precision; mixed-weight precision, not all-INT4'),indent=2)+'\n')
+        if a.hqq:
+            from onnxruntime.quantization.matmul_nbits_quantizer import HQQWeightOnlyQuantConfig
+            q=MatMulNBitsQuantizer(load_model_with_shape_infer(fp),algo_config=HQQWeightOnlyQuantConfig(block_size=32,bits=4))
+            q.process();q.model.save_model_to_file(str(a.out/'recognizer-hqq4-block32.onnx'),use_external_data_format=False)
+    if a.coreml:
+        import coremltools as ct
+        from coremltools.optimize.coreml import OptimizationConfig,OpLinearQuantizerConfig,linear_quantize_weights
+        # Dynamic width with exact mask semantics; small bounded batch for mobile.
+        traced=torch.jit.trace(compact,(x,w))
+        ml=ct.convert(traced,inputs=[ct.TensorType(name='pixels',shape=(ct.RangeDim(1,8,default=1),1,64,ct.RangeDim(8,8192,default=512))),
+            ct.TensorType(name='widths',shape=(ct.RangeDim(1,8,default=1),),dtype=np.int32)],
+            outputs=[ct.TensorType(name='tokens'),ct.TensorType(name='confidence'),ct.TensorType(name='lengths')],
+            minimum_deployment_target=ct.target.iOS18,compute_precision=ct.precision.FLOAT32,
+            compute_units=ct.ComputeUnit.CPU_AND_GPU)
+        ml.save(str(a.out/'recognizer-fp32.mlpackage'))
+        for block in map(int,a.blocks.split(',')):
+            config=OptimizationConfig(global_config=OpLinearQuantizerConfig(mode='linear_symmetric',dtype='int4',
+                granularity='per_block',block_size=block,weight_threshold=512))
+            quant=linear_quantize_weights(ml,config=config)
+            quant.save(str(a.out/f'recognizer-w4-block{block}.mlpackage'))
+    print(json.dumps(identity),flush=True)
+
+if __name__=='__main__':main()

recipe/export_litert.py

@@ -0,0 +1,45 @@
+"""Linux CPU exporter; use an isolated venv, never the training environment."""
+import os
+os.environ.update(CUDA_VISIBLE_DEVICES='',OMP_NUM_THREADS='2',OPENBLAS_NUM_THREADS='1',TF_NUM_INTRAOP_THREADS='2',TF_NUM_INTEROP_THREADS='1')
+import argparse,json,time,sys
+from pathlib import Path
+import torch
+import litert_torch
+from ai_edge_quantizer import quantizer,recipe
+from build_recognizer import load,R
+from portable_model import PortableRecognizer,Compact
+
+def main():
+    parser=argparse.ArgumentParser();parser.add_argument('--detector',action='store_true');parser.add_argument('--checkpoint',type=Path);args=parser.parse_args()
+    torch.set_num_threads(2);out=R/'litert';out.mkdir(exist_ok=True)
+    if args.detector:
+        sys.path.append('/home/rezo/bina02-surya-layout-env/lib/python3.10/site-packages')
+        from export_detector import EfficientViTConfig,EfficientViTForSemanticSegmentation,BASE,S,Model
+        m=EfficientViTForSemanticSegmentation(EfficientViTConfig.from_pretrained(str(BASE))).eval()
+        m.load_state_dict(torch.load(args.checkpoint or S/'run/checkpoints/step-001000.pt',map_location='cpu',weights_only=True)['model'])
+        converter=litert_torch.signature('detector',Model(m).eval(),(torch.zeros(1,3,1200,1200),))
+        prefix='detector'
+    else:
+        model=Compact(PortableRecognizer(load(args.checkpoint or R/'source/recognizer-095000.pt'))).eval()
+        model.model.index_dtype=torch.int32
+        converter=None
+        for width in [512,1024,2048,4096,8192]:
+            x=torch.zeros(1,1,64,width);w=torch.tensor([width])
+            if converter is None:converter=litert_torch.signature('width'+str(width),model,(x,w))
+            else:converter.signature('width'+str(width),model,(x,w))
+        prefix='recognizer'
+    with torch.no_grad():edge=converter.convert()
+    fp=out/(prefix+'-fp32.tflite');edge.export(str(fp))
+    results=[]
+    for name,rp in [('weight-only4',recipe.weight_only_wi4_afp32()),('dynamic4-block32',recipe.dynamic_wi4b32_afp32())]:
+        t=time.monotonic()
+        try:
+            q=quantizer.Quantizer(str(fp));q.load_quantization_recipe(rp)
+            target=out/f'{prefix}-{name}.tflite';q.quantize().export_model(str(target))
+            results.append(dict(name=name,path=str(target),bytes=target.stat().st_size,recipe=rp,seconds=time.monotonic()-t))
+        except Exception as e:results.append(dict(name=name,error=repr(e)))
+        (out/(prefix+'-build.json')).write_text(json.dumps(dict(converter='litert-torch 0.10.0',quantizer='ai-edge-quantizer 0.10.0',
+            input='Five static width signatures512/1024/2048/4096/8192; true widths input; zero right pad, never truncate',results=results),indent=2,default=str)+'\n')
+    print(json.dumps(results,default=str),flush=True)
+
+if __name__=='__main__':main()

recipe/portable_model.py

@@ -0,0 +1,53 @@
+"""Algebraic pointwise-Conv -> Linear lowering; unchanged masked SE semantics."""
+import copy
+import torch
+from torch import nn
+from torch.nn import functional as F
+
+def linear(conv):
+    layer = nn.Linear(conv.in_channels, conv.out_channels)
+    with torch.no_grad():
+        layer.weight.copy_(conv.weight[..., 0])
+        layer.bias.copy_(conv.bias)
+    return layer
+
+class PortableBlock(nn.Module):
+    def __init__(self, block):
+        super().__init__()
+        self.dw = copy.deepcopy(block.dw)
+        for name in ['expand', 'fc1', 'fc2', 'project']:
+            setattr(self, name, linear(getattr(block, name)))
+
+    def forward(self, x, mask):
+        z = F.hardswish(self.expand(self.dw(x.transpose(1,2)).transpose(1,2)))
+        pool = (z * mask).sum(1, keepdim=True) / mask.sum(1, keepdim=True).clamp_min(1.)
+        gate = torch.sigmoid(self.fc2(F.relu(self.fc1(pool))))
+        return (x + self.project(z * gate)) * mask
+
+class PortableRecognizer(nn.Module):
+    """Input widths are true unpadded widths; canvas must be a multiple of eight."""
+    def __init__(self, source):
+        super().__init__()
+        self.index_dtype = torch.int64
+        self.stem = copy.deepcopy(source.stem)
+        self.bn = copy.deepcopy(source.bn)
+        self.blocks = nn.ModuleList(PortableBlock(b) for b in source.blocks)
+        self.head = linear(source.head)
+
+    def forward(self, pixels, widths):
+        z = self.stem(F.pad(pixels, (0,4,0,0))).squeeze(2)
+        lengths = (widths + 3) // 4
+        mask = (torch.arange(z.shape[-1], device=z.device, dtype=self.index_dtype)[None,:] < lengths[:,None])[:,:,None].to(z.dtype)
+        z = F.hardswish(self.bn(z)).transpose(1,2) * mask
+        for block in self.blocks:
+            z = block(z, mask)
+        return self.head(z), lengths
+
+class Compact(nn.Module):
+    def __init__(self, source):
+        super().__init__()
+        self.model = source
+    def forward(self, pixels, widths):
+        logits, lengths = self.model(pixels, widths)
+        confidence, tokens = logits.softmax(-1).max(-1)
+        return tokens.to(torch.int32), confidence, lengths.to(torch.int32)

recipe/requirements-litert.txt

@@ -0,0 +1,5 @@
+torch==2.10.0
+numpy==1.26.4
+litert-torch==0.10.0
+ai-edge-quantizer==0.10.0
+ai-edge-litert==2.3.0