| --- |
| license: apache-2.0 |
| language: |
| - ja |
| - en |
| - zh |
| tags: |
| - ocr |
| - manga |
| - onnx |
| - webgpu |
| - on-device |
| - quantized |
| base_model: genshiai-daichi/baberu-ocr |
| pipeline_tag: image-to-text |
| --- |
| |
| # Baberu OCR experimental full-WebGPU ONNX bundles |
|
|
| > **Personal model hosting/download mirror.** These files are hosted so I can |
| > download and test my two experimental browser models. This is not an official |
| > Baberu release, not production ready, and not a recommendation for public |
| > deployment without independent accuracy, browser, memory, and security tests. |
|
|
| This repository contains two complete client-only OCR bundles derived from |
| [Baberu OCR](https://huggingface.co/genshiai-daichi/baberu-ocr), together with |
| the ONNX export, validation, benchmark, and browser WebGPU source. No inference |
| server is required after the browser downloads the files. |
|
|
| ## Latest recommended models |
|
|
| The latest decoder uses one complete six-layer QDQ graph for both prefill and |
| cached token steps. It accepts an INT32 token ID, gathers the selected INT8 |
| embedding row first, and only then runs `DequantizeLinear`. This exact rewrite |
| avoids materializing the full `[14630,512]` FP32 embedding table (28.6 MiB). |
|
|
| | Folder | Vision encoder | Latest decoder | Latest ONNX download | |
| | --- | --- | --- | ---: | |
| | `variants/webgpu-121/` | INT4 `vision_int4.onnx`, 52.29 MB | `decoder_unified_gather_qdq_int8.onnx`, 34.65 MB | **86.94 MB** | |
| | `variants/webgpu-242/` | FP16 `vision_fp16.onnx`, 172.92 MB | `decoder_unified_gather_qdq_int8.onnx`, 34.65 MB | **207.57 MB** | |
|
|
| The smaller downloads come from removing the duplicate decoder weights that |
| were embedded separately in the old prefill and step files. They are not |
| reduced models: all six layers, hidden size 512, intermediate size 1,536, eight |
| attention heads, two KV heads, and the full 14,630-token vocabulary remain. |
| The decoder retains all 33,069,056 quantized weight elements. |
|
|
| Each variant also retains the old split prefill/step graphs for compatibility |
| and `decoder_unified_qdq_int8.onnx` as a one-hot unified baseline. Hugging Face |
| deduplicates identical decoder blobs stored under both variant folders. |
|
|
| ## WebGPU execution path |
|
|
| The vision encoder resizes the complete RGB text crop to 224 x 224, applies |
| ImageNet normalization, and returns `vision_embeds` with shape `[1,256,512]`. |
| The unified decoder consumes those embeddings and keeps all six key/six value |
| cache pairs GPU-resident during generation. |
|
|
| The vision encoder, decoder compute, and KV tensors execute with ONNX Runtime |
| WebGPU. Only the FP32 logits return to JavaScript for the published greedy token |
| selection policy. A GPU ArgMax graph and fused GroupQueryAttention graph were |
| tested locally but removed because both were slower on the tested runtime. |
|
|
| The QDQ decoder stores INT8 weights and executes `DequantizeLinear` followed by |
| FP32 `MatMul` on WebGPU. The token embedding is the exception: its INT8 Gather |
| runs before dequantization. It contains neither `DynamicQuantizeLinear` nor |
| `MatMulInteger`, which would leave the intended browser WebGPU path. |
|
|
| ## 13 difficult showcase-image check |
|
|
| These public Hayai OCR showcase crops are useful conversion checks, but they |
| are a small selected set rather than a neutral accuracy corpus. CER uses NFKC |
| normalization and removes whitespace. |
|
|
| | Variant | Provider | nCER | Exact | Median/crop | |
| | --- | --- | ---: | ---: | ---: | |
| | Upstream native 121 | ONNX Runtime CPU | 13.87% | 5/13 | 197 ms | |
| | Upstream native 242 | ONNX Runtime CPU | 12.61% | 5/13 | 137 ms | |
| | Latest `webgpu-121` Gather | ONNX Runtime WebGPU | **14.29%** | 4/13 | **about 300 ms** | |
| | Latest `webgpu-242` Gather | ONNX Runtime WebGPU | **11.76%** | 5/13 | **190 ms in the latest warm-cache check** | |
|
|
| All 13 outputs from the optimized 121 Gather graph were identical to the |
| previous unified one-hot graph. The 242 Gather graph also retained the earlier |
| 242 WebGPU result of 11.76% nCER and 5/13 exact. Latency depends heavily on GPU, |
| shader cache, browser, and system load; these numbers are not portable. |
|
|
| The Gather-before-dequantize decoder is also bit-exact against the previous |
| token-Gather decoder on ONNX Runtime CPU at prefill, cache length 257, and the |
| maximum tested cache length 383. Three full WebGPU runs per vision tier retained |
| all 13 previous output strings. |
|
|
| In one same-process 121 comparison, the optimized graph sampled an 833 MB |
| shared GPU-process peak and 577 MB renderer peak, versus 866 MB and 613 MB for |
| the unified one-hot baseline. Those values must not be added as model-owned |
| RAM: the maxima were not necessarily simultaneous, the GPU process is shared, |
| and Apple silicon uses unified memory. |
|
|
| Full outputs are in `benchmarks/`. Source images are not redistributed; |
| `download_hayai13.py` downloads the public examples from a pinned upstream |
| revision. |
|
|
| ## Accuracy status |
|
|
| **Full converted-model accuracy remains unknown.** The upstream 121 and 242 |
| releases report Japanese nCER 0.0893 and 0.0867, but those values must not be |
| assigned to these conversions. The exact 2,000-crop reproduction remains |
| blocked on gated Manga109-v2026 data and the unpublished exact crop manifest |
| and evaluation scripts. |
|
|
| The 13-image results and synthetic FP32/QDQ top-token comparisons are port |
| validation, not evidence that these files are production ready. |
|
|
| ## Run the hosted models |
|
|
| Clone the repository with Git LFS/Xet support, then: |
|
|
| ```sh |
| npm install |
| sh run-browser-server.sh |
| ``` |
|
|
| Open the optimized 121-capability bundle: |
|
|
| ```text |
| http://127.0.0.1:8765/webgpu-e2e.html?layout=hf&bundle=compact-unified-gather |
| ``` |
|
|
| Or the optimized 242-capability bundle: |
|
|
| ```text |
| http://127.0.0.1:8765/webgpu-e2e.html?layout=hf&bundle=balanced-unified-gather |
| ``` |
|
|
| Choose a tightly cropped text image. Add `&suite=hayai13` after running |
| `python3 download_hayai13.py` to reproduce the 13-image checks. Both test paths |
| explicitly release their ONNX sessions and destroy the dedicated GPU device |
| when complete. |
|
|
| ## Included port source |
|
|
| - exact FP32 decoder and vision exporters; |
| - complete unified and token-ID Gather decoder exporters; |
| - per-output-channel INT8-QDQ converter and CPU parity checks; |
| - browser WebGPU end-to-end and decoder smoke harnesses; |
| - exact Gather-before-dequantize and experimental execution optimizers; |
| - staged vision-worker and decoder-only cold/warm memory harnesses; |
| - native CPU and MangaOCR comparison harnesses; |
| - isolated Python/npm dependencies and local server; |
| - benchmark helpers and generated validation reports. |
|
|
| Detailed local experiment notes are in `docs/experiment-results.md`. |
|
|
| ## Provenance and license |
|
|
| - Base model: `genshiai-daichi/baberu-ocr` |
| - Base revision: `d9cc13153e9a1cd8fdfa3b7b1cc329da2020aeae` |
| - ONNX opset: 17 |
| - Decoder conversion: symmetric per-output-channel INT8 weight-only QDQ |
| - Browser runtime tested: ONNX Runtime Web 1.27, WebGPU provider |
| - License: Apache-2.0; see `LICENSE` and `NOTICE` |
|
|
| The vision ONNX files are unmodified upstream copies. The QDQ decoder graphs |
| and browser port are an independent experiment, not an official Baberu release. |
|
|
