--- license: apache-2.0 language: - ja - en - zh tags: - ocr - manga - onnx - webgpu - on-device - quantized base_model: genshiai-daichi/baberu-ocr pipeline_tag: image-to-text --- # Baberu OCR experimental full-WebGPU ONNX bundles > **Personal model hosting/download mirror.** These files are hosted so I can > download and test my two experimental browser models. This is not an official > Baberu release, not production ready, and not a recommendation for public > deployment without independent accuracy, browser, memory, and security tests. This repository contains two complete client-only OCR bundles derived from [Baberu OCR](https://huggingface.co/genshiai-daichi/baberu-ocr), together with the ONNX export, validation, benchmark, and browser WebGPU source. No inference server is required after the browser downloads the files. ## Latest recommended models The latest decoder uses one complete six-layer QDQ graph for both prefill and cached token steps. It accepts an INT32 token ID, gathers the selected INT8 embedding row first, and only then runs `DequantizeLinear`. This exact rewrite avoids materializing the full `[14630,512]` FP32 embedding table (28.6 MiB). | Folder | Vision encoder | Latest decoder | Latest ONNX download | | --- | --- | --- | ---: | | `variants/webgpu-121/` | INT4 `vision_int4.onnx`, 52.29 MB | `decoder_unified_gather_qdq_int8.onnx`, 34.65 MB | **86.94 MB** | | `variants/webgpu-242/` | FP16 `vision_fp16.onnx`, 172.92 MB | `decoder_unified_gather_qdq_int8.onnx`, 34.65 MB | **207.57 MB** | The smaller downloads come from removing the duplicate decoder weights that were embedded separately in the old prefill and step files. They are not reduced models: all six layers, hidden size 512, intermediate size 1,536, eight attention heads, two KV heads, and the full 14,630-token vocabulary remain. The decoder retains all 33,069,056 quantized weight elements. Each variant also retains the old split prefill/step graphs for compatibility and `decoder_unified_qdq_int8.onnx` as a one-hot unified baseline. Hugging Face deduplicates identical decoder blobs stored under both variant folders. ## WebGPU execution path The vision encoder resizes the complete RGB text crop to 224 x 224, applies ImageNet normalization, and returns `vision_embeds` with shape `[1,256,512]`. The unified decoder consumes those embeddings and keeps all six key/six value cache pairs GPU-resident during generation. The vision encoder, decoder compute, and KV tensors execute with ONNX Runtime WebGPU. Only the FP32 logits return to JavaScript for the published greedy token selection policy. A GPU ArgMax graph and fused GroupQueryAttention graph were tested locally but removed because both were slower on the tested runtime. The QDQ decoder stores INT8 weights and executes `DequantizeLinear` followed by FP32 `MatMul` on WebGPU. The token embedding is the exception: its INT8 Gather runs before dequantization. It contains neither `DynamicQuantizeLinear` nor `MatMulInteger`, which would leave the intended browser WebGPU path. ## 13 difficult showcase-image check These public Hayai OCR showcase crops are useful conversion checks, but they are a small selected set rather than a neutral accuracy corpus. CER uses NFKC normalization and removes whitespace. | Variant | Provider | nCER | Exact | Median/crop | | --- | --- | ---: | ---: | ---: | | Upstream native 121 | ONNX Runtime CPU | 13.87% | 5/13 | 197 ms | | Upstream native 242 | ONNX Runtime CPU | 12.61% | 5/13 | 137 ms | | Latest `webgpu-121` Gather | ONNX Runtime WebGPU | **14.29%** | 4/13 | **about 300 ms** | | Latest `webgpu-242` Gather | ONNX Runtime WebGPU | **11.76%** | 5/13 | **190 ms in the latest warm-cache check** | All 13 outputs from the optimized 121 Gather graph were identical to the previous unified one-hot graph. The 242 Gather graph also retained the earlier 242 WebGPU result of 11.76% nCER and 5/13 exact. Latency depends heavily on GPU, shader cache, browser, and system load; these numbers are not portable. The Gather-before-dequantize decoder is also bit-exact against the previous token-Gather decoder on ONNX Runtime CPU at prefill, cache length 257, and the maximum tested cache length 383. Three full WebGPU runs per vision tier retained all 13 previous output strings. In one same-process 121 comparison, the optimized graph sampled an 833 MB shared GPU-process peak and 577 MB renderer peak, versus 866 MB and 613 MB for the unified one-hot baseline. Those values must not be added as model-owned RAM: the maxima were not necessarily simultaneous, the GPU process is shared, and Apple silicon uses unified memory. Full outputs are in `benchmarks/`. Source images are not redistributed; `download_hayai13.py` downloads the public examples from a pinned upstream revision. ## Accuracy status **Full converted-model accuracy remains unknown.** The upstream 121 and 242 releases report Japanese nCER 0.0893 and 0.0867, but those values must not be assigned to these conversions. The exact 2,000-crop reproduction remains blocked on gated Manga109-v2026 data and the unpublished exact crop manifest and evaluation scripts. The 13-image results and synthetic FP32/QDQ top-token comparisons are port validation, not evidence that these files are production ready. ## Run the hosted models Clone the repository with Git LFS/Xet support, then: ```sh npm install sh run-browser-server.sh ``` Open the optimized 121-capability bundle: ```text http://127.0.0.1:8765/webgpu-e2e.html?layout=hf&bundle=compact-unified-gather ``` Or the optimized 242-capability bundle: ```text http://127.0.0.1:8765/webgpu-e2e.html?layout=hf&bundle=balanced-unified-gather ``` Choose a tightly cropped text image. Add `&suite=hayai13` after running `python3 download_hayai13.py` to reproduce the 13-image checks. Both test paths explicitly release their ONNX sessions and destroy the dedicated GPU device when complete. ## Included port source - exact FP32 decoder and vision exporters; - complete unified and token-ID Gather decoder exporters; - per-output-channel INT8-QDQ converter and CPU parity checks; - browser WebGPU end-to-end and decoder smoke harnesses; - exact Gather-before-dequantize and experimental execution optimizers; - staged vision-worker and decoder-only cold/warm memory harnesses; - native CPU and MangaOCR comparison harnesses; - isolated Python/npm dependencies and local server; - benchmark helpers and generated validation reports. Detailed local experiment notes are in `docs/experiment-results.md`. ## Provenance and license - Base model: `genshiai-daichi/baberu-ocr` - Base revision: `d9cc13153e9a1cd8fdfa3b7b1cc329da2020aeae` - ONNX opset: 17 - Decoder conversion: symmetric per-output-channel INT8 weight-only QDQ - Browser runtime tested: ONNX Runtime Web 1.27, WebGPU provider - License: Apache-2.0; see `LICENSE` and `NOTICE` The vision ONNX files are unmodified upstream copies. The QDQ decoder graphs and browser port are an independent experiment, not an official Baberu release.