Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,216 @@
+---
+license: apache-2.0
+tags:
+- executorch
+- xnnpack
+- pte
+- on-device
+- image-to-text
+base_model:
+- kha-white/manga-ocr-base
+---
+# manga-ocr — ExecuTorch (Japanese text out of a picture of Japanese text)
+
+Hand one line of Japanese to this and get the characters back. Vertical or horizontal,
+furigana and all, printed or laid over artwork. It **generates** the characters rather
+than classifying them: a ViT-base encoder reads the crop once, and a two-layer BERT
+decoder emits one character at a time until it stops.
+
+The shelf's other OCR is docTR, which reads Latin script in two stages — find the boxes,
+then read each crop. This one is the second stage for Japanese, and it is the only shelf
+model that reads vertical text.
+
+```
+manga_ocr_encoder  pixel_values (1, 3, 224, 224) fp32
+    -> encoder_hidden (1, 197, 768) fp32
+manga_ocr_decoder  input_ids (1, L) int64, encoder_hidden (1, 197, 768) fp32
+    -> logits (1, 6144) fp32      # the next character, at the last position only
+```
+
+- **Source**: [kha-white/manga-ocr-base](https://huggingface.co/kha-white/manga-ocr-base),
+  trained on Manga109s — 111.0M parameters, 6,144-character vocabulary
+- **License**: Apache-2.0
+- **Reference implementation**: [kha-white/manga_ocr](https://github.com/kha-white/manga_ocr)
+
+## Files
+
+Pick one row. The two halves must match: the encoder's hidden state is what the decoder
+was measured against.
+
+| build | encoder | decoder | encoder speed | decoder speed |
+|---|---|---|---|---|
+| **Core ML + XNNPACK int8** | `manga_ocr_encoder_coreml_all.pte` **172.1 MB** | `manga_ocr_decoder_xnnpack_int8.pte` **45.0 MB** | **4.1 ms** | 2.6 ms |
+| XNNPACK int8 | `manga_ocr_encoder_xnnpack_int8.pte` 88.9 MB | `manga_ocr_decoder_xnnpack_int8.pte` 45.0 MB | 33.6 ms | 2.6 ms |
+| XNNPACK fp32 | `manga_ocr_encoder_xnnpack_fp32.pte` 343.4 MB | `manga_ocr_decoder_xnnpack_fp32.pte` 117.4 MB | 36.0 ms | 3.0 ms |
+| XNNPACK fp16 | `manga_ocr_encoder_xnnpack_fp16.pte` 173.1 MB | `manga_ocr_decoder_xnnpack_fp16.pte` 58.8 MB | 69.5 ms | 4.6 ms |
+
+Host numbers, on a loaded machine, decoder timed at a 3-character prefix — read them as a
+lower bound and as ratios rather than as device latency. Eager fp32 on the same host is
+29.2 ms for the encoder and 4.4 ms for the decoder.
+
+**fp16 is the row nothing should pick.** It is the same size as the Core ML encoder and
+half the speed of fp32 on the encoder, because XNNPACK inserts a cast at every boundary
+it cannot fuse. It is listed because it was built and measured, not because it wins
+anything.
+
+**int8 is the row to pick on Android**: half of fp16 on the encoder and three-quarters of
+it on the decoder, faster than fp32 on both, and no character different from eager
+anywhere it was measured. What that costs is margin, not characters — see below.
+
+## Running it
+
+**1. Preprocess — the checkpoint's recipe, not PIL's defaults.**
+
+```python
+image = image.convert("L").convert("RGB").resize((224, 224), processor.resample)
+mean = np.array(processor.image_mean, dtype=np.float32)   # 0.5, 0.5, 0.5
+std = np.array(processor.image_std, dtype=np.float32)     # 0.5, 0.5, 0.5
+x = (np.asarray(image, dtype=np.float32) / 255.0 - mean) / std
+pixel_values = torch.from_numpy(np.ascontiguousarray(x.transpose(2, 0, 1)))[None]
+```
+
+Three details that are each load-bearing. **Greyscale and back to RGB** — the reference
+pipeline does this, so a colour page and a scan of the same panel give the same tensor.
+**`resample` from the config, which is bilinear** — `Image.resize` defaults to bicubic,
+and a filter the model was not trained under changes what it reads. **The aspect ratio is
+squashed to a square on purpose**: 224x224 regardless of whether the line is a wide
+horizontal strip or a tall vertical one, which is what the model saw in training.
+
+**2. The encoder, once per crop. The decoder, once per character.**
+
+```python
+hidden = encoder.execute([pixel_values])[0]
+ids = torch.tensor([[2]])                  # decoder_start_token_id
+for _ in range(300):
+    token = int(decoder.execute([ids, hidden])[0][0].argmax())
+    if token == 3:                         # eos_token_id
+        break
+    ids = torch.cat([ids, torch.tensor([[token]])], dim=1)
+```
+
+**3. Decode with `vocab.txt` — there is no tokenizer to install.** The checkpoint names
+`BertJapaneseTokenizer`, whose MeCab step exists only to split *text* into words, and this
+model never reads text. `subword_tokenizer_type` is `character` and the vocabulary holds
+no `##` continuations, so a character is a line of `vocab.txt` and decoding is a lookup
+and a join:
+
+```python
+vocab = open("vocab.txt", encoding="utf-8").read().splitlines()
+text = "".join(vocab[i] for i in ids[0, 1:].tolist()
+               if vocab[i] not in {"[PAD]", "[UNK]", "[CLS]", "[SEP]", "[MASK]"})
+```
+
+Worth knowing because transformers 5.x cannot instantiate that tokenizer class at all —
+it has dropped the slow tokenizers. Reading the vocabulary directly is not a workaround,
+it is what an app on the device does anyway.
+
+## Why there is no KV cache
+
+The decoder is **two** layers over a 6,144-entry vocabulary. Re-running the whole prefix
+each step is cheap enough that a cache would buy less than it costs: it would turn one
+method with a dynamic length into a cache protocol the caller has to hold, allocate and
+reset, and it would pin a maximum length into the file.
+
+The decoder also **returns only the last row of logits**. The other L-1 rows are the model
+re-deriving characters the caller already has; dropping them keeps 6,144 floats crossing
+the boundary per step instead of L times that.
+
+## Greedy, not beam search
+
+The checkpoint's own generation config asks for 4 beams. Greedy is what the graph is
+shaped for, and it is what these files were measured with. On the shelf's gate images
+greedy and beam-4 read the same string. Beam search remains possible from outside — the
+decoder is a pure function of `(prefix, hidden)`, so a caller can run it once per beam per
+step — but nothing here has measured what that buys.
+
+## Core ML: the encoder yes, the decoder no
+
+**The encoder is the whole speed story**: 100% delegated, 4.1 ms against 29.2 ms for eager
+on the same host, at 172.1 MB. Worst-output correlation against fp32 eager is 0.999985.
+
+**The decoder is not shipped for Core ML, and the reason is specific.** It converts, it
+delegates 100%, and — once eight zero-element constants that transformers' BERT attention
+leaves behind are pruned — it loads. What it will not do is run at a length other than the
+one it was exported at: `execute() failed with error 0x32`. A decoder whose prefix grows by
+one character per step is exactly the case that needs the dynamic axis, so the decoder
+stays on XNNPACK, where it runs at every length from 1 to 300. This is not "Core ML rejects
+dynamic shapes" — it accepts the export and pins the shape.
+
+## What was measured
+
+The gate is **the string that came out**, not correlation. The encoder's output is a hidden
+state nobody reads, and the decoder's output is one row of logits whose argmax is all that
+survives — a correlation of 0.999 and a different character are the same file.
+
+**84 generated lines**: 12 sentences x 4 typefaces (Hiragino gothic W3 and W6, rounded
+gothic, mincho) x both orientations x 7 degradations, from clean through gaussian blur,
+sensor noise, low contrast, heavy smear, heavy grain, and a near-contrastless JPEG at
+quality 12.
+
+The set is generated deterministically, so the numbers below reproduce run to run.
+
+| build | lines identical to eager | mean CER vs eager |
+|---|---|---|
+| XNNPACK fp32 | 84 / 84 | 0.0000 |
+| XNNPACK fp16 | 84 / 84 | 0.0000 |
+| XNNPACK int8 | 84 / 84 | 0.0000 |
+
+Eager itself gets 82 of 84 exactly right (CER 0.0022 against the rendered truth), so the
+model is not perfect on this set and every build reproduces it character for character
+anyway — its two mistakes included.
+
+### How close it came
+
+84 lines agreeing is 84 samples. What generalises is how much room each decision had. A
+character changes only if a build moves the winning logit past its best rival, so this is
+measured at **every one of the 1,008 decoding steps**: the winner's lead in the build's
+own logits, as a fraction of the lead the fp32 reference left it. 1.0 is untouched, 0.0
+is a tie, below 0.0 is a changed character.
+
+| build | median | worst step | steps changed |
+|---|---|---|---|
+| XNNPACK fp32 | 1.0000 | 1.0000 | 0 / 1008 |
+| XNNPACK fp16 | 1.0000 | 0.9884 | 0 / 1008 |
+| XNNPACK int8 | 1.0000 | **0.8661** | 0 / 1008 |
+
+**int8's worst moment still kept 87% of the margin.** That is the number behind the
+recommendation, not the 84 identical strings.
+
+A worst-case bound over the same steps — the reference gap divided by twice the largest
+error anywhere in the 6,144 logits — puts int8 at 0.87, below 1.0. That bound assumes the
+build's largest error landed exactly on the two logits that matter, which is not what
+happened; it says a flip cannot be ruled out from that statistic alone, not that one
+nearly occurred. Both are printed by `convert/check_manga_ocr.py`.
+
+### What a whole line costs
+
+The decoder re-reads the prefix each step, so its cost grows with the characters already
+emitted. Measured per call, median of 20:
+
+| prefix | fp32 | fp16 | int8 |
+|---|---|---|---|
+| 1 | 2.39 ms | 3.66 ms | 2.17 ms |
+| 8 | 3.49 ms | 5.07 ms | 3.11 ms |
+| 32 | 4.91 ms | 7.70 ms | 4.98 ms |
+| 64 | 6.43 ms | 10.95 ms | 5.98 ms |
+
+**Sixty-four times the prefix costs 2.8 times the step**, which is what makes the missing
+cache a fair trade. Summed over the prefixes a real line walks through, the decoder side
+of a **12-character line is 35 ms on int8** (40 ms fp32, 59 ms fp16); add one encoder run —
+4.1 ms on Core ML, 33.6 ms on XNNPACK int8 — for the whole read.
+
+### What this does not say
+
+- **These are rendered lines, not photographed manga.** The set varies what a renderer can
+  vary; it has no screentone, no handwriting, no speech-bubble clutter, no page curl. Read
+  it as how far a build has drifted from eager, not as an accuracy score for the model.
+- **The set never broke eager.** Even the heaviest degradations left eager reading
+  perfectly, so the set cannot say what the builds do in a regime where the model itself
+  starts failing.
+- **Nothing here ran on a phone.** Every number is host CPU, and the machine was busy.
+
+## Source
+
+Converted with [executorch-convert](https://github.com/john-rocky) — `convert/export_manga_ocr.py`
+for the two files, `convert/check_manga_ocr.py` for everything measured above.
+ExecuTorch 1.4.0, PyTorch 2.13.0.