Upload README.md with huggingface_hub

README.md

@@ -39,23 +39,23 @@ was measured against.
 
 | build | encoder | decoder | encoder speed | decoder speed |
 |---|---|---|---|---|
-| **Core ML + XNNPACK int8** | `manga_ocr_encoder_coreml_all.pte` **172.1 MB** | `manga_ocr_decoder_xnnpack_int8.pte` **45.0 MB** | **4.1 ms** | 2.6 ms |
-| XNNPACK int8 | `manga_ocr_encoder_xnnpack_int8.pte` 88.9 MB | `manga_ocr_decoder_xnnpack_int8.pte` 45.0 MB | 33.6 ms | 2.6 ms |
-| XNNPACK fp32 | `manga_ocr_encoder_xnnpack_fp32.pte` 343.4 MB | `manga_ocr_decoder_xnnpack_fp32.pte` 117.4 MB | 36.0 ms | 3.0 ms |
-| XNNPACK fp16 | `manga_ocr_encoder_xnnpack_fp16.pte` 173.1 MB | `manga_ocr_decoder_xnnpack_fp16.pte` 58.8 MB | 69.5 ms | 4.6 ms |
+| **Core ML + XNNPACK int8** | `manga_ocr_encoder_coreml_all.pte` **172.1 MB** | `manga_ocr_decoder_xnnpack_int8.pte` **45.0 MB** | **4.2 ms** | 2.5 ms |
+| XNNPACK int8 | `manga_ocr_encoder_xnnpack_int8.pte` 88.9 MB | `manga_ocr_decoder_xnnpack_int8.pte` 45.0 MB | 25.5 ms | 2.5 ms |
+| XNNPACK fp32 | `manga_ocr_encoder_xnnpack_fp32.pte` 343.3 MB | `manga_ocr_decoder_xnnpack_fp32.pte` 117.4 MB | 29.5 ms | 3.2 ms |
+| XNNPACK fp16 | `manga_ocr_encoder_xnnpack_fp16.pte` 173.1 MB | `manga_ocr_decoder_xnnpack_fp16.pte` 58.8 MB | 71.2 ms | 3.1 ms |
 
 Host numbers, on a loaded machine, decoder timed at a 3-character prefix — read them as a
 lower bound and as ratios rather than as device latency. Eager fp32 on the same host is
-29.2 ms for the encoder and 4.4 ms for the decoder.
+29.9 ms for the encoder and 4.6 ms for the decoder. XNNPACK takes 82.9% of the encoder
+and 51.9% of the decoder; Core ML takes all of the encoder.
 
-**fp16 is the row nothing should pick.** It is the same size as the Core ML encoder and
-half the speed of fp32 on the encoder, because XNNPACK inserts a cast at every boundary
-it cannot fuse. It is listed because it was built and measured, not because it wins
-anything.
+**fp16 is the row nothing should pick.** Its encoder is the size of the Core ML build and
+less than half the speed of fp32, because XNNPACK inserts a cast at every boundary it
+cannot fuse. It is listed because it was built and measured, not because it wins anything.
 
 **int8 is the row to pick on Android**: half of fp16 on the encoder and three-quarters of
-it on the decoder, faster than fp32 on both, and no character different from eager
-anywhere it was measured. What that costs is margin, not characters — see below.
+it on the decoder, **faster than eager PyTorch on both**, and no character different from
+eager anywhere it was measured. What that costs is margin, not characters — see below.
 
 ## Running it
 
@@ -118,15 +118,16 @@ the boundary per step instead of L times that.
 ## Greedy, not beam search
 
 The checkpoint's own generation config asks for 4 beams. Greedy is what the graph is
-shaped for, and it is what these files were measured with. On the shelf's gate images
-greedy and beam-4 read the same string. Beam search remains possible from outside — the
-decoder is a pure function of `(prefix, hidden)`, so a caller can run it once per beam per
-step — but nothing here has measured what that buys.
+shaped for and what every number here was measured with; on the horizontal gate image
+greedy and beam-4 read the same string, and the 84-line set never needed more than greedy
+to match eager. Beam search remains possible from outside — the decoder is a pure function
+of `(prefix, hidden)`, so a caller can run it once per beam per step — but what it buys on
+harder input than this has not been measured.
 
 ## Core ML: the encoder yes, the decoder no
 
-**The encoder is the whole speed story**: 100% delegated, 4.1 ms against 29.2 ms for eager
-on the same host, at 172.1 MB. Worst-output correlation against fp32 eager is 0.999985.
+**The encoder is the whole speed story**: 100% delegated, 4.2 ms against 29.9 ms for eager
+on the same host, at 172.1 MB. Worst-output correlation against fp32 eager is 0.999984.
 
 **The decoder is not shipped for Core ML, and the reason is specific.** It converts, it
 delegates 100%, and — once eight zero-element constants that transformers' BERT attention
@@ -170,17 +171,17 @@ is a tie, below 0.0 is a changed character.
 | build | median | worst step | steps changed |
 |---|---|---|---|
 | XNNPACK fp32 | 1.0000 | 1.0000 | 0 / 1008 |
-| XNNPACK fp16 | 1.0000 | 0.9884 | 0 / 1008 |
-| XNNPACK int8 | 1.0000 | **0.8661** | 0 / 1008 |
+| XNNPACK fp16 | 0.9999 | 0.9820 | 0 / 1008 |
+| XNNPACK int8 | 0.9999 | **0.6513** | 0 / 1008 |
 
-**int8's worst moment still kept 87% of the margin.** That is the number behind the
-recommendation, not the 84 identical strings.
+**int8's worst moment still kept 65% of the margin**, and it is the only step below 0.9.
+That is the number behind the recommendation, not the 84 identical strings.
 
-A worst-case bound over the same steps — the reference gap divided by twice the largest
-error anywhere in the 6,144 logits — puts int8 at 0.87, below 1.0. That bound assumes the
-build's largest error landed exactly on the two logits that matter, which is not what
-happened; it says a flip cannot be ruled out from that statistic alone, not that one
-nearly occurred. Both are printed by `convert/check_manga_ocr.py`.
+The same steps also carry a worst-case bound — the reference gap over twice the largest
+error anywhere in the 6,144 logits, which assumes that error landed exactly where it would
+do the most damage. Its minimum is **1.16 for int8**, above 1.0, so on this set a changed
+character is not merely absent but arithmetically impossible. Both numbers are printed by
+`convert/check_manga_ocr.py`.
 
 ### What a whole line costs
 
@@ -189,15 +190,15 @@ emitted. Measured per call, median of 20:
 
 | prefix | fp32 | fp16 | int8 |
 |---|---|---|---|
-| 1 | 2.39 ms | 3.66 ms | 2.17 ms |
-| 8 | 3.49 ms | 5.07 ms | 3.11 ms |
-| 32 | 4.91 ms | 7.70 ms | 4.98 ms |
-| 64 | 6.43 ms | 10.95 ms | 5.98 ms |
-
-**Sixty-four times the prefix costs 2.8 times the step**, which is what makes the missing
-cache a fair trade. Summed over the prefixes a real line walks through, the decoder side
-of a **12-character line is 35 ms on int8** (40 ms fp32, 59 ms fp16); add one encoder run —
-4.1 ms on Core ML, 33.6 ms on XNNPACK int8 — for the whole read.
+| 1 | 2.36 ms | 2.27 ms | 2.18 ms |
+| 8 | 3.37 ms | 3.45 ms | 2.90 ms |
+| 32 | 3.94 ms | 4.88 ms | 4.26 ms |
+| 64 | 5.31 ms | 6.94 ms | 4.99 ms |
+
+**Sixty-four times the prefix costs 2.3 times the step**, which is what makes the missing
+cache a fair trade. Summed over the prefixes a real line walks through, the decoder side of
+a **12-character line is 33 ms on int8** (39 ms fp32); add one encoder run — 4.2 ms on
+Core ML, 25.5 ms on XNNPACK int8 — for the whole read.
 
 ### What this does not say