Gabriel commited on
Commit
b0dacaa
·
verified ·
1 Parent(s): 7a31b73

One stream each: TrOCR reads its lines one by one (its one-batch time beside it), the replay starts all three together, our server's vLLM timings under the race, one region box, Jev's rows folded with marked errors

Browse files
HOW_IT_WORKS.md CHANGED
@@ -1,24 +1,39 @@
1
  ### What is timed
2
 
3
- All three read the same region, one after the other on the same GPU (ZeroGPU: half an RTX PRO 6000 Blackwell). Each
4
- is timed from the images to the text, preprocessing included, after a warm-up that is not timed:
 
5
 
6
  - **Swedish Lion TrOCR** (Riksarkivet's Swedish Lion Libre 2026, large, character-based) cannot read a region. It
7
- gets the region as its text lines, cut out beforehand along the ground truth's line outlines, and reads them all in
8
- one batch, greedily, in float32 as Riksarkivet runs it. Finding the lines is a model of its own in practice; that
9
- step is not timed here.
10
  - **MiniCPM-OCR reader alone** reads the region as one image, one token per pass, greedily.
11
  - **Reader + DSpark draft**: the draft proposes 15 tokens at a time; the reader checks them in one pass and keeps the
12
  ones it agrees with, plus one of its own. Greedy decoding, so the text is the reader's own.
13
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  ### What you see
15
 
16
- Each reader notes when every token became known, and the page replays its reading at that pace: first as each reading
17
- arrives, then side by side from the replay buttons. Nothing is sent to the page inside a timed stretch, so the display
18
- slows no reader down.
19
 
20
- - On each manuscript a highlighter follows the reader: TrOCR's moves along every line at once (a decoding step adds
21
- one character to each line of the batch), the reader's goes line by line.
22
  - The strip above a reader's text has one block per reader pass, as wide as the tokens that pass produced: many thin
23
  ones alone, a few wide ones with the draft.
24
  - In the DSpark lane, shaded text was proposed by the draft and kept; underlined text is the reader's own token, one
@@ -45,23 +60,41 @@ the time. So:
45
  cannot be cut back after rejected guesses. The checking pass keeps each such layer's inputs for the block, and the
46
  state is rebuilt over the accepted tokens only (a few kernels instead of a second pass).
47
 
48
- Measured on our server (a full RTX PRO 6000 with nothing else on it, 12 dev regions and 20 test lines): the regions
49
- took 12.4 s with the reader alone and 3.2 s with the draft (**3.9×**); the lines 2.8 s and 2.2 s (1.3×). The reader
50
- alone writes the same tokens as transformers' own `generate`.
 
 
 
 
51
 
52
- vLLM is still faster than this Space's PyTorch: on the same GPU and regions it took 5.9 s with the reader alone and
53
- 2.2 s with the draft, about 2× and 1.4× faster.
 
 
 
 
 
 
54
 
55
- ### The same models in vLLM
 
 
56
 
57
- On 1,000 test lines and 200 dev regions, one GPU per server (the draft's model card):
 
58
 
59
- | | one at a time | 64 at a time |
60
  |---|---|---|
61
- | lines: reader alone → with DSpark | 13.3 → 18.2 lines/s (1.37×) | 74.6 → 82.2 lines/s (1.10×) |
62
- | regions: reader alone → with DSpark | 1.95 → 4.08 regions/s (2.09×) | 12.4 → 13.2 regions/s (1.07×) |
 
63
 
64
- Acceptance length 8.72; the character error rate is the same with and without the draft.
 
 
 
 
65
 
66
  ### Jev
67
 
@@ -81,3 +114,5 @@ does not prove a reading right.
81
  where the loop starts.
82
  - Timings vary with the GPU slice and the queue. A fresh GPU worker captures its CUDA graphs first (a few seconds, not
83
  counted in the timings shown).
 
 
 
1
  ### What is timed
2
 
3
+ All three read the same region, one after the other on the same GPU (ZeroGPU: half an RTX PRO 6000 Blackwell), with
4
+ **one stream each and nothing batched**. Each is timed from the images to the text, preprocessing included, after a
5
+ warm-up that is not timed:
6
 
7
  - **Swedish Lion TrOCR** (Riksarkivet's Swedish Lion Libre 2026, large, character-based) cannot read a region. It
8
+ gets the region as its text lines, cut out beforehand along the ground truth's line outlines, and reads them one
9
+ after the other, greedily, in float32 as Riksarkivet runs it. Finding the lines is a model of its own in practice;
10
+ that step is not timed here.
11
  - **MiniCPM-OCR reader alone** reads the region as one image, one token per pass, greedily.
12
  - **Reader + DSpark draft**: the draft proposes 15 tokens at a time; the reader checks them in one pass and keeps the
13
  ones it agrees with, plus one of its own. Greedy decoding, so the text is the reader's own.
14
 
15
+ ### Comparing them fairly
16
+
17
+ A batch lets a GPU decode many sequences side by side. TrOCR can batch a region's lines. The reader reads a region as
18
+ one sequence, which has nothing to batch; what it batches is whole regions, in vLLM, which this Space cannot run.
19
+ Racing a batch of 8 to 18 lines against one sequence would compare two different things, so there are three views,
20
+ each with one rule for everyone:
21
+
22
+ - **Nothing batched** (the race): TrOCR reads its lines one after the other, the reader one region.
23
+ - **TrOCR's lines as one batch** is timed too and drawn as the second bar, so nothing is hidden: on these regions it
24
+ is about as fast as the reader alone in this Space's PyTorch, and 2 to 4 times slower than the reader with its
25
+ draft.
26
+ - **Everything batched** (the three rates under the race): on our server, TrOCR in batches of 32 lines against the
27
+ reader with 64 requests at a time in vLLM, which reads about ten times as many lines a second.
28
+
29
  ### What you see
30
 
31
+ Each reader notes when every token became known. Once all three are timed, the page replays their readings from the
32
+ same start, each at its measured pace; the buttons replay it, also at half and quarter speed. Nothing is sent to the
33
+ page inside a timed stretch, so the display slows no reader down.
34
 
35
+ - On each manuscript a highlighter follows the reader: TrOCR's goes down the lines one by one, a character per
36
+ decoding step; the reader's follows its tokens through the region.
37
  - The strip above a reader's text has one block per reader pass, as wide as the tokens that pass produced: many thin
38
  ones alone, a few wide ones with the draft.
39
  - In the DSpark lane, shaded text was proposed by the draft and kept; underlined text is the reader's own token, one
 
60
  cannot be cut back after rejected guesses. The checking pass keeps each such layer's inputs for the block, and the
61
  state is rebuilt over the accepted tokens only (a few kernels instead of a second pass).
62
 
63
+ ### The same region on our server, and vLLM
64
+
65
+ The outlined bars under the race were measured beforehand on our server (`measure_reference.py`, `reference.json`):
66
+ one whole RTX PRO 6000 Blackwell with nothing else running on it, TrOCR in transformers as here, the reader served by
67
+ vLLM alone and with its draft, one request at a time, the median of 5 runs. vLLM keeps the processed form of an image
68
+ it has seen, which saves 0.05 to 0.1 s a region, so every request sent the region with one pixel changed by one
69
+ level: new to the server, as a region is in production. Over the six example regions:
70
 
71
+ | a region read by | seconds |
72
+ |---|---|
73
+ | TrOCR, one line after the other | 1.25 – 3.92 |
74
+ | TrOCR, its lines as one batch | 0.60 – 1.54 |
75
+ | the reader in vLLM | 0.38 – 0.73 |
76
+ | the reader with its draft in vLLM | 0.27 – 0.45 |
77
+ | the reader in this Space's PyTorch, on that GPU | 0.53 – 1.21 |
78
+ | the reader with its draft in this Space's PyTorch, on that GPU | 0.23 – 0.41 |
79
 
80
+ So with the draft this Space's decoding is as fast as vLLM's for one region at a time, and alone it is 1.2 to 1.8
81
+ times slower; the live bars are slower again because ZeroGPU gives half a GPU. (The PyTorch timings start from the
82
+ decoded image, vLLM's from the request.) The reader alone writes the same tokens as transformers' own `generate`.
83
 
84
+ What vLLM adds is reading many requests at once. On the first 1,000 test lines and 200 dev regions, one RTX PRO
85
+ 6000 Blackwell:
86
 
87
+ | | one at a time | many at a time |
88
  |---|---|---|
89
+ | lines: Swedish Lion TrOCR | 4.3 lines/s (batch 1) | 6.5 lines/s (batch 32) |
90
+ | lines: reader alone → with DSpark | 13.3 → 18.2 lines/s (1.37×) | 67 → 77 lines/s (64 at a time) |
91
+ | regions: reader alone → with DSpark | 1.95 → 4.08 regions/s (2.09×) | 11.9 → 12.7 regions/s (64 at a time) |
92
 
93
+ The 64-at-a-time rows are from a run where every image was new to the servers (two servers on one GPU, asked in
94
+ turn); with each server on a GPU of its own they were 75 → 82 lines/s and 12.4 → 13.2 regions/s. Acceptance length
95
+ 8.72; the character error rate is the same with and without the draft. OpenBMB's deployment guide also serves the
96
+ reader and a DSpark draft with SGLang (through a pull request still open there in October 2026); we have measured
97
+ vLLM only.
98
 
99
  ### Jev
100
 
 
114
  where the loop starts.
115
  - Timings vary with the GPU slice and the queue. A fresh GPU worker captures its CUDA graphs first (a few seconds, not
116
  counted in the timings shown).
117
+ - With the draft the reader's text can differ from its text alone by a token where two tokens were nearly tied: a
118
+ check of 16 tokens rounds differently from a pass of one. vLLM shows the same between 1 and 64 requests at a time.
README.md CHANGED
@@ -19,16 +19,20 @@ models:
19
 
20
  # Swedish HTR speed and Jev
21
 
22
- Reads the same handwritten text region three ways on the same GPU, times each and replays the three side by side: the
23
- Swedish Lion TrOCR on the region's text lines, the
24
  [MiniCPM-OCR HTR reader](https://huggingface.co/Gabriel/diffu-minicpm-ocr-htr) on the region as one image, and the same
25
- reader with its [DSpark draft](https://huggingface.co/Gabriel/diffu-minicpm-ocr-htr-dspark). Then
26
- [Jev](https://huggingface.co/Gabriel/diffu-minicpm-ocr-htr-jev), the reader's proof-reader, scores both readers' reading
27
- of each line.
 
28
 
29
  - `app.py`: the Gradio app; the readers run one after the other and the page replays them at their measured pace.
30
  - `dspark_hf.py`: DSpark speculative decoding in PyTorch on CUDA graphs (ZeroGPU cannot host vLLM).
31
- - `race_view.py`, `race_view.js`, `race_view.css`: the custom component that draws the race and Jev's rows.
 
 
 
32
  - `check_decode.py`: compares the reader alone and with its draft on a GPU, token for token, and times them.
33
 
34
  See the app's "How it works" tab. Part of [diffu](https://github.com/Borg93/diffu2).
 
19
 
20
  # Swedish HTR speed and Jev
21
 
22
+ Reads the same handwritten text region with one stream each on the same GPU, times each and replays them from the same
23
+ start: the Swedish Lion TrOCR on the region's text lines, one after the other (and once more as one batch), the
24
  [MiniCPM-OCR HTR reader](https://huggingface.co/Gabriel/diffu-minicpm-ocr-htr) on the region as one image, and the same
25
+ reader with its [DSpark draft](https://huggingface.co/Gabriel/diffu-minicpm-ocr-htr-dspark). Under the race: the same
26
+ region's timings on our server, where vLLM serves the reader, and what the readers do there when everything is batched.
27
+ Then [Jev](https://huggingface.co/Gabriel/diffu-minicpm-ocr-htr-jev), the reader's proof-reader, scores both readers'
28
+ reading of each line.
29
 
30
  - `app.py`: the Gradio app; the readers run one after the other and the page replays them at their measured pace.
31
  - `dspark_hf.py`: DSpark speculative decoding in PyTorch on CUDA graphs (ZeroGPU cannot host vLLM).
32
+ - `race_view.py`, `race_view.js`, `race_view.css`: the custom component that draws the race, our server's timings and
33
+ Jev's rows.
34
+ - `reference.json`, `reference.py`, `measure_reference.py`: each example region's timings on our server (TrOCR in
35
+ transformers, the reader in vLLM), and the script that measured them.
36
  - `check_decode.py`: compares the reader alone and with its draft on a GPU, token for token, and times them.
37
 
38
  See the app's "How it works" tab. Part of [diffu](https://github.com/Borg93/diffu2).
app.py CHANGED
@@ -3,14 +3,18 @@ draft, timed on the same region and replayed side by side; and Jev, the reader's
3
 
4
  Everything runs in PyTorch on ZeroGPU (vLLM, where the draft normally runs, cannot run here): the reader's passes and
5
  the draft's proposals are replayed from CUDA graphs (dspark_hf.py), for the reader alone and with the draft alike.
6
- The readers run one after the other; each notes when every token became known, and the page replays them at that pace
7
- (race_view.py). Nothing is sent to the page inside a timed stretch.
 
 
 
8
  """
9
 
10
  # ZeroGPU patches CUDA when `spaces` is imported, so it comes before torch and anything that imports torch.
11
  import spaces # isort: skip
12
 
13
  import base64
 
14
  import io
15
  import json
16
  import os
@@ -18,14 +22,14 @@ import threading
18
  import time
19
  from collections.abc import Callable, Iterator
20
  from pathlib import Path
21
- from typing import Any, cast
22
  from uuid import uuid4
23
 
24
  import gradio as gr
25
  import torch
26
  from huggingface_hub import snapshot_download
27
  from huggingface_hub.errors import HfHubHTTPError
28
- from PIL import Image
29
  from pydantic import BaseModel, Field
30
  from rapidfuzz.distance import Levenshtein
31
  from transformers import (
@@ -42,6 +46,7 @@ import dspark_hf as D
42
  from jev_prompt import model_prompt
43
  from jev_render import render_text, renderer_version
44
  from race_view import RaceView
 
45
 
46
  READER = "Gabriel/diffu-minicpm-ocr-htr"
47
  DRAFT = "Gabriel/diffu-minicpm-ocr-htr-dspark"
@@ -51,6 +56,7 @@ TROCR = os.environ.get("TROCR_REPO", "Gabriel/swedish-lion-trocr-large-handwritt
51
  HERE = Path(__file__).parent
52
  FONT = HERE / "DejaVuSans.ttf"
53
  EXAMPLES: list[dict[str, Any]] = json.loads((HERE / "examples" / "examples.json").read_text(encoding="utf-8"))
 
54
  MAX_LEN = 4096 # the reader's cache: a region's prompt (up to about 2,600 tokens) and its reading; shorter is faster
55
  MINICPM_IMAGES = {"downsample_mode": "4x", "max_slice_nums": 9} # OpenBMB's settings, as the reader was trained
56
  # new tokens at most: an example region, an upload (it must fit its GPU slot), a line
@@ -83,17 +89,20 @@ gr.set_static_paths([HERE / "examples"]) # the line crops in Jev's rows
83
 
84
 
85
  class TrocrLine(BaseModel):
86
- """A line's reading and how many of its characters each decoding step had reached."""
 
87
 
88
  text: str
 
89
  ends: list[int]
90
 
91
 
92
  class TrocrLane(BaseModel):
93
- """TrOCR's reading of a region's lines in one batch: its seconds and when each decoding step ended."""
 
94
 
95
  seconds: float
96
- steps: list[float]
97
  lines: list[TrocrLine]
98
  cer: float | None = None
99
 
@@ -118,24 +127,53 @@ class ReaderLane(BaseModel):
118
  return "".join(piece for _, piece, _ in self.pieces).strip()
119
 
120
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
121
  class JevRow(BaseModel):
122
- """One line: its crop, its transcription, and Jev's score of TrOCR's reading and of the reader's, each of the
123
- line on its own."""
124
 
125
  index: int
126
  image: str
127
  truth: str
128
- trocr: str
129
- trocr_score: float
130
- trocr_exact: bool
131
- minicpm: str
132
- minicpm_score: float
133
- minicpm_exact: bool
134
 
135
 
136
  class Race(BaseModel):
137
  """What RaceView draws: the region with its lines' outlines (fractions of the image) and each line's cumulative
138
- share of the transcription, the readers' lanes as they finish, then Jev's rows."""
 
139
 
140
  run: str = Field(default_factory=lambda: uuid4().hex[:12])
141
  image: str | None = None
@@ -144,26 +182,52 @@ class Race(BaseModel):
144
  shares: list[float] = Field(default_factory=list)
145
  status: str = ""
146
  notice: str = ""
 
147
  done: bool = False
148
  trocr: TrocrLane | None = None
149
  plain: ReaderLane | None = None
150
  dspark: ReaderLane | None = None
151
  summary: str = ""
 
 
 
 
152
  jev: list[JevRow] = Field(default_factory=list)
153
  jev_total: int = 0
154
  jev_note: str = ""
155
 
156
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
157
  class StepClock(StoppingCriteria):
158
  """Notes when each decoding step ends; it never stops the generation itself."""
159
 
160
  def __init__(self) -> None:
161
  self.times: list[float] = []
 
162
 
163
  def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs: Any) -> torch.BoolTensor:
164
  self.times.append(time.perf_counter())
165
- never = torch.zeros(input_ids.shape[0], dtype=torch.bool, device=input_ids.device)
166
- return cast(torch.BoolTensor, never)
 
167
 
168
 
169
  def _example(example_id: str) -> dict[str, Any]:
@@ -195,40 +259,51 @@ def _region_inputs(image: Image.Image) -> dict[str, Any]:
195
  return inputs
196
 
197
 
198
- def _trocr_read(lines: list[Image.Image], max_new_tokens: int = 128) -> TrocrLane:
199
- """TrOCR's greedy readings of a batch of lines, timed from the images to the text, as Riksarkivet's evaluation
200
- runs it: the Swedish Lion starts decoding from its config's token (its generation_config names another, which
201
- costs about 2 points of CER), and its 192 x 1024 inputs need the position encoding interpolated."""
202
  model, images = trocr, trocr_processor
203
  if model is None or images is None:
204
  raise RuntimeError("TrOCR is not loaded")
205
- torch.cuda.synchronize()
206
- started = time.perf_counter()
207
- clock = StepClock()
208
  pixels = images(images=lines, return_tensors="pt").pixel_values.to("cuda")
209
- ids = model.generate(
210
  pixels,
211
  decoder_start_token_id=model.config.decoder_start_token_id,
212
  interpolate_pos_encoding=True,
213
  num_beams=1,
214
  max_new_tokens=max_new_tokens,
215
- stopping_criteria=StoppingCriteriaList([clock]),
216
  )
 
 
 
 
 
 
 
 
217
  torch.cuda.synchronize()
 
 
 
 
 
 
218
  seconds = time.perf_counter() - started
219
- steps = [round(at - started, 4) for at in clock.times]
220
  tokenizer = images.tokenizer
221
  read = []
222
- for row in ids[:, 1:].tolist(): # after the start token
223
- text = tokenizer.decode(row, skip_special_tokens=True).strip()
224
  ends, shown = [], 0
225
  for step in range(1, len(steps) + 1):
226
  if shown < len(text):
227
- partial = tokenizer.decode(row[:step], skip_special_tokens=True).strip()
228
  shown = max(shown, min(len(text), len(partial)))
229
  ends.append(shown)
230
- read.append(TrocrLine(text=text, ends=ends))
231
- return TrocrLane(seconds=seconds, steps=steps, lines=read)
232
 
233
 
234
  def _reader_lane(decoded: D.Decoded, seconds: float) -> ReaderLane:
@@ -284,7 +359,7 @@ def _engine() -> D.FastDSpark:
284
  warm = Image.open(HERE / "examples" / EXAMPLES[0]["id"] / "region.png").convert("RGB")
285
  engine.warm(_region_inputs(warm), EOS)
286
  if trocr is not None:
287
- _trocr_read([warm.crop((0, 0, 200, 40))], max_new_tokens=4)
288
  _jev_score(_jev_crop(warm.crop((0, 0, 200, 40))), "warm")
289
  _engines[os.getpid()] = engine
290
  return engine
@@ -299,51 +374,115 @@ def _timed[T](fn: Callable[[], T]) -> tuple[T, float]:
299
 
300
 
301
  def _summary(race: Race, identical: bool) -> str:
302
- plain, spec = race.plain, race.dspark
303
  if plain is None or spec is None:
304
  return ""
305
- against_trocr = f" and {race.trocr.seconds / spec.seconds:.1f}× faster than TrOCR" if race.trocr else ""
306
- same = (
307
- "It wrote exactly what the reader writes alone."
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
308
  if identical
309
  else "Its reading differs from the reader's alone at a near-tied token: the reader checks 16 tokens at once, "
310
- "and a batch of another size can tip a tie the other way (the reader alone does the same between batch sizes)."
311
- )
312
- return (
313
- f"With the draft, the reader needed {spec.passes} passes instead of {plain.passes} for {spec.tokens} tokens, "
314
- f"keeping {spec.acceptance or 0:.1f} tokens per check on average (at most 16): "
315
- f"{plain.seconds / spec.seconds:.1f}× faster than alone{against_trocr}. {same}"
316
  )
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
317
 
318
 
319
  def _jev_note(race: Race, truth: list[str]) -> str:
 
320
  whole = " ".join(truth)
321
- trocr_cer = _cer(" ".join(row.trocr for row in race.jev), whole)
322
- minicpm_cer = _cer(" ".join(row.minicpm for row in race.jev), whole)
323
- picked = [row.minicpm if row.minicpm_score > row.trocr_score else row.trocr for row in race.jev]
324
- picked_cer = _cer(" ".join(picked), whole)
325
- return (
326
- f"CER over these lines, each read on its own: TrOCR {100 * trocr_cer:.1f} %, MiniCPM "
327
- f"{100 * minicpm_cer:.1f} %, the reading Jev scores higher on each line {100 * picked_cer:.1f} %."
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
328
  )
 
329
 
330
 
331
- def _duration(example_id: str, upload: Image.Image | None, with_jev: bool) -> int:
332
- return 75 if upload is not None else 50 + (15 if with_jev else 0)
333
 
334
 
335
  @spaces.GPU(duration=_duration)
336
- def compare(example_id: str, upload: Image.Image | None, with_jev: bool) -> Iterator[Race]:
337
- """Read one region three ways, one after the other, timing each from the images to the text: TrOCR on its lines
338
- in one batch, the MiniCPM-OCR reader alone, and the reader with its DSpark draft (the last two on the same CUDA
339
- graphs). For an example region, the reader then reads each line on its own, and Jev scores that reading and
340
- TrOCR's. Yields the race so far between the timed stretches."""
341
  race = Race(status="Preparing the GPU worker: a fresh one captures its CUDA graphs first…")
342
  yield race
343
  with _lock:
344
  engine = _engine()
345
- if upload is not None:
346
- region, lines, truth, limit = upload.convert("RGB"), [], None, UPLOAD_TOKENS
347
  else:
348
  folder = HERE / "examples" / example_id
349
  truth = _example(example_id)["lines"]
@@ -355,7 +494,7 @@ def compare(example_id: str, upload: Image.Image | None, with_jev: bool) -> Iter
355
  if trocr is None:
356
  race.notice = NO_TROCR
357
  elif lines:
358
- race.status = f"TrOCR is reading the region's {len(lines)} lines…"
359
  yield race
360
  race.trocr = _trocr_read(lines)
361
  race.trocr.cer = _cer("\n".join(race.trocr.texts), whole) if whole else None
@@ -372,63 +511,84 @@ def compare(example_id: str, upload: Image.Image | None, with_jev: bool) -> Iter
372
  race.dspark = _reader_lane(spec, seconds)
373
  race.dspark.cer = _cer(race.dspark.text, whole) if whole else None
374
  race.summary = _summary(race, identical=plain.tokens == spec.tokens)
 
375
 
376
- if with_jev and race.trocr is not None and truth is not None:
377
  race.jev_total = len(lines)
378
  race.status = "The reader reads each line on its own, and Jev scores its reading and TrOCR's…"
379
  yield race
380
  for i, line in enumerate(lines):
381
- crop = _jev_crop(line)
382
- guess = race.trocr.lines[i].text
383
  own = _text(engine.decode(_region_inputs(line), LINE_TOKENS, EOS).tokens)
384
- row = JevRow(
385
- index=i,
386
- image=f"/gradio_api/file={HERE / 'examples' / example_id / f'line_{i:02d}.jpg'}",
387
- truth=truth[i],
388
- trocr=guess,
389
- trocr_score=_jev_score(crop, guess),
390
- trocr_exact=_same(guess, truth[i]),
391
- minicpm=own,
392
- minicpm_score=_jev_score(crop, own),
393
- minicpm_exact=_same(own, truth[i]),
394
- )
395
- race.jev.append(row)
396
  yield race
397
  race.jev_note = _jev_note(race, truth)
398
  race.status, race.done = "", True
399
  yield race
400
 
401
 
402
- def _stage(example_id: str, upload: Image.Image | None) -> Race:
403
- """The race before anyone has read: the region's image and, for an example, its lines' outlines."""
404
- if upload is not None:
405
- shown = upload.convert("RGB")
 
406
  shown.thumbnail((1000, 1000))
407
  buffer = io.BytesIO()
408
  shown.save(buffer, format="JPEG", quality=85)
409
  image = "data:image/jpeg;base64," + base64.b64encode(buffer.getvalue()).decode()
410
- return Race(image=image, aspect=shown.width / shown.height)
411
  example = _example(example_id)
412
  width, height = example["region_size"]
413
  lengths = [max(1, len(line)) for line in example["lines"]]
414
  shares = [sum(lengths[: i + 1]) / sum(lengths) for i in range(len(lengths))]
 
415
  return Race(
416
  image=f"/gradio_api/file={HERE / 'examples' / example_id / 'region.png'}",
417
  aspect=width / height,
418
  outlines=example["outlines"],
419
  shares=shares,
 
 
 
 
 
 
420
  )
421
 
422
 
423
- def run(example_label: str, upload: Image.Image | None, with_jev: bool) -> Iterator[dict[str, Any]]:
424
- """Read the chosen example region (or an uploaded one) with TrOCR, the MiniCPM-OCR reader and the reader with its
425
- DSpark draft, then let Jev score each line. Streams the race as each reader finishes: every reading with the time
426
- each token became known, the summary, and Jev's rows."""
427
- example_id = LABELS[example_label]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
428
  stage = _stage(example_id, upload)
429
  stage.status = "Waiting for a GPU…"
430
  yield stage.model_dump()
431
- keep = {name: getattr(stage, name) for name in ("run", "image", "aspect", "outlines", "shares")}
 
 
 
432
  for race in compare(example_id, upload, with_jev):
433
  yield race.model_copy(update=keep).model_dump()
434
 
@@ -447,46 +607,58 @@ def score_line(line: Image.Image | None, text: str) -> tuple[dict[str, float], I
447
 
448
  def _label(example: dict[str, Any]) -> str:
449
  collection = example["collection"].removeprefix("GT-").rsplit("_", 1)[0].replace("_", " ")
450
- return f"{collection} · {len(example['lines'])} lines"
 
451
 
 
 
 
 
 
 
452
 
453
- LABELS = {_label(e): e["id"] for e in EXAMPLES}
454
 
455
  with gr.Blocks(title="Swedish HTR: TrOCR, MiniCPM-OCR and DSpark") as demo:
456
  gr.Markdown(
457
  "# Reading Swedish handwriting faster\n"
458
- "The same text region from an archive volume, read three ways on the same GPU. Riksarkivet's **Swedish Lion "
459
- "TrOCR** cannot read a region, so it gets the region as its text lines, cut out beforehand, and reads them as "
460
- f"one batch. Our **[MiniCPM-OCR reader](https://huggingface.co/{READER})** reads the region as one image, "
461
- f"alone and with its **[DSpark draft](https://huggingface.co/{DRAFT})**, which guesses 15 tokens at a time "
462
- f"for the reader to check in one pass. Then **[Jev](https://huggingface.co/{JEV})**, the reader's "
463
- "proof-reader, scores both readers' reading of each line. The regions are from volumes none of the models "
464
- "trained on."
 
465
  )
466
  with gr.Tab("Compare"):
467
  with gr.Row():
468
- with gr.Column(scale=1, min_width=300):
469
- choice = gr.Dropdown(list(LABELS), value=next(iter(LABELS)), label="Example region")
470
- with_jev = gr.Checkbox(value=True, label="Score each line with Jev")
471
- go = gr.Button("Read", variant="primary")
472
- with gr.Accordion("Or read your own region", open=False):
473
- upload = gr.Image(type="pil", label="A region's crop")
474
- gr.Markdown("Only the reader reads an upload: TrOCR and Jev need the region's lines.")
475
- with gr.Column(scale=2):
476
- preview = gr.Image(
477
- value=str(HERE / "examples" / EXAMPLES[0]["id"] / "region.png"),
478
- label="Region",
479
- interactive=False,
480
- height=330,
481
  )
482
- race_view = RaceView()
483
- choice.change(
484
- lambda label: str(HERE / "examples" / LABELS[label] / "region.png"),
485
- choice,
486
- preview,
487
- api_visibility="private",
 
 
 
 
 
 
 
488
  )
489
- go.click(run, [choice, upload, with_jev], race_view, api_name="compare", show_progress="minimal")
 
 
490
  with gr.Tab("Try Jev"):
491
  gr.Markdown(
492
  "Jev scores whether a text is exactly what a **single handwritten line** says. Upload a line crop and type "
 
3
 
4
  Everything runs in PyTorch on ZeroGPU (vLLM, where the draft normally runs, cannot run here): the reader's passes and
5
  the draft's proposals are replayed from CUDA graphs (dspark_hf.py), for the reader alone and with the draft alike.
6
+ The readers run one after the other, one stream each: TrOCR reads the region's lines one by one (and once more as
7
+ one batch, shown beside), the reader reads the region as one sequence. Each notes when every token became known, and
8
+ once all are timed the page replays them from the same start at that pace (race_view.py). Nothing is sent to the page
9
+ inside a timed stretch. Under the live timings the page shows the same region's on our server, where vLLM serves the
10
+ reader (reference.py), and what the readers do there when everything is batched.
11
  """
12
 
13
  # ZeroGPU patches CUDA when `spaces` is imported, so it comes before torch and anything that imports torch.
14
  import spaces # isort: skip
15
 
16
  import base64
17
+ import filecmp
18
  import io
19
  import json
20
  import os
 
22
  import time
23
  from collections.abc import Callable, Iterator
24
  from pathlib import Path
25
+ from typing import Any, Literal, cast
26
  from uuid import uuid4
27
 
28
  import gradio as gr
29
  import torch
30
  from huggingface_hub import snapshot_download
31
  from huggingface_hub.errors import HfHubHTTPError
32
+ from PIL import Image, ImageOps
33
  from pydantic import BaseModel, Field
34
  from rapidfuzz.distance import Levenshtein
35
  from transformers import (
 
46
  from jev_prompt import model_prompt
47
  from jev_render import render_text, renderer_version
48
  from race_view import RaceView
49
+ from reference import Reference, RegionTimes
50
 
51
  READER = "Gabriel/diffu-minicpm-ocr-htr"
52
  DRAFT = "Gabriel/diffu-minicpm-ocr-htr-dspark"
 
56
  HERE = Path(__file__).parent
57
  FONT = HERE / "DejaVuSans.ttf"
58
  EXAMPLES: list[dict[str, Any]] = json.loads((HERE / "examples" / "examples.json").read_text(encoding="utf-8"))
59
+ REFERENCE = Reference.model_validate_json((HERE / "reference.json").read_text(encoding="utf-8"))
60
  MAX_LEN = 4096 # the reader's cache: a region's prompt (up to about 2,600 tokens) and its reading; shorter is faster
61
  MINICPM_IMAGES = {"downsample_mode": "4x", "max_slice_nums": 9} # OpenBMB's settings, as the reader was trained
62
  # new tokens at most: an example region, an upload (it must fit its GPU slot), a line
 
89
 
90
 
91
  class TrocrLine(BaseModel):
92
+ """A line's reading, when each of its decoding steps ended (seconds from the start of the region's first line) and
93
+ how many of its characters each step had reached."""
94
 
95
  text: str
96
+ steps: list[float]
97
  ends: list[int]
98
 
99
 
100
  class TrocrLane(BaseModel):
101
+ """TrOCR's reading of a region's lines, one line after the other: its seconds and each line's steps; and the
102
+ seconds the same lines took as one batch."""
103
 
104
  seconds: float
105
+ batch_seconds: float
106
  lines: list[TrocrLine]
107
  cer: float | None = None
108
 
 
127
  return "".join(piece for _, piece, _ in self.pieces).strip()
128
 
129
 
130
+ class ServerTimes(RegionTimes):
131
+ """A region's timings on our server, with its GPU, the day they were measured and the runs they are medians of."""
132
+
133
+ gpu: str
134
+ measured: str
135
+ runs: int
136
+
137
+
138
+ class Rate(BaseModel):
139
+ """What a reader reads a second when everything is batched, and how it was batched."""
140
+
141
+ lane: Literal["trocr", "plain", "dspark"]
142
+ name: str
143
+ how: str
144
+ per_second: float
145
+
146
+
147
+ class JevReading(BaseModel):
148
+ """One reader's reading of a line: its text in stretches, each with whether it agrees with the ground truth, Jev's
149
+ score, and whether the whole reading is exactly the ground truth."""
150
+
151
+ marks: list[tuple[str, bool]]
152
+ score: float
153
+ exact: bool
154
+
155
+ @property
156
+ def text(self) -> str:
157
+ return "".join(piece for piece, _ in self.marks)
158
+
159
+
160
  class JevRow(BaseModel):
161
+ """One line: its crop, its ground truth, TrOCR's reading and the reader's (each of the line on its own) with Jev's
162
+ scores, the reader Jev scores higher ("" where both read the same) and what that amounts to."""
163
 
164
  index: int
165
  image: str
166
  truth: str
167
+ trocr: JevReading
168
+ minicpm: JevReading
169
+ pick: Literal["trocr", "minicpm", ""]
170
+ verdict: str
 
 
171
 
172
 
173
  class Race(BaseModel):
174
  """What RaceView draws: the region with its lines' outlines (fractions of the image) and each line's cumulative
175
+ share of the transcription, the readers' lanes as they are timed (``ready`` once all are: the replay starts), the
176
+ same region's timings on our server and the batched rates, then Jev's rows."""
177
 
178
  run: str = Field(default_factory=lambda: uuid4().hex[:12])
179
  image: str | None = None
 
182
  shares: list[float] = Field(default_factory=list)
183
  status: str = ""
184
  notice: str = ""
185
+ ready: bool = False
186
  done: bool = False
187
  trocr: TrocrLane | None = None
188
  plain: ReaderLane | None = None
189
  dspark: ReaderLane | None = None
190
  summary: str = ""
191
+ server: ServerTimes | None = None
192
+ server_said: str = ""
193
+ batched: list[Rate] = Field(default_factory=list)
194
+ batched_note: str = ""
195
  jev: list[JevRow] = Field(default_factory=list)
196
  jev_total: int = 0
197
  jev_note: str = ""
198
 
199
 
200
+ # Everything batched, on our server: the first 1,000 test lines, a whole GPU for TrOCR, the two vLLM servers sharing
201
+ # one and asked in turn, every image new to them (docs/HTR_RUNS.md: bench_three.sh on 9 Nov, bench_cold.sh on 11 Nov)
202
+ BATCHED = [
203
+ Rate(lane="trocr", name="Swedish Lion TrOCR", how="batches of 32 lines", per_second=6.54),
204
+ Rate(lane="plain", name="Reader alone, in vLLM", how="64 requests at a time", per_second=67.26),
205
+ Rate(lane="dspark", name="Reader + DSpark, in vLLM", how="64 requests at a time", per_second=77.15),
206
+ ]
207
+ # This Space's decoding against vLLM on that server's GPU, the six example regions (docs/HTR_RUNS.md, 11 Nov 2026)
208
+ SAME_GPU = (
209
+ "On that same whole GPU this Space's PyTorch decoding is as fast as vLLM with the draft (0.23–0.41 s a region "
210
+ "against 0.27–0.45 s) and 1.2–1.8× slower alone; the rest of the gap to the live bars is the half GPU."
211
+ )
212
+ BATCHED_NOTE = (
213
+ "The first 1,000 test lines on one RTX PRO 6000 Blackwell, every image new to the server. One at a time on the "
214
+ "same lines: TrOCR 4.3 lines a second, the reader 13.3, with its draft 18.2. Whole regions, 64 at a time: 11.9 a "
215
+ "second with the reader alone, 12.7 with its draft."
216
+ )
217
+
218
+
219
  class StepClock(StoppingCriteria):
220
  """Notes when each decoding step ends; it never stops the generation itself."""
221
 
222
  def __init__(self) -> None:
223
  self.times: list[float] = []
224
+ self._never: torch.Tensor | None = None # made once: a tensor a step would cost the reader it times
225
 
226
  def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs: Any) -> torch.BoolTensor:
227
  self.times.append(time.perf_counter())
228
+ if self._never is None:
229
+ self._never = torch.zeros(input_ids.shape[0], dtype=torch.bool, device=input_ids.device)
230
+ return cast(torch.BoolTensor, self._never)
231
 
232
 
233
  def _example(example_id: str) -> dict[str, Any]:
 
259
  return inputs
260
 
261
 
262
+ def _trocr_ids(lines: list[Image.Image], clock: StepClock | None = None, max_new_tokens: int = 128) -> torch.Tensor:
263
+ """TrOCR's greedy token ids for a batch of lines, as Riksarkivet's evaluation runs it: the Swedish Lion starts
264
+ decoding from its config's token (its generation_config names another, which costs about 2 points of CER), and
265
+ its 192 x 1024 inputs need the position encoding interpolated."""
266
  model, images = trocr, trocr_processor
267
  if model is None or images is None:
268
  raise RuntimeError("TrOCR is not loaded")
 
 
 
269
  pixels = images(images=lines, return_tensors="pt").pixel_values.to("cuda")
270
+ return model.generate(
271
  pixels,
272
  decoder_start_token_id=model.config.decoder_start_token_id,
273
  interpolate_pos_encoding=True,
274
  num_beams=1,
275
  max_new_tokens=max_new_tokens,
276
+ stopping_criteria=StoppingCriteriaList([clock] if clock else []),
277
  )
278
+
279
+
280
+ def _trocr_read(lines: list[Image.Image]) -> TrocrLane:
281
+ """TrOCR's readings of a region's lines, one line after the other, timed from the images to the token ids; then
282
+ the seconds the same lines take as one batch."""
283
+ images = trocr_processor
284
+ if images is None:
285
+ raise RuntimeError("TrOCR is not loaded")
286
  torch.cuda.synchronize()
287
+ started = time.perf_counter()
288
+ rows = []
289
+ for line in lines:
290
+ clock = StepClock()
291
+ ids = _trocr_ids([line], clock)[0, 1:].tolist() # after the start token; the copy waits for the GPU
292
+ rows.append((ids, [round(at - started, 4) for at in clock.times]))
293
  seconds = time.perf_counter() - started
294
+ _, batch_seconds = _timed(lambda: _trocr_ids(lines))
295
  tokenizer = images.tokenizer
296
  read = []
297
+ for ids, steps in rows:
298
+ text = tokenizer.decode(ids, skip_special_tokens=True).strip()
299
  ends, shown = [], 0
300
  for step in range(1, len(steps) + 1):
301
  if shown < len(text):
302
+ partial = tokenizer.decode(ids[:step], skip_special_tokens=True).strip()
303
  shown = max(shown, min(len(text), len(partial)))
304
  ends.append(shown)
305
+ read.append(TrocrLine(text=text, steps=steps, ends=ends))
306
+ return TrocrLane(seconds=seconds, batch_seconds=batch_seconds, lines=read)
307
 
308
 
309
  def _reader_lane(decoded: D.Decoded, seconds: float) -> ReaderLane:
 
359
  warm = Image.open(HERE / "examples" / EXAMPLES[0]["id"] / "region.png").convert("RGB")
360
  engine.warm(_region_inputs(warm), EOS)
361
  if trocr is not None:
362
+ _trocr_ids([warm.crop((0, 0, 200, 40))], max_new_tokens=4)
363
  _jev_score(_jev_crop(warm.crop((0, 0, 200, 40))), "warm")
364
  _engines[os.getpid()] = engine
365
  return engine
 
374
 
375
 
376
  def _summary(race: Race, identical: bool) -> str:
377
+ plain, spec, lines = race.plain, race.dspark, race.trocr
378
  if plain is None or spec is None:
379
  return ""
380
+ said = [
381
+ f"One stream each, nothing batched: with its draft the reader read the region in {spec.seconds:.2f} s, "
382
+ f"{plain.seconds / spec.seconds:.1f}× faster than alone"
383
+ ]
384
+ if lines is not None:
385
+ said[0] += (
386
+ f" and {lines.seconds / spec.seconds:.1f}× faster than TrOCR reading the {len(lines.lines)} lines one "
387
+ "after the other"
388
+ )
389
+ said.append(
390
+ f"As one batch the lines decode side by side, and TrOCR took {lines.batch_seconds:.2f} s "
391
+ f"({lines.batch_seconds / spec.seconds:.1f}× the reader with its draft); a region is one sequence, so "
392
+ "the reader has nothing to batch here. It batches whole regions, in vLLM"
393
+ )
394
+ said.append(
395
+ f"The draft: {spec.passes} reader passes instead of {plain.passes} for {spec.tokens} tokens, "
396
+ f"{spec.acceptance or 0:.1f} tokens kept per check on average (at most 16)"
397
+ )
398
+ said.append(
399
+ "It wrote exactly what the reader writes alone"
400
  if identical
401
  else "Its reading differs from the reader's alone at a near-tied token: the reader checks 16 tokens at once, "
402
+ "and a batch of another size can tip a tie the other way (the reader alone does the same between batch sizes)"
 
 
 
 
 
403
  )
404
+ return ". ".join(said) + "."
405
+
406
+
407
+ def _marks(reading: str, truth: str) -> list[tuple[str, bool]]:
408
+ """A reading in stretches, each with whether it agrees with the ground truth; where the reading lacks characters,
409
+ a caret stands for them."""
410
+ reading, truth = " ".join(reading.split()), " ".join(truth.split())
411
+ marks = []
412
+ for step in Levenshtein.opcodes(reading, truth):
413
+ if step.tag == "insert":
414
+ marks.append(("‸", False))
415
+ else:
416
+ marks.append((reading[step.src_start : step.src_end], step.tag == "equal"))
417
+ return marks
418
+
419
+
420
+ def _jev_row(index: int, image: str, truth: str, crop: Image.Image, guess: str, own: str) -> JevRow:
421
+ """A line's row: Jev's score of TrOCR's reading and of the reader's, and what its higher score amounts to."""
422
+ first = JevReading(marks=_marks(guess, truth), score=_jev_score(crop, guess), exact=_same(guess, truth))
423
+ if _same(guess, own):
424
+ second, pick = first, ""
425
+ verdict = "both right" if first.exact else "both wrong, the same way"
426
+ else:
427
+ second = JevReading(marks=_marks(own, truth), score=_jev_score(crop, own), exact=_same(own, truth))
428
+ pick = "minicpm" if second.score > first.score else "trocr"
429
+ if first.exact == second.exact:
430
+ verdict = "both wrong" # two different readings cannot both be the ground truth
431
+ else:
432
+ right = "trocr" if first.exact else "minicpm"
433
+ verdict = "Jev picks the right one" if pick == right else "Jev picks the wrong one"
434
+ return JevRow(index=index, image=image, truth=truth, trocr=first, minicpm=second, pick=pick, verdict=verdict)
435
 
436
 
437
  def _jev_note(race: Race, truth: list[str]) -> str:
438
+ rows = race.jev
439
  whole = " ".join(truth)
440
+ picked = [row.minicpm.text if row.pick == "minicpm" else row.trocr.text for row in rows]
441
+ cer = [
442
+ _cer(" ".join(texts), whole) for texts in ([r.trocr.text for r in rows], [r.minicpm.text for r in rows], picked)
443
+ ]
444
+ one_right = [row for row in rows if row.trocr.exact != row.minicpm.exact]
445
+ said = [f"Both readers are right on {sum(row.verdict == 'both right' for row in rows)} of {len(rows)} lines"]
446
+ hits = sum(row.verdict == "Jev picks the right one" for row in one_right)
447
+ if len(one_right) == 1:
448
+ said.append(
449
+ f"On the one where only one of them is right, Jev scored the {'right' if hits else 'wrong'} one higher"
450
+ )
451
+ elif one_right:
452
+ said.append(
453
+ f"On the {len(one_right)} where only one of them is right, Jev scored the right one higher on {hits}"
454
+ )
455
+ readings = [reading for row in rows for reading in (row.trocr, row.minicpm)]
456
+ right = [reading.score for reading in readings if reading.exact]
457
+ wrong = [reading.score for reading in readings if not reading.exact]
458
+ if right and wrong:
459
+ said.append(
460
+ f"Jev's score averages {sum(right) / len(right):.2f} on right readings and "
461
+ f"{sum(wrong) / len(wrong):.2f} on wrong ones"
462
+ )
463
+ said.append(
464
+ f"CER over these lines, each read on its own: TrOCR {100 * cer[0]:.1f} %, MiniCPM-OCR {100 * cer[1]:.1f} %, "
465
+ f"Jev's pick on each line {100 * cer[2]:.1f} %"
466
  )
467
+ return ". ".join(said) + "."
468
 
469
 
470
+ def _duration(example_id: str | None, upload: Image.Image | None, with_jev: bool) -> int:
471
+ return 75 if upload is not None else 55 + (15 if with_jev else 0)
472
 
473
 
474
  @spaces.GPU(duration=_duration)
475
+ def compare(example_id: str | None, upload: Image.Image | None, with_jev: bool) -> Iterator[Race]:
476
+ """Read one region with one stream each, one reader after the other, timing each from the images to the text:
477
+ TrOCR on its lines one by one (and once more as one batch), the MiniCPM-OCR reader alone, and the reader with its
478
+ DSpark draft (the last two on the same CUDA graphs). For an example region, the reader then reads each line on
479
+ its own, and Jev scores that reading and TrOCR's. Yields the race so far between the timed stretches."""
480
  race = Race(status="Preparing the GPU worker: a fresh one captures its CUDA graphs first…")
481
  yield race
482
  with _lock:
483
  engine = _engine()
484
+ if upload is not None or example_id is None:
485
+ region, lines, truth, limit = cast(Image.Image, upload), [], None, UPLOAD_TOKENS
486
  else:
487
  folder = HERE / "examples" / example_id
488
  truth = _example(example_id)["lines"]
 
494
  if trocr is None:
495
  race.notice = NO_TROCR
496
  elif lines:
497
+ race.status = f"TrOCR is reading the region's {len(lines)} lines, one after the other, then as one batch…"
498
  yield race
499
  race.trocr = _trocr_read(lines)
500
  race.trocr.cer = _cer("\n".join(race.trocr.texts), whole) if whole else None
 
511
  race.dspark = _reader_lane(spec, seconds)
512
  race.dspark.cer = _cer(race.dspark.text, whole) if whole else None
513
  race.summary = _summary(race, identical=plain.tokens == spec.tokens)
514
+ race.ready = True
515
 
516
+ if with_jev and race.trocr is not None and truth is not None and example_id is not None:
517
  race.jev_total = len(lines)
518
  race.status = "The reader reads each line on its own, and Jev scores its reading and TrOCR's…"
519
  yield race
520
  for i, line in enumerate(lines):
 
 
521
  own = _text(engine.decode(_region_inputs(line), LINE_TOKENS, EOS).tokens)
522
+ image = f"/gradio_api/file={HERE / 'examples' / example_id / f'line_{i:02d}.jpg'}"
523
+ race.jev.append(_jev_row(i, image, truth[i], _jev_crop(line), race.trocr.lines[i].text, own))
 
 
 
 
 
 
 
 
 
 
524
  yield race
525
  race.jev_note = _jev_note(race, truth)
526
  race.status, race.done = "", True
527
  yield race
528
 
529
 
530
+ def _stage(example_id: str | None, upload: Image.Image | None) -> Race:
531
+ """The race before anyone has read: the region's image and, for an example, its lines' outlines and its timings
532
+ on our server."""
533
+ if upload is not None or example_id is None:
534
+ shown = cast(Image.Image, upload).copy()
535
  shown.thumbnail((1000, 1000))
536
  buffer = io.BytesIO()
537
  shown.save(buffer, format="JPEG", quality=85)
538
  image = "data:image/jpeg;base64," + base64.b64encode(buffer.getvalue()).decode()
539
+ return Race(image=image, aspect=shown.width / shown.height, batched=BATCHED, batched_note=BATCHED_NOTE)
540
  example = _example(example_id)
541
  width, height = example["region_size"]
542
  lengths = [max(1, len(line)) for line in example["lines"]]
543
  shares = [sum(lengths[: i + 1]) / sum(lengths) for i in range(len(lengths))]
544
+ times = REFERENCE.regions.get(example_id)
545
  return Race(
546
  image=f"/gradio_api/file={HERE / 'examples' / example_id / 'region.png'}",
547
  aspect=width / height,
548
  outlines=example["outlines"],
549
  shares=shares,
550
+ server=ServerTimes(**times.model_dump(), gpu=REFERENCE.gpu, measured=REFERENCE.measured, runs=REFERENCE.runs)
551
+ if times
552
+ else None,
553
+ server_said=_server_said(times) if times else "",
554
+ batched=BATCHED,
555
+ batched_note=BATCHED_NOTE,
556
  )
557
 
558
 
559
+ def _server_said(times: RegionTimes) -> str:
560
+ return (
561
+ f"Served by vLLM, the reader read this region in {times.vllm:.2f} s alone and {times.vllm_dspark:.2f} s with "
562
+ f"its draft: {times.trocr_single / times.vllm:.1f}× and {times.trocr_single / times.vllm_dspark:.1f}× faster "
563
+ f"than TrOCR reading the {times.lines} lines one after the other, {times.trocr_batch / times.vllm:.1f}× and "
564
+ f"{times.trocr_batch / times.vllm_dspark:.1f}× faster than TrOCR's one batch. {SAME_GPU}"
565
+ )
566
+
567
+
568
+ def _example_of(image: str) -> str | None:
569
+ """The example whose region is the image in the box, or None for a visitor's own image."""
570
+ for example in EXAMPLES:
571
+ if filecmp.cmp(image, HERE / "examples" / example["id"] / "region.png", shallow=False):
572
+ return example["id"]
573
+ return None
574
+
575
+
576
+ def run(region: str | None, with_jev: bool) -> Iterator[dict[str, Any]]:
577
+ """Read the region in the box (an example, or a crop of your own) with one stream each: TrOCR on an example's
578
+ lines, the MiniCPM-OCR reader and the reader with its DSpark draft; then let Jev score each line of an example.
579
+ Streams the race as each reader is timed: every reading with the time each token became known, the summary, the
580
+ same region's timings on our server, and Jev's rows."""
581
+ if region is None:
582
+ raise gr.Error("Pick an example region, or drop a region's crop on the box.")
583
+ example_id = _example_of(region)
584
+ upload = None if example_id else ImageOps.exif_transpose(Image.open(region)).convert("RGB")
585
  stage = _stage(example_id, upload)
586
  stage.status = "Waiting for a GPU…"
587
  yield stage.model_dump()
588
+ keep = {
589
+ name: getattr(stage, name)
590
+ for name in ("run", "image", "aspect", "outlines", "shares", "server", "server_said", "batched", "batched_note")
591
+ }
592
  for race in compare(example_id, upload, with_jev):
593
  yield race.model_copy(update=keep).model_dump()
594
 
 
607
 
608
  def _label(example: dict[str, Any]) -> str:
609
  collection = example["collection"].removeprefix("GT-").rsplit("_", 1)[0].replace("_", " ")
610
+ return f"{len(example['lines'])} lines · {collection}"
611
+
612
 
613
+ def _region(example: dict[str, Any]) -> str:
614
+ return str(HERE / "examples" / example["id"] / "region.png")
615
+
616
+
617
+ def _picked(event: gr.SelectData) -> str:
618
+ return _region(EXAMPLES[event.index])
619
 
 
620
 
621
  with gr.Blocks(title="Swedish HTR: TrOCR, MiniCPM-OCR and DSpark") as demo:
622
  gr.Markdown(
623
  "# Reading Swedish handwriting faster\n"
624
+ "One text region from an archive volume, read on the same GPU with **one stream each and nothing batched**. "
625
+ "Riksarkivet's **Swedish Lion TrOCR** cannot read a region, so it gets the region's text lines, cut out "
626
+ "beforehand, and reads them one after the other (its time for all of them as one batch is shown beside). Our "
627
+ f"**[MiniCPM-OCR reader](https://huggingface.co/{READER})** reads the region as one image, alone and with "
628
+ f"its **[DSpark draft](https://huggingface.co/{DRAFT})**, which guesses 15 tokens at a time for the reader "
629
+ f"to check in one pass. Then **[Jev](https://huggingface.co/{JEV})**, the reader's proof-reader, scores both "
630
+ "readers' reading of each line. Under the race: the same region on our server, where **vLLM** serves the "
631
+ "reader, and what the readers do there when everything is batched."
632
  )
633
  with gr.Tab("Compare"):
634
  with gr.Row():
635
+ with gr.Column(scale=3):
636
+ region = gr.Image(
637
+ value=_region(EXAMPLES[0]),
638
+ type="filepath",
639
+ image_mode=None,
640
+ sources=["upload", "clipboard"],
641
+ label="The region to read: an example from below, or your own crop (upload or paste it)",
642
+ height=340,
643
+ buttons=["fullscreen"],
 
 
 
 
644
  )
645
+ with gr.Column(scale=1, min_width=260):
646
+ with_jev = gr.Checkbox(value=True, label="Score each line with Jev (examples only)")
647
+ go = gr.Button("Read", variant="primary")
648
+ gr.Markdown("Only the reader reads a crop of your own: TrOCR and Jev need the region's lines.")
649
+ gallery = gr.Gallery(
650
+ [(str(HERE / "examples" / e["id"] / "thumb.jpg"), _label(e)) for e in EXAMPLES],
651
+ label="Example regions, from volumes none of the models trained on: click one to put it in the box",
652
+ columns=6,
653
+ height=230,
654
+ object_fit="cover",
655
+ allow_preview=False,
656
+ interactive=False,
657
+ buttons=[],
658
  )
659
+ race_view = RaceView()
660
+ gallery.select(_picked, None, region, api_visibility="private")
661
+ go.click(run, [region, with_jev], race_view, api_name="compare", show_progress="minimal")
662
  with gr.Tab("Try Jev"):
663
  gr.Markdown(
664
  "Jev scores whether a text is exactly what a **single handwritten line** says. Upload a line crop and type "
examples/19a1b2195bab_right_0/thumb.jpg ADDED
examples/35ffe6b8b600_left_0/thumb.jpg ADDED
examples/38c4ce6331ad_left_1/thumb.jpg ADDED
examples/8d24e7df7dea_2/thumb.jpg ADDED
examples/eed569a6699c_left_3/thumb.jpg ADDED
examples/f970d28ac08b_left_0/thumb.jpg ADDED
measure_reference.py ADDED
@@ -0,0 +1,147 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """The Space's reference timings: each example region read on this machine by the Swedish Lion TrOCR (transformers,
2
+ its lines one after the other and as one batch, as the app runs it) and by the reader in vLLM, alone and with its
3
+ DSpark draft, one request at a time. The app shows them beside its live race, which runs in PyTorch because ZeroGPU
4
+ cannot host vLLM.
5
+
6
+ Serve the reader twice first, as the draft's card says (prefix caching off), the second server once the first
7
+ answers, on a GPU nothing else is using:
8
+
9
+ vllm serve READER --served-model-name htr ... --port 8057
10
+ vllm serve READER --served-model-name htr ... --port 8056 --speculative-config '{"method": "dspark", ...}'
11
+ python measure_reference.py --examples examples --gpu "RTX PRO 6000 Blackwell" --out reference.json
12
+
13
+ Each reading is timed from the encoded images to the reading: the median of RUNS runs after one that is not timed.
14
+ vLLM keeps the processed form of an image it has seen, so every request sends the region with one pixel, drawn at
15
+ random, changed by one level: the region is new to the server each time, as a region is in production.
16
+ """
17
+
18
+ from __future__ import annotations
19
+
20
+ import argparse
21
+ import base64
22
+ import io
23
+ import json
24
+ import secrets
25
+ import statistics
26
+ import time
27
+ from collections.abc import Callable
28
+ from datetime import date
29
+ from pathlib import Path
30
+ from typing import Any
31
+
32
+ import httpx
33
+ import torch
34
+ from PIL import Image
35
+ from transformers import TrOCRProcessor, VisionEncoderDecoderModel
36
+
37
+ from reference import Reference, RegionTimes
38
+
39
+ SAMPLING = {"temperature": 0.01, "top_p": 0.001, "top_k": 1, "repetition_penalty": 1.0} # OpenBMB's, near greedy
40
+ IMAGES = {"downsample_mode": "4x", "max_slice_nums": 9}
41
+
42
+
43
+ def median_seconds(read: Callable[[int], None], runs: int) -> float:
44
+ """The median time of `runs` calls of `read`, after one that is not timed; each call gets its own number."""
45
+ read(0)
46
+ times = []
47
+ for run in range(1, runs + 1):
48
+ started = time.perf_counter()
49
+ read(run)
50
+ times.append(time.perf_counter() - started)
51
+ return round(statistics.median(times), 3)
52
+
53
+
54
+ def requests_for(region: Path, count: int) -> list[dict[str, Any]]:
55
+ """`count` requests for one region, each with one pixel, drawn at random, changed by one level."""
56
+ image = Image.open(region).convert("RGB")
57
+ pixels = image.tobytes()
58
+ bodies = []
59
+ for _ in range(count):
60
+ changed = bytearray(pixels)
61
+ changed[3 * secrets.randbelow(image.width * image.height)] ^= 1 # a pixel's red
62
+ buffer = io.BytesIO()
63
+ Image.frombytes("RGB", image.size, bytes(changed)).save(buffer, format="PNG")
64
+ url = "data:image/png;base64," + base64.b64encode(buffer.getvalue()).decode()
65
+ content = [{"type": "image_url", "image_url": {"url": url}}, {"type": "text", "text": "Text Recognition:"}]
66
+ bodies.append(
67
+ {
68
+ "model": "htr",
69
+ **SAMPLING,
70
+ "max_tokens": 4096,
71
+ "chat_template_kwargs": {"enable_thinking": False},
72
+ "mm_processor_kwargs": IMAGES,
73
+ "messages": [{"role": "user", "content": content}],
74
+ }
75
+ )
76
+ return bodies
77
+
78
+
79
+ class Vllm:
80
+ """One vLLM server, asked for one region at a time; it keeps what the region's requests answered."""
81
+
82
+ def __init__(self, endpoint: str, client: httpx.Client) -> None:
83
+ self.url = f"{endpoint}/v1/chat/completions"
84
+ self.client = client
85
+ self.texts: set[str] = set()
86
+ self.tokens = 0
87
+
88
+ def read(self, body: dict[str, Any]) -> None:
89
+ reply = self.client.post(self.url, json=body)
90
+ reply.raise_for_status()
91
+ answer = reply.json()
92
+ self.texts.add(answer["choices"][0]["message"]["content"])
93
+ self.tokens = answer["usage"]["completion_tokens"]
94
+
95
+
96
+ def main() -> None:
97
+ parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
98
+ parser.add_argument("--examples", type=Path, required=True, help="the folder build_examples.py wrote")
99
+ parser.add_argument("--trocr", default="Gabriel/swedish-lion-trocr-large-handwritten-char")
100
+ parser.add_argument("--alone", default="http://localhost:8057", help="vLLM serving the reader alone")
101
+ parser.add_argument("--draft", default="http://localhost:8056", help="vLLM serving the reader with its draft")
102
+ parser.add_argument("--gpu", required=True, help="the GPU's name, as the app will show it")
103
+ parser.add_argument("--runs", type=int, default=5)
104
+ parser.add_argument("--out", type=Path, required=True)
105
+ args = parser.parse_args()
106
+
107
+ processor = TrOCRProcessor.from_pretrained(args.trocr)
108
+ trocr = VisionEncoderDecoderModel.from_pretrained(args.trocr).to("cuda").eval()
109
+
110
+ def read_lines(files: list[bytes], batch: int) -> None:
111
+ lines = [Image.open(io.BytesIO(data)).convert("RGB") for data in files]
112
+ for first in range(0, len(lines), batch):
113
+ pixels = processor(images=lines[first : first + batch], return_tensors="pt").pixel_values.to("cuda")
114
+ with torch.inference_mode():
115
+ trocr.generate(
116
+ pixels,
117
+ decoder_start_token_id=trocr.config.decoder_start_token_id,
118
+ interpolate_pos_encoding=True,
119
+ num_beams=1,
120
+ max_new_tokens=128,
121
+ )
122
+ torch.cuda.synchronize()
123
+
124
+ regions = {}
125
+ with httpx.Client(timeout=600) as client:
126
+ for example in json.loads((args.examples / "examples.json").read_text(encoding="utf-8")):
127
+ folder = args.examples / example["id"]
128
+ files = [(folder / f"line_{i:02d}.jpg").read_bytes() for i in range(len(example["lines"]))]
129
+ bodies = requests_for(folder / "region.png", args.runs + 1)
130
+ alone, draft = Vllm(args.alone, client), Vllm(args.draft, client)
131
+ times = RegionTimes(
132
+ lines=len(files),
133
+ trocr_single=median_seconds(lambda _, files=files: read_lines(files, 1), args.runs),
134
+ trocr_batch=median_seconds(lambda _, files=files: read_lines(files, len(files)), args.runs),
135
+ vllm=median_seconds(lambda run, alone=alone, bodies=bodies: alone.read(bodies[run]), args.runs),
136
+ vllm_dspark=median_seconds(lambda run, draft=draft, bodies=bodies: draft.read(bodies[run]), args.runs),
137
+ tokens=alone.tokens,
138
+ )
139
+ regions[example["id"]] = times
140
+ same = "the same text" if alone.texts == draft.texts and len(alone.texts) == 1 else "texts differ"
141
+ print(example["id"], times.model_dump(), f"{len(alone.texts)} and {len(draft.texts)} readings, {same}")
142
+ reference = Reference(gpu=args.gpu, measured=date.today().isoformat(), runs=args.runs, regions=regions)
143
+ args.out.write_text(reference.model_dump_json(indent=1) + "\n", encoding="utf-8")
144
+
145
+
146
+ if __name__ == "__main__":
147
+ main()
race_view.css CHANGED
@@ -2,10 +2,11 @@
2
  --rv-trocr: #b07a2a;
3
  --rv-plain: #1f62b0;
4
  --rv-dspark: #1a8a5a;
 
5
  --rv-muted: color-mix(in srgb, var(--body-text-color) 70%, transparent);
6
  display: flex;
7
  flex-direction: column;
8
- gap: 14px;
9
  color: var(--body-text-color);
10
  font-size: 14px;
11
  line-height: 1.5;
@@ -14,6 +15,7 @@
14
  --rv-trocr: #b8842f;
15
  --rv-plain: #4f8fdb;
16
  --rv-dspark: #3fa97a;
 
17
  }
18
  .rv-root [hidden] { display: none; }
19
  .rv-hint, .rv-status { color: var(--rv-muted); margin: 0; }
@@ -26,11 +28,15 @@
26
  max-width: 100ch;
27
  }
28
  .rv-status::before { content: "● "; animation: rv-pulse 1.2s ease-in-out infinite; }
29
- .rv-summary { margin: 0; max-width: 90ch; }
 
 
 
30
  .rv-k { font-size: 12px; color: var(--rv-muted); letter-spacing: 0.02em; }
31
 
 
32
  .rv-track { display: flex; flex-direction: column; gap: 8px; }
33
- .rv-row { display: grid; grid-template-columns: minmax(150px, 230px) minmax(0, 1fr) 64px; gap: 10px; align-items: center; }
34
  .rv-name, .rv-head { display: flex; align-items: center; gap: 8px; min-width: 0; }
35
  .rv-swatch { width: 10px; height: 10px; border-radius: 2px; flex: none; background: var(--rv-lane); }
36
  [data-lane="trocr"] { --rv-lane: var(--rv-trocr); }
@@ -39,6 +45,27 @@
39
  .rv-bar { height: 14px; border-radius: 4px; background: var(--background-fill-secondary); overflow: hidden; }
40
  .rv-fillbar { height: 100%; width: 0; border-radius: 4px; background: var(--rv-lane); }
41
  .rv-time { text-align: right; font-variant-numeric: tabular-nums; }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
42
 
43
  .rv-controls { display: flex; flex-wrap: wrap; align-items: center; gap: 8px; }
44
  .rv-button {
@@ -53,7 +80,7 @@
53
  .rv-button:hover { border-color: var(--rv-muted); }
54
  .rv-main { font-weight: 600; color: #fff; background: var(--rv-dspark); border-color: var(--rv-dspark); padding: 5px 16px; }
55
  .rv-main:hover { filter: brightness(1.08); border-color: var(--rv-dspark); }
56
- .rv-button:focus-visible { outline: 2px solid var(--rv-plain); outline-offset: 2px; }
57
 
58
  .rv-lanes { display: grid; grid-template-columns: repeat(auto-fit, minmax(250px, 1fr)); gap: 12px; }
59
  .rv-lane {
@@ -106,18 +133,40 @@
106
  .rv-text .rv-draft { animation: rv-flash 0.5s ease-out; }
107
  .rv-own { text-decoration: underline; text-decoration-color: var(--rv-dspark); text-decoration-thickness: 2px; text-underline-offset: 3px; }
108
 
109
- .rv-jhead { margin: 6px 0 0; font-size: 15px; }
110
- .rv-jintro { margin: 4px 0; max-width: 90ch; }
111
- .rv-jnote { margin: 0; font-weight: 600; }
112
- .rv-jlist { display: flex; flex-direction: column; margin-top: 8px; }
113
- .rv-jrow { display: grid; gap: 4px; padding: 10px 0; border-top: 1px solid var(--border-color-primary); }
114
- .rv-jimg { max-width: 100%; max-height: 64px; border-radius: 3px; background: #fff; }
115
- .rv-truth, .rv-cand { display: grid; grid-template-columns: 96px minmax(0, 1fr) 90px 36px 112px 76px; gap: 8px; align-items: center; }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
116
  .rv-reading { overflow-wrap: anywhere; }
 
 
 
 
 
 
 
 
 
117
  .rv-meter { height: 8px; border-radius: 4px; background: var(--background-fill-secondary); overflow: hidden; }
118
  .rv-meter i { display: block; height: 100%; border-radius: 4px; background: var(--rv-lane); }
119
  .rv-score { text-align: right; font-variant-numeric: tabular-nums; }
120
- .rv-tag { font-size: 12px; color: var(--rv-muted); }
121
  .rv-pick { font-size: 12px; font-weight: 600; }
122
  .rv-picked .rv-reading { font-weight: 600; }
123
 
@@ -126,9 +175,11 @@
126
  @media (max-width: 640px) {
127
  .rv-row { grid-template-columns: minmax(0, 1fr) 64px; }
128
  .rv-row .rv-bar { grid-column: 1 / -1; grid-row: 2; }
129
- .rv-truth, .rv-cand { grid-template-columns: 70px minmax(0, 1fr) 36px; }
130
- .rv-cand .rv-meter, .rv-cand .rv-tag, .rv-cand .rv-pick { grid-column: 2 / -1; }
 
131
  }
132
  @media (prefers-reduced-motion: reduce) {
133
  .rv-text .rv-draft, .rv-status::before { animation: none; }
 
134
  }
 
2
  --rv-trocr: #b07a2a;
3
  --rv-plain: #1f62b0;
4
  --rv-dspark: #1a8a5a;
5
+ --rv-bad: #c4342d;
6
  --rv-muted: color-mix(in srgb, var(--body-text-color) 70%, transparent);
7
  display: flex;
8
  flex-direction: column;
9
+ gap: 18px;
10
  color: var(--body-text-color);
11
  font-size: 14px;
12
  line-height: 1.5;
 
15
  --rv-trocr: #b8842f;
16
  --rv-plain: #4f8fdb;
17
  --rv-dspark: #3fa97a;
18
+ --rv-bad: #ff8a7a;
19
  }
20
  .rv-root [hidden] { display: none; }
21
  .rv-hint, .rv-status { color: var(--rv-muted); margin: 0; }
 
28
  max-width: 100ch;
29
  }
30
  .rv-status::before { content: "● "; animation: rv-pulse 1.2s ease-in-out infinite; }
31
+ .rv-group { display: flex; flex-direction: column; gap: 10px; }
32
+ .rv-title { margin: 0; font-size: 15px; font-weight: 600; }
33
+ .rv-summary, .rv-said, .rv-note, .rv-jintro { margin: 0; max-width: 100ch; }
34
+ .rv-note, .rv-jintro { font-size: 13px; color: var(--rv-muted); }
35
  .rv-k { font-size: 12px; color: var(--rv-muted); letter-spacing: 0.02em; }
36
 
37
+ /* a bar per timed stretch; the live bars and our server's share one scale */
38
  .rv-track { display: flex; flex-direction: column; gap: 8px; }
39
+ .rv-row { display: grid; grid-template-columns: minmax(170px, 250px) minmax(0, 1fr) 64px; gap: 10px; align-items: center; }
40
  .rv-name, .rv-head { display: flex; align-items: center; gap: 8px; min-width: 0; }
41
  .rv-swatch { width: 10px; height: 10px; border-radius: 2px; flex: none; background: var(--rv-lane); }
42
  [data-lane="trocr"] { --rv-lane: var(--rv-trocr); }
 
45
  .rv-bar { height: 14px; border-radius: 4px; background: var(--background-fill-secondary); overflow: hidden; }
46
  .rv-fillbar { height: 100%; width: 0; border-radius: 4px; background: var(--rv-lane); }
47
  .rv-time { text-align: right; font-variant-numeric: tabular-nums; }
48
+ /* measured beforehand, not here: tinted and outlined instead of solid */
49
+ .rv-server .rv-fillbar {
50
+ background: color-mix(in srgb, var(--rv-lane) 38%, transparent);
51
+ box-shadow: inset 0 0 0 1.5px var(--rv-lane);
52
+ transition: width 0.4s ease;
53
+ }
54
+ .rv-server .rv-swatch { background: color-mix(in srgb, var(--rv-lane) 38%, transparent); box-shadow: inset 0 0 0 1.5px var(--rv-lane); }
55
+
56
+ .rv-tiles { display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 10px; }
57
+ .rv-tile {
58
+ display: flex;
59
+ flex-direction: column;
60
+ gap: 2px;
61
+ padding: 10px 12px;
62
+ border: 1px solid var(--border-color-primary);
63
+ border-top: 3px solid var(--rv-lane);
64
+ border-radius: 6px;
65
+ }
66
+ .rv-tilename { font-weight: 600; }
67
+ .rv-rate { font-size: 30px; font-weight: 650; line-height: 1.15; font-variant-numeric: tabular-nums; }
68
+ .rv-unit { font-size: 13px; font-weight: 400; color: var(--rv-muted); }
69
 
70
  .rv-controls { display: flex; flex-wrap: wrap; align-items: center; gap: 8px; }
71
  .rv-button {
 
80
  .rv-button:hover { border-color: var(--rv-muted); }
81
  .rv-main { font-weight: 600; color: #fff; background: var(--rv-dspark); border-color: var(--rv-dspark); padding: 5px 16px; }
82
  .rv-main:hover { filter: brightness(1.08); border-color: var(--rv-dspark); }
83
+ .rv-button:focus-visible, .rv-foldhead:focus-visible { outline: 2px solid var(--rv-plain); outline-offset: 2px; }
84
 
85
  .rv-lanes { display: grid; grid-template-columns: repeat(auto-fit, minmax(250px, 1fr)); gap: 12px; }
86
  .rv-lane {
 
133
  .rv-text .rv-draft { animation: rv-flash 0.5s ease-out; }
134
  .rv-own { text-decoration: underline; text-decoration-color: var(--rv-dspark); text-decoration-thickness: 2px; text-underline-offset: 3px; }
135
 
136
+ /* Jev: what it found in a sentence, then the lines, folded away */
137
+ .rv-jnote { margin: 0; max-width: 100ch; font-weight: 600; }
138
+ .rv-fold { border: 1px solid var(--border-color-primary); border-radius: 6px; padding: 0 12px; }
139
+ .rv-foldhead { padding: 9px 0; cursor: pointer; font-weight: 600; }
140
+ .rv-fold[open] .rv-foldhead { border-bottom: 1px solid var(--border-color-primary); margin-bottom: 10px; }
141
+ .rv-jr { display: grid; grid-template-columns: 92px minmax(0, 1fr) 68px 138px 72px; gap: 10px; align-items: center; }
142
+ .rv-jhdr { margin-top: 10px; padding-bottom: 6px; font-size: 12px; font-weight: 600; color: var(--rv-muted); }
143
+ .rv-jlist { display: flex; flex-direction: column; }
144
+ .rv-jrow { display: grid; gap: 5px; padding: 10px 0; border-top: 1px solid var(--border-color-primary); }
145
+ .rv-jtop { display: flex; flex-wrap: wrap; align-items: center; justify-content: space-between; gap: 8px; }
146
+ .rv-jimg { max-width: min(100%, 720px); max-height: 64px; border-radius: 3px; background: #fff; }
147
+ .rv-verdict {
148
+ font-size: 12px;
149
+ padding: 1px 9px;
150
+ border-radius: 999px;
151
+ border: 1px solid var(--border-color-primary);
152
+ color: var(--rv-muted);
153
+ white-space: nowrap;
154
+ }
155
+ .rv-verdict[data-kind="good"] { color: var(--body-text-color); border-color: var(--rv-dspark); font-weight: 600; }
156
+ .rv-verdict[data-kind="bad"] { color: var(--body-text-color); border-color: var(--rv-bad); font-weight: 600; }
157
  .rv-reading { overflow-wrap: anywhere; }
158
+ .rv-bad {
159
+ background: color-mix(in srgb, var(--rv-bad) 20%, transparent);
160
+ text-decoration: underline wavy var(--rv-bad);
161
+ text-underline-offset: 3px;
162
+ border-radius: 2px;
163
+ }
164
+ .rv-right { font-size: 12px; color: var(--rv-muted); }
165
+ .rv-wrong { font-size: 12px; font-weight: 600; color: var(--rv-bad); }
166
+ .rv-scorebox { display: grid; grid-template-columns: minmax(0, 1fr) 42px; gap: 8px; align-items: center; }
167
  .rv-meter { height: 8px; border-radius: 4px; background: var(--background-fill-secondary); overflow: hidden; }
168
  .rv-meter i { display: block; height: 100%; border-radius: 4px; background: var(--rv-lane); }
169
  .rv-score { text-align: right; font-variant-numeric: tabular-nums; }
 
170
  .rv-pick { font-size: 12px; font-weight: 600; }
171
  .rv-picked .rv-reading { font-weight: 600; }
172
 
 
175
  @media (max-width: 640px) {
176
  .rv-row { grid-template-columns: minmax(0, 1fr) 64px; }
177
  .rv-row .rv-bar { grid-column: 1 / -1; grid-row: 2; }
178
+ .rv-jr { grid-template-columns: 76px minmax(0, 1fr) 60px; }
179
+ .rv-jr .rv-scorebox, .rv-jr .rv-pick { grid-column: 2 / -1; }
180
+ .rv-jhdr span:nth-child(n + 4) { display: none; }
181
  }
182
  @media (prefers-reduced-motion: reduce) {
183
  .rv-text .rv-draft, .rv-status::before { animation: none; }
184
+ .rv-server .rv-fillbar { transition: none; }
185
  }
race_view.js CHANGED
@@ -1,20 +1,42 @@
1
- // RaceView: replays three readers at their measured pace, on the manuscript and as text, then lists Jev's scores.
 
2
  // Gradio runs this once with `element`, `props` and `watch` in scope; `props.value` is a Race payload (app.py).
3
  // Every text is set with textContent: readings are model output and are never parsed as HTML.
4
 
5
  const SVG = "http://www.w3.org/2000/svg";
6
  const LANES = [
7
- { key: "trocr", name: "Swedish Lion TrOCR", how: "the region as its text lines, cut out beforehand; one batch" },
8
  { key: "plain", name: "MiniCPM-OCR reader alone", how: "the region as one image; one token per reader pass" },
9
  { key: "dspark", name: "Reader + DSpark draft", how: "the region as one image; up to 16 tokens per reader pass" },
10
  ];
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  const SPEEDS = [
12
- { label: "▶ Race them side by side", speed: 1, main: true },
13
  { label: "½ speed", speed: 0.5 },
14
  { label: "¼ speed", speed: 0.25 },
15
  ];
 
 
 
 
 
 
 
16
  const still = window.matchMedia && window.matchMedia("(prefers-reduced-motion: reduce)").matches;
17
- const state = { run: null, mode: "live", speed: 1, raceStart: 0, started: {}, frame: 0 };
18
  const derived = new WeakMap(); // per lane payload: what drawing needs, worked out once
19
 
20
  function make(tag, cls, text) {
@@ -46,6 +68,12 @@ function percent(rate) {
46
  return rate === null || rate === undefined ? "" : ` · CER ${(100 * rate).toFixed(1)} %`;
47
  }
48
 
 
 
 
 
 
 
49
  // A reader lane's pieces as drawing needs them: when each came, the characters read by then, and its passes
50
  // (pieces that became known together).
51
  function readerFacts(lane) {
@@ -111,37 +139,28 @@ function inkLines(column, amount) {
111
  });
112
  }
113
 
114
- // The skeleton, built again whenever Gradio re-renders the template or a new run starts.
115
- function ensure(race) {
116
- let root = element.querySelector(".rv-root");
117
- if (!root) {
118
- root = make("div", "rv-root");
119
- element.appendChild(root);
120
- }
121
- root.dataset.theme = document.body.classList.contains("dark") ? "dark" : "light";
122
- const built = root.dataset.run === race.run && root.dataset.image === (race.image || "");
123
- if (built && root.querySelector(".rv-lanes")) return root;
124
- root.dataset.run = race.run;
125
- root.dataset.image = race.image || "";
126
- const track = make("div", "rv-track");
127
  const lanes = make("div", "rv-lanes");
128
  for (const lane of LANES) {
129
- const row = make("div", "rv-row");
130
- row.dataset.lane = lane.key;
131
- const name = make("div", "rv-name");
132
- name.append(make("i", "rv-swatch"), make("span", "", lane.name));
133
- const bar = make("div", "rv-bar");
134
- bar.append(make("div", "rv-fillbar"));
135
- row.append(name, bar, make("div", "rv-time", "waiting"));
136
- track.append(row);
137
-
138
  const column = make("section", "rv-lane");
139
  column.dataset.lane = lane.key;
140
  const head = make("header", "rv-head");
141
  head.append(make("i", "rv-swatch"), make("b", "", lane.name));
142
  const body = make("div", "rv-text");
143
  body.dataset.n = "0";
144
- column.append(head, make("div", "rv-how", lane.how), make("div", "rv-meta", "\u00a0"), page(race, lane.key));
145
  if (lane.key !== "trocr") column.append(make("div", "rv-passes"));
146
  column.append(body);
147
  if (lane.key === "dspark") {
@@ -155,6 +174,25 @@ function ensure(race) {
155
  }
156
  lanes.append(column);
157
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
158
  const controls = make("div", "rv-controls");
159
  controls.hidden = true;
160
  for (const choice of SPEEDS) {
@@ -163,22 +201,35 @@ function ensure(race) {
163
  button.dataset.speed = String(choice.speed);
164
  controls.append(button);
165
  }
 
 
 
 
 
 
 
 
 
 
 
 
166
  root.replaceChildren(
167
  make("div", "rv-notice"),
168
  make("div", "rv-status"),
169
- track,
170
- make("p", "rv-summary"),
171
- controls,
172
- lanes,
173
- make("div", "rv-jev"),
174
  );
175
  return root;
176
  }
177
 
178
- function elapsed(key, now) {
 
 
179
  if (still) return Infinity;
180
- if (state.mode === "race") return ((now - state.raceStart) / 1000) * state.speed;
181
- return state.started[key] === undefined ? 0 : (now - state.started[key]) / 1000;
182
  }
183
 
184
  function countUntil(times, at, from) {
@@ -240,9 +291,9 @@ function drawReader(column, race, lane, at, finished, key) {
240
  }
241
  }
242
 
 
243
  function drawTrocr(column, lane, at, finished) {
244
  const body = column.querySelector(".rv-text");
245
- const step = finished ? lane.steps.length : countUntil(lane.steps, at, Number(body.dataset.n || 0));
246
  if (body.dataset.kind !== "trocr" || body.childElementCount !== lane.lines.length) {
247
  body.replaceChildren(
248
  ...lane.lines.map((line, i) => {
@@ -253,77 +304,127 @@ function drawTrocr(column, lane, at, finished) {
253
  );
254
  body.dataset.kind = "trocr";
255
  }
256
- const reached = (line) => (step === 0 ? 0 : line.ends[Math.min(step, line.ends.length) - 1]);
 
 
 
 
 
 
257
  lane.lines.forEach((line, i) => {
258
- body.children[i].textContent = line.text.slice(0, reached(line)) || " ";
259
  });
260
- body.dataset.n = String(step);
261
- inkLines(column, (i) => (lane.lines[i] ? reached(lane.lines[i]) / Math.max(1, lane.lines[i].text.length) : 0));
262
  const meta = column.querySelector(".rv-meta");
263
- meta.textContent = finished
264
- ? `${lane.lines.length} lines in ${lane.steps.length} decoding steps${percent(lane.cer)}`
265
- : `step ${step} of ${lane.steps.length}, every line at once`;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
266
  }
267
 
268
- function candidate(name, key, text, score, exact, picked) {
269
- const box = make("div", picked ? "rv-cand rv-picked" : "rv-cand");
270
- box.dataset.lane = key;
 
 
 
271
  const meter = make("span", "rv-meter");
272
  const fill = make("i");
273
- fill.style.width = `${Math.round(100 * score)}%`;
274
  meter.append(fill);
275
- box.append(
 
 
276
  make("span", "rv-k", name),
277
- make("span", "rv-reading", text || "(empty)"),
278
- meter,
279
- make("span", "rv-score", score.toFixed(2)),
280
- make("span", "rv-tag", exact ? "= ground truth" : "≠ ground truth"),
281
  );
282
- if (picked) box.append(make("span", "rv-pick", "Jev's pick"));
283
- return box;
284
  }
285
 
286
  function drawJev(root, race) {
287
  const box = root.querySelector(".rv-jev");
288
  const rows = race.jev || [];
289
- if (!race.jev_total) {
290
- box.replaceChildren();
291
- return;
292
- }
293
  let list = box.querySelector(".rv-jlist");
294
  if (!list) {
295
  list = make("div", "rv-jlist");
296
- const intro =
297
- "Each line is read on its own by TrOCR and by the MiniCPM reader. Jev looks at the line and at each reading " +
298
- "drawn as an image, and scores from 0 to 1 whether the reading is exactly what the line says.";
299
- box.replaceChildren(
300
- make("h4", "rv-jhead", "Jev, line by line"),
301
- make("p", "rv-jintro", intro),
302
- make("p", "rv-jnote"),
303
- list,
304
- );
305
  }
306
  for (let i = list.childElementCount; i < rows.length; i += 1) {
307
  const row = rows[i];
308
  const item = make("div", "rv-jrow");
309
  item.dataset.line = String(row.index);
 
310
  const image = make("img", "rv-jimg");
311
  image.src = row.image;
312
  image.alt = `Line ${row.index + 1} of the region`;
313
  image.loading = "lazy";
314
- const truth = make("div", "rv-truth");
 
 
 
315
  truth.append(make("span", "rv-k", "ground truth"), make("span", "rv-reading", row.truth));
316
- const pickMinicpm = row.minicpm_score > row.trocr_score;
317
  item.append(
318
- image,
319
  truth,
320
- candidate("TrOCR", "trocr", row.trocr, row.trocr_score, row.trocr_exact, !pickMinicpm),
321
- candidate("MiniCPM", "plain", row.minicpm, row.minicpm_score, row.minicpm_exact, pickMinicpm),
322
  );
323
  list.append(item);
324
  }
325
  box.querySelector(".rv-jnote").textContent =
326
  race.jev_note || `Jev has scored ${rows.length} of ${race.jev_total} lines…`;
 
 
327
  }
328
 
329
  function draw(now) {
@@ -331,37 +432,47 @@ function draw(now) {
331
  const race = payload();
332
  if (!race) return;
333
  const root = ensure(race);
334
- const status = root.querySelector(".rv-status");
335
- status.textContent = race.status || "";
336
- status.hidden = !race.status;
337
- const notice = root.querySelector(".rv-notice");
338
- notice.textContent = race.notice || "";
339
- notice.hidden = !race.notice;
340
- const summary = root.querySelector(".rv-summary");
341
- summary.textContent = race.summary || "";
342
- summary.hidden = !race.summary;
343
- root.querySelector(".rv-controls").hidden = !race.done || still;
344
 
345
- const scale = Math.max(0.001, ...LANES.map((lane) => (race[lane.key] ? race[lane.key].seconds : 0)));
 
 
 
 
346
  let running = false;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
347
  for (const lane of LANES) {
348
  const data = race[lane.key];
349
- const row = root.querySelector(`.rv-row[data-lane="${lane.key}"]`);
350
  const column = root.querySelector(`.rv-lane[data-lane="${lane.key}"]`);
351
- if (!data) {
352
- const skipped = race.done || (lane.key === "trocr" && race.plain);
353
- row.querySelector(".rv-time").textContent = skipped ? "not run" : "waiting";
354
- column.classList.toggle("rv-skipped", Boolean(skipped));
355
  continue;
356
  }
357
- const at = Math.min(Math.max(0, elapsed(lane.key, now)), data.seconds);
358
- const finished = at >= data.seconds;
359
- running = running || !finished;
360
- row.querySelector(".rv-fillbar").style.width = `${(100 * at) / scale}%`;
361
- row.querySelector(".rv-time").textContent = `${at.toFixed(2)} s`;
362
- if (lane.key === "trocr") drawTrocr(column, data, at, finished);
363
- else drawReader(column, race, data, at, finished, lane.key);
364
  }
 
365
  drawJev(root, race);
366
  if (running) state.frame = requestAnimationFrame(draw);
367
  }
@@ -369,21 +480,17 @@ function draw(now) {
369
  function sync() {
370
  const race = payload();
371
  if (!race) return;
372
- const now = performance.now();
373
- if (race.run !== state.run) {
374
- Object.assign(state, { run: race.run, mode: "live", speed: 1, started: {} });
375
- }
376
- for (const lane of LANES) {
377
- if (race[lane.key] && state.started[lane.key] === undefined) state.started[lane.key] = now;
378
- }
379
  if (state.frame) cancelAnimationFrame(state.frame);
380
- draw(now);
381
  }
382
 
383
  element.addEventListener("click", (event) => {
384
  const button = event.target.closest("[data-speed]");
385
  if (!button) return;
386
- Object.assign(state, { mode: "race", speed: Number(button.dataset.speed), raceStart: performance.now() });
387
  if (state.frame) cancelAnimationFrame(state.frame);
388
  state.frame = requestAnimationFrame(draw);
389
  });
 
1
+ // RaceView: replays three readers from the same start at their measured pace, on the manuscript and as text; shows
2
+ // the same region's timings on our server; then lists Jev's scores line by line.
3
  // Gradio runs this once with `element`, `props` and `watch` in scope; `props.value` is a Race payload (app.py).
4
  // Every text is set with textContent: readings are model output and are never parsed as HTML.
5
 
6
  const SVG = "http://www.w3.org/2000/svg";
7
  const LANES = [
8
+ { key: "trocr", name: "Swedish Lion TrOCR", how: "the region as its text lines, cut out beforehand; one line after the other" },
9
  { key: "plain", name: "MiniCPM-OCR reader alone", how: "the region as one image; one token per reader pass" },
10
  { key: "dspark", name: "Reader + DSpark draft", how: "the region as one image; up to 16 tokens per reader pass" },
11
  ];
12
+ // The stretches timed here: each lane's own, and TrOCR's lines once more as one batch.
13
+ const BARS = [
14
+ { key: "trocr", lane: "trocr", name: "TrOCR · one line after the other", seconds: (race) => race.trocr && race.trocr.seconds },
15
+ { key: "batch", lane: "trocr", name: "TrOCR · its lines as one batch", seconds: (race) => race.trocr && race.trocr.batch_seconds },
16
+ { key: "plain", lane: "plain", name: "MiniCPM-OCR reader alone", seconds: (race) => race.plain && race.plain.seconds },
17
+ { key: "dspark", lane: "dspark", name: "Reader + DSpark draft", seconds: (race) => race.dspark && race.dspark.seconds },
18
+ ];
19
+ // The same stretches on our server (Race.server), the reader served by vLLM.
20
+ const SERVER = [
21
+ { field: "trocr_single", lane: "trocr", name: "TrOCR · one line after the other" },
22
+ { field: "trocr_batch", lane: "trocr", name: "TrOCR · its lines as one batch" },
23
+ { field: "vllm", lane: "plain", name: "Reader alone · in vLLM" },
24
+ { field: "vllm_dspark", lane: "dspark", name: "Reader + DSpark · in vLLM" },
25
+ ];
26
  const SPEEDS = [
27
+ { label: "▶ Race them again", speed: 1, main: true },
28
  { label: "½ speed", speed: 0.5 },
29
  { label: "¼ speed", speed: 0.25 },
30
  ];
31
+ const JEV_INTRO =
32
+ "Each line is read on its own by TrOCR and by the MiniCPM-OCR reader. Marked characters differ from the ground " +
33
+ "truth (the line as a person transcribed it); ‸ stands for missing ones. Jev looks at the line and at a reading " +
34
+ "drawn as an image, and scores from 0 to 1 how sure it is that the reading is exactly what the line says. Where " +
35
+ "the two readings differ, the one Jev scores higher is its pick. Jev catches a reading that is plainly another " +
36
+ "text far better than one that is a letter or two off (see How it works), so a high score does not prove a " +
37
+ "reading right.";
38
  const still = window.matchMedia && window.matchMedia("(prefers-reduced-motion: reduce)").matches;
39
+ const state = { run: null, raced: false, speed: 1, raceStart: 0, frame: 0 };
40
  const derived = new WeakMap(); // per lane payload: what drawing needs, worked out once
41
 
42
  function make(tag, cls, text) {
 
68
  return rate === null || rate === undefined ? "" : ` · CER ${(100 * rate).toFixed(1)} %`;
69
  }
70
 
71
+ function write(root, selector, text) {
72
+ const node = root.querySelector(selector);
73
+ node.textContent = text || "";
74
+ node.hidden = !text;
75
+ }
76
+
77
  // A reader lane's pieces as drawing needs them: when each came, the characters read by then, and its passes
78
  // (pieces that became known together).
79
  function readerFacts(lane) {
 
139
  });
140
  }
141
 
142
+ function barRow(key, lane, name) {
143
+ const row = make("div", "rv-row");
144
+ row.dataset.bar = key;
145
+ row.dataset.lane = lane;
146
+ const label = make("div", "rv-name");
147
+ label.append(make("i", "rv-swatch"), make("span", "", name));
148
+ const bar = make("div", "rv-bar");
149
+ bar.append(make("div", "rv-fillbar"));
150
+ row.append(label, bar, make("div", "rv-time"));
151
+ return row;
152
+ }
153
+
154
+ function lanesOf(race) {
155
  const lanes = make("div", "rv-lanes");
156
  for (const lane of LANES) {
 
 
 
 
 
 
 
 
 
157
  const column = make("section", "rv-lane");
158
  column.dataset.lane = lane.key;
159
  const head = make("header", "rv-head");
160
  head.append(make("i", "rv-swatch"), make("b", "", lane.name));
161
  const body = make("div", "rv-text");
162
  body.dataset.n = "0";
163
+ column.append(head, make("div", "rv-how", lane.how), make("div", "rv-meta", " "), page(race, lane.key));
164
  if (lane.key !== "trocr") column.append(make("div", "rv-passes"));
165
  column.append(body);
166
  if (lane.key === "dspark") {
 
174
  }
175
  lanes.append(column);
176
  }
177
+ return lanes;
178
+ }
179
+
180
+ // The skeleton, built again whenever Gradio re-renders the template or a new run starts.
181
+ function ensure(race) {
182
+ let root = element.querySelector(".rv-root");
183
+ if (!root) {
184
+ root = make("div", "rv-root");
185
+ element.appendChild(root);
186
+ }
187
+ root.dataset.theme = document.body.classList.contains("dark") ? "dark" : "light";
188
+ const built = root.dataset.run === race.run && root.dataset.image === (race.image || "");
189
+ if (built && root.querySelector(".rv-lanes")) return root;
190
+ root.dataset.run = race.run;
191
+ root.dataset.image = race.image || "";
192
+
193
+ const live = make("section", "rv-group rv-live");
194
+ const track = make("div", "rv-track");
195
+ for (const bar of BARS) track.append(barRow(bar.key, bar.lane, bar.name));
196
  const controls = make("div", "rv-controls");
197
  controls.hidden = true;
198
  for (const choice of SPEEDS) {
 
201
  button.dataset.speed = String(choice.speed);
202
  controls.append(button);
203
  }
204
+ const liveTitle = "Here, live: one stream each, in PyTorch on half a GPU";
205
+ live.append(make("h4", "rv-title", liveTitle), track, controls, make("p", "rv-summary"));
206
+
207
+ const server = make("section", "rv-group rv-server");
208
+ const serverTrack = make("div", "rv-track");
209
+ for (const bar of SERVER) serverTrack.append(barRow(bar.field, bar.lane, bar.name));
210
+ server.append(make("h4", "rv-title"), serverTrack, make("p", "rv-said"), make("p", "rv-note"));
211
+
212
+ const batched = make("section", "rv-group rv-batched");
213
+ const batchedTitle = "Everything batched, on our server: lines read a second";
214
+ batched.append(make("h4", "rv-title", batchedTitle), make("div", "rv-tiles"), make("p", "rv-note"));
215
+
216
  root.replaceChildren(
217
  make("div", "rv-notice"),
218
  make("div", "rv-status"),
219
+ live,
220
+ lanesOf(race),
221
+ server,
222
+ batched,
223
+ make("section", "rv-group rv-jev"),
224
  );
225
  return root;
226
  }
227
 
228
+ // Seconds into the replay: none before every reader is timed, all of them where motion is unwelcome.
229
+ function elapsed(now) {
230
+ if (!state.raced) return 0;
231
  if (still) return Infinity;
232
+ return ((now - state.raceStart) / 1000) * state.speed;
 
233
  }
234
 
235
  function countUntil(times, at, from) {
 
291
  }
292
  }
293
 
294
+ // TrOCR reads its lines one after the other: each line's steps carry their own times.
295
  function drawTrocr(column, lane, at, finished) {
296
  const body = column.querySelector(".rv-text");
 
297
  if (body.dataset.kind !== "trocr" || body.childElementCount !== lane.lines.length) {
298
  body.replaceChildren(
299
  ...lane.lines.map((line, i) => {
 
304
  );
305
  body.dataset.kind = "trocr";
306
  }
307
+ let current = lane.lines.length;
308
+ const reached = lane.lines.map((line, i) => {
309
+ if (finished) return line.text.length;
310
+ const steps = countUntil(line.steps, at, 0);
311
+ if (steps < line.steps.length && current === lane.lines.length) current = i;
312
+ return steps === 0 ? 0 : line.ends[steps - 1];
313
+ });
314
  lane.lines.forEach((line, i) => {
315
+ body.children[i].textContent = line.text.slice(0, reached[i]) || " ";
316
  });
317
+ inkLines(column, (i) => (lane.lines[i] ? reached[i] / Math.max(1, lane.lines[i].text.length) : 0));
 
318
  const meta = column.querySelector(".rv-meta");
319
+ if (finished) {
320
+ const steps = lane.lines.reduce((sum, line) => sum + line.steps.length, 0);
321
+ meta.textContent = `${lane.lines.length} lines in ${steps} decoding steps${percent(lane.cer)}`;
322
+ } else {
323
+ meta.textContent = `line ${Math.min(current + 1, lane.lines.length)} of ${lane.lines.length}`;
324
+ }
325
+ }
326
+
327
+ function drawServer(root, race, scale) {
328
+ const box = root.querySelector(".rv-server");
329
+ const server = race.server;
330
+ box.hidden = !server;
331
+ if (server) {
332
+ box.querySelector(".rv-title").textContent =
333
+ `The same region on our server: a whole ${server.gpu}, the reader served by vLLM`;
334
+ for (const bar of SERVER) {
335
+ const row = box.querySelector(`[data-bar="${bar.field}"]`);
336
+ row.querySelector(".rv-fillbar").style.width = `${(100 * server[bar.field]) / scale}%`;
337
+ row.querySelector(".rv-time").textContent = `${server[bar.field].toFixed(2)} s`;
338
+ }
339
+ box.querySelector(".rv-said").textContent = race.server_said || "";
340
+ box.querySelector(".rv-note").textContent =
341
+ `Measured beforehand (${server.measured}): one request at a time, each image new to the server, the median ` +
342
+ `of ${server.runs} runs. This Space gets half of such a GPU and decodes in PyTorch, since ZeroGPU cannot ` +
343
+ "host vLLM. These bars and the live ones share one scale.";
344
+ }
345
+ const batched = root.querySelector(".rv-batched");
346
+ const rates = race.batched || [];
347
+ batched.hidden = !rates.length;
348
+ const tiles = batched.querySelector(".rv-tiles");
349
+ if (tiles.childElementCount !== rates.length) {
350
+ tiles.replaceChildren(
351
+ ...rates.map((rate) => {
352
+ const tile = make("div", "rv-tile");
353
+ tile.dataset.lane = rate.lane;
354
+ const figure = make("div", "rv-rate", rate.per_second.toFixed(rate.per_second < 10 ? 1 : 0));
355
+ figure.append(make("span", "rv-unit", " lines a second"));
356
+ tile.append(make("div", "rv-tilename", rate.name), figure, make("div", "rv-how", rate.how));
357
+ return tile;
358
+ }),
359
+ );
360
+ }
361
+ batched.querySelector(".rv-note").textContent = race.batched_note || "";
362
  }
363
 
364
+ function reading(name, key, data, picked) {
365
+ const row = make("div", picked ? "rv-jr rv-picked" : "rv-jr");
366
+ row.dataset.lane = key;
367
+ const text = make("span", "rv-reading");
368
+ for (const [piece, agrees] of data.marks) text.append(make("span", agrees ? "" : "rv-bad", piece));
369
+ if (!data.marks.length) text.textContent = "(empty)";
370
  const meter = make("span", "rv-meter");
371
  const fill = make("i");
372
+ fill.style.width = `${Math.round(100 * data.score)}%`;
373
  meter.append(fill);
374
+ const score = make("span", "rv-scorebox");
375
+ score.append(meter, make("span", "rv-score", data.score.toFixed(3)));
376
+ row.append(
377
  make("span", "rv-k", name),
378
+ text,
379
+ make("span", data.exact ? "rv-right" : "rv-wrong", data.exact ? "✓ right" : "✗ wrong"),
380
+ score,
381
+ make("span", "rv-pick", picked ? "Jev's pick" : ""),
382
  );
383
+ return row;
 
384
  }
385
 
386
  function drawJev(root, race) {
387
  const box = root.querySelector(".rv-jev");
388
  const rows = race.jev || [];
389
+ box.hidden = !race.jev_total;
390
+ if (!race.jev_total) return;
 
 
391
  let list = box.querySelector(".rv-jlist");
392
  if (!list) {
393
  list = make("div", "rv-jlist");
394
+ const header = make("div", "rv-jr rv-jhdr");
395
+ for (const name of ["Reader", "Its reading of the line", "Right?", "Jev's score, 0 to 1", ""]) {
396
+ header.append(make("span", "", name));
397
+ }
398
+ const fold = make("details", "rv-fold");
399
+ fold.append(make("summary", "rv-foldhead"), make("p", "rv-jintro", JEV_INTRO), header, list);
400
+ box.replaceChildren(make("h4", "rv-title", "Jev, the reader's proof-reader"), make("p", "rv-jnote"), fold);
 
 
401
  }
402
  for (let i = list.childElementCount; i < rows.length; i += 1) {
403
  const row = rows[i];
404
  const item = make("div", "rv-jrow");
405
  item.dataset.line = String(row.index);
406
+ const top = make("div", "rv-jtop");
407
  const image = make("img", "rv-jimg");
408
  image.src = row.image;
409
  image.alt = `Line ${row.index + 1} of the region`;
410
  image.loading = "lazy";
411
+ const verdict = make("span", "rv-verdict", row.verdict);
412
+ verdict.dataset.kind = row.verdict.startsWith("Jev picks the right") ? "good" : row.verdict === "both right" ? "" : "bad";
413
+ top.append(image, verdict);
414
+ const truth = make("div", "rv-jr rv-truth");
415
  truth.append(make("span", "rv-k", "ground truth"), make("span", "rv-reading", row.truth));
 
416
  item.append(
417
+ top,
418
  truth,
419
+ reading("TrOCR", "trocr", row.trocr, row.pick === "trocr"),
420
+ reading("MiniCPM-OCR", "plain", row.minicpm, row.pick === "minicpm"),
421
  );
422
  list.append(item);
423
  }
424
  box.querySelector(".rv-jnote").textContent =
425
  race.jev_note || `Jev has scored ${rows.length} of ${race.jev_total} lines…`;
426
+ box.querySelector(".rv-foldhead").textContent =
427
+ `Line by line: both readings and Jev's scores (${rows.length} of ${race.jev_total} lines)`;
428
  }
429
 
430
  function draw(now) {
 
432
  const race = payload();
433
  if (!race) return;
434
  const root = ensure(race);
435
+ write(root, ".rv-status", race.status);
436
+ write(root, ".rv-notice", race.notice);
437
+ write(root, ".rv-summary", race.summary);
438
+ root.querySelector(".rv-controls").hidden = !race.ready || still;
 
 
 
 
 
 
439
 
440
+ const server = race.server;
441
+ const known = BARS.map((bar) => bar.seconds(race) || 0);
442
+ if (server) known.push(...SERVER.map((bar) => server[bar.field]));
443
+ const scale = Math.max(0.001, ...known);
444
+ const at = elapsed(now);
445
  let running = false;
446
+ for (const bar of BARS) {
447
+ const row = root.querySelector(`.rv-live [data-bar="${bar.key}"]`);
448
+ const total = bar.seconds(race);
449
+ const fill = row.querySelector(".rv-fillbar");
450
+ const time = row.querySelector(".rv-time");
451
+ if (!total || !state.raced) {
452
+ const skipped = !total && (race.done || (bar.lane === "trocr" && race.plain));
453
+ fill.style.width = "0";
454
+ time.textContent = total ? "timed" : skipped ? "not run" : "waiting";
455
+ continue;
456
+ }
457
+ const shown = Math.min(at, total);
458
+ running = running || shown < total;
459
+ fill.style.width = `${(100 * shown) / scale}%`;
460
+ time.textContent = `${shown.toFixed(2)} s`;
461
+ }
462
  for (const lane of LANES) {
463
  const data = race[lane.key];
 
464
  const column = root.querySelector(`.rv-lane[data-lane="${lane.key}"]`);
465
+ column.classList.toggle("rv-skipped", !data && Boolean(race.done || (lane.key === "trocr" && race.plain)));
466
+ if (!data) continue;
467
+ if (!state.raced) {
468
+ column.querySelector(".rv-meta").textContent = "timed; the replay starts when all are";
469
  continue;
470
  }
471
+ const shown = Math.min(at, data.seconds);
472
+ if (lane.key === "trocr") drawTrocr(column, data, shown, shown >= data.seconds);
473
+ else drawReader(column, race, data, shown, shown >= data.seconds, lane.key);
 
 
 
 
474
  }
475
+ drawServer(root, race, scale);
476
  drawJev(root, race);
477
  if (running) state.frame = requestAnimationFrame(draw);
478
  }
 
480
  function sync() {
481
  const race = payload();
482
  if (!race) return;
483
+ if (race.run !== state.run) Object.assign(state, { run: race.run, raced: false, speed: 1 });
484
+ // every reader is timed: they start together
485
+ if (race.ready && !state.raced) Object.assign(state, { raced: true, speed: 1, raceStart: performance.now() });
 
 
 
 
486
  if (state.frame) cancelAnimationFrame(state.frame);
487
+ draw(performance.now());
488
  }
489
 
490
  element.addEventListener("click", (event) => {
491
  const button = event.target.closest("[data-speed]");
492
  if (!button) return;
493
+ Object.assign(state, { raced: true, speed: Number(button.dataset.speed), raceStart: performance.now() });
494
  if (state.frame) cancelAnimationFrame(state.frame);
495
  state.frame = requestAnimationFrame(draw);
496
  });
race_view.py CHANGED
@@ -1,5 +1,6 @@
1
- """RaceView, a custom Gradio component: three readers' readings replayed at their measured pace, then Jev's scores
2
- line by line. Its value is a `Race` payload (app.py); the drawing is in race_view.js and race_view.css."""
 
3
 
4
  from __future__ import annotations
5
 
@@ -10,15 +11,16 @@ import gradio as gr
10
 
11
  _HERE = Path(__file__).parent
12
  HINT = (
13
- "Pick a region and press Read: the three readers run one after the other, and each reading is replayed here at "
14
- "the pace it was measured."
15
  )
16
  TEMPLATE = f'<div class="rv-root"><p class="rv-hint">{HINT}</p></div>'
17
 
18
 
19
  class RaceView(gr.HTML):
20
- """Replays a `Race`: a bar and a text lane per reader, growing as the reader produced its tokens, and one row per
21
- line with Jev's scores. Readings are written as text, never as HTML."""
 
22
 
23
  def __init__(self, value: dict[str, Any] | None = None, **kwargs: Any) -> None:
24
  super().__init__(
 
1
+ """RaceView, a custom Gradio component: three readers' readings replayed from the same start at their measured pace,
2
+ the same region's timings on our server, then Jev's scores line by line. Its value is a `Race` payload (app.py); the
3
+ drawing is in race_view.js and race_view.css."""
4
 
5
  from __future__ import annotations
6
 
 
11
 
12
  _HERE = Path(__file__).parent
13
  HINT = (
14
+ "Pick a region and press Read: the three readers run one after the other, and once all are timed their "
15
+ "readings are replayed here from the same start, each at the pace it was measured."
16
  )
17
  TEMPLATE = f'<div class="rv-root"><p class="rv-hint">{HINT}</p></div>'
18
 
19
 
20
  class RaceView(gr.HTML):
21
+ """Replays a `Race`: a bar per timed stretch and a text lane per reader, growing as the reader produced its
22
+ tokens; the same stretches on our server; and one row per line with Jev's scores. Readings are written as text,
23
+ never as HTML."""
24
 
25
  def __init__(self, value: dict[str, Any] | None = None, **kwargs: Any) -> None:
26
  super().__init__(
reference.json ADDED
@@ -0,0 +1,55 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "gpu": "RTX PRO 6000 Blackwell",
3
+ "measured": "2026-11-11",
4
+ "runs": 5,
5
+ "regions": {
6
+ "8d24e7df7dea_2": {
7
+ "lines": 10,
8
+ "tokens": 124,
9
+ "trocr_single": 1.661,
10
+ "trocr_batch": 0.837,
11
+ "vllm": 0.497,
12
+ "vllm_dspark": 0.321
13
+ },
14
+ "19a1b2195bab_right_0": {
15
+ "lines": 8,
16
+ "tokens": 76,
17
+ "trocr_single": 1.246,
18
+ "trocr_batch": 0.603,
19
+ "vllm": 0.379,
20
+ "vllm_dspark": 0.266
21
+ },
22
+ "eed569a6699c_left_3": {
23
+ "lines": 9,
24
+ "tokens": 96,
25
+ "trocr_single": 1.859,
26
+ "trocr_batch": 0.752,
27
+ "vllm": 0.525,
28
+ "vllm_dspark": 0.345
29
+ },
30
+ "f970d28ac08b_left_0": {
31
+ "lines": 11,
32
+ "tokens": 129,
33
+ "trocr_single": 1.474,
34
+ "trocr_batch": 0.668,
35
+ "vllm": 0.569,
36
+ "vllm_dspark": 0.335
37
+ },
38
+ "38c4ce6331ad_left_1": {
39
+ "lines": 17,
40
+ "tokens": 209,
41
+ "trocr_single": 3.922,
42
+ "trocr_batch": 1.543,
43
+ "vllm": 0.688,
44
+ "vllm_dspark": 0.334
45
+ },
46
+ "35ffe6b8b600_left_0": {
47
+ "lines": 18,
48
+ "tokens": 198,
49
+ "trocr_single": 3.412,
50
+ "trocr_batch": 1.323,
51
+ "vllm": 0.729,
52
+ "vllm_dspark": 0.45
53
+ }
54
+ }
55
+ }
reference.py ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """The Space's reference timings (reference.json): measure_reference.py writes them and app.py shows them beside its
2
+ live race."""
3
+
4
+ from __future__ import annotations
5
+
6
+ from pydantic import BaseModel
7
+
8
+
9
+ class RegionTimes(BaseModel):
10
+ """One region's median seconds, each from the encoded images to the reading: TrOCR on its lines one after the
11
+ other and as one batch, and the reader in vLLM alone and with its DSpark draft."""
12
+
13
+ lines: int
14
+ tokens: int
15
+ trocr_single: float
16
+ trocr_batch: float
17
+ vllm: float
18
+ vllm_dspark: float
19
+
20
+
21
+ class Reference(BaseModel):
22
+ """The timings of every example region, with the GPU and the day they were measured on."""
23
+
24
+ gpu: str
25
+ measured: str
26
+ runs: int
27
+ regions: dict[str, RegionTimes]