diffu-htr-speed / README.md
Gabriel's picture
One stream each: TrOCR reads its lines one by one (its one-batch time beside it), the replay starts all three together, our server's vLLM timings under the race, one region box, Jev's rows folded with marked errors
b0dacaa verified
|
Raw History Blame Contribute Delete
1.94 kB

A newer version of the Gradio SDK is available: 6.32.0

Upgrade
metadata
title: Swedish HTR speed and Jev
emoji: 🪶
colorFrom: green
colorTo: gray
sdk: gradio
sdk_version: 6.31.0
app_file: app.py
python_version: '3.12'
short_description: TrOCR vs MiniCPM-OCR with a DSpark draft, and Jev
license: apache-2.0
startup_duration_timeout: 1h
models:
  - Gabriel/diffu-minicpm-ocr-htr
  - Gabriel/diffu-minicpm-ocr-htr-dspark
  - Gabriel/diffu-minicpm-ocr-htr-jev
  - Gabriel/swedish-lion-trocr-large-handwritten-char

Swedish HTR speed and Jev

Reads the same handwritten text region with one stream each on the same GPU, times each and replays them from the same start: the Swedish Lion TrOCR on the region's text lines, one after the other (and once more as one batch), the MiniCPM-OCR HTR reader on the region as one image, and the same reader with its DSpark draft. Under the race: the same region's timings on our server, where vLLM serves the reader, and what the readers do there when everything is batched. Then Jev, the reader's proof-reader, scores both readers' reading of each line.

  • app.py: the Gradio app; the readers run one after the other and the page replays them at their measured pace.
  • dspark_hf.py: DSpark speculative decoding in PyTorch on CUDA graphs (ZeroGPU cannot host vLLM).
  • race_view.py, race_view.js, race_view.css: the custom component that draws the race, our server's timings and Jev's rows.
  • reference.json, reference.py, measure_reference.py: each example region's timings on our server (TrOCR in transformers, the reader in vLLM), and the script that measured them.
  • check_decode.py: compares the reader alone and with its draft on a GPU, token for token, and times them.

See the app's "How it works" tab. Part of diffu.