File size: 16,254 Bytes

---
language:
  - de
license: apache-2.0
library_name: onnxruntime
pipeline_tag: image-to-text
tags:
  - handwriting-recognition
  - optical-character-recognition
  - ctc
  - onnx
  - browser
  - german
metrics:
  - cer
  - wer
---

# DE·HTR: modern German handwriting recognition for the browser

> [!NOTE]
> **Superseded by [de-htr-web-v2](https://huggingface.co/naeyn/de-htr-web-v2)** — 8.01% CER vs 8.61% on the same sealed test split, with a fully audited license-clean training-data ancestry. v2 has a **new input contract** (grayscale input, in-model normalization, 112-symbol alphabet); this v1 remains stable for existing integrations.


DE·HTR is a compact line-level recognizer for **modern German handwriting**. The release artifact is a 9.84 MB int8 ONNX model that runs fully client-side with ONNX Runtime Web.

- **Live demo:** https://de-htr-web.vercel.app
- **Source and reproducibility:** https://github.com/naeyn/de-htr-web
- **Input:** one pre-cropped handwriting line (PNG, JPEG, or WebP)
- **Output:** a Unicode transcription produced by greedy CTC decoding
- **Privacy:** after the static page and model load, inference makes no network requests

## Why this model exists

Large general-purpose vision-language models will often be more accurate on unconstrained handwriting because they bring language priors and broad visual knowledge. DE·HTR instead targets a different operating point: a small inspectable artifact, predictable latency, offline execution, no per-call cost, and no image upload. On the same frozen 601-line test split, the measured 4-bit Qwen2.5-VL-7B baseline reached 9.44% CER (95% CI 8.33–10.65), while the three DE·HTR CTC seeds averaged 8.15% CER (seed 42: 7.40–9.10; that mean is the CUDA training figure — the published int8 artifact measures 8.5681% on CPU, see [ONNX and browser parity](#onnx-and-browser-parity)). Those intervals overlap substantially and no paired significance test was run, so the two should be read as **statistically indistinguishable on this test set** rather than as an accuracy win. The release argument is size, offline execution, and latency; this is not a claim that this model beats current large VLMs generally.

## Intended use and scope

Use this model to transcribe **pre-cropped single lines of contemporary German handwriting** in a browser or local ONNX Runtime process.

Out of scope:

- full-page layout analysis or line segmentation;
- historical Kurrent or Sütterlin handwriting;
- mobile-camera capture, dewarping, or deskew;
- personally identifiable information detection;
- multilingual recognition;
- correction or fine-tuning interfaces.

The model must not be represented as a full-page OCR system. It has no layout detector.

## Artifacts

| Path | Purpose |
|---|---|
| `model.onnx` | int8 ONNX release model; batch size 1, dynamic image width |
| `pytorch_model.pt` | selected seed-42 PyTorch inference checkpoint |
| `config.json` | alphabet, normalization, tensor names, and image geometry; its `model` field resolves `model.onnx` |
| `examples/infer_onnx.py` | runnable end-to-end inference example (preprocessing + greedy CTC decode) |
| `examples/0107-000.png` | one held-out **test**-split line for a first run, CC BY 4.0 (see `training-data/SOURCES.md`) |
| `results/` | committed metric JSON, predictions, confidence intervals, and browser evidence |
| `training-data/` | frozen page-disjoint split and complete dataset/font/dependency license registers |
| `LICENSE`, `NOTICE` | Apache License 2.0 and copyright/attribution notices |
| `THIRD_PARTY_NOTICES.md`, `third_party/` | third-party component licenses in full |

Artifact digests:

| Artifact | Bytes | SHA-256 |
|---|---:|---|
| `model.onnx` | 9,843,872 | `05cce04d3938a64edaee79caf2a74587eaa9f8c46c146d6e04f0512bfce51d82` |
| `pytorch_model.pt` | 38,978,889 | `9765f609c21ab0fe04ed66260d4000025c63a54b4fa370e9a73788da2dc6065f` |

## Usage

```bash
pip install onnxruntime pillow numpy huggingface_hub
huggingface-cli download naeyn/de-htr-web examples/infer_onnx.py examples/0107-000.png --local-dir de-htr
python de-htr/examples/infer_onnx.py de-htr/examples/0107-000.png
```

The script downloads `model.onnx` and `config.json` on first run and prints the transcription. On the bundled example it emits:

```
Process (eigene Schreibweise PROCESS) ist eine crormediale
```

which matches this repo's committed int8 prediction for that line (`results/m3/preds_onnx_int8_python.jsonl`); the reference transcription is `... eine crossmediale`, so the one visible error is expected rather than a setup problem. To run fully offline, pass `--model` and `--config` explicitly.

Doing it yourself takes four steps, all implemented in that file: convert to RGB and resize to height 64 preserving aspect ratio (cap width 2048, centered white padding); normalize with the ImageNet mean/std in `config.json`; run the session with input `images` and output `log_probs`; then greedy-CTC decode over the first `ceil(width / 4)` timesteps, collapsing repeats and dropping blanks (`blank_index` 0, so character `i` of `alphabet` is class `i + 1`).

## Model architecture

The recognizer has 9,730,308 parameters:

1. an ImageNet-pretrained ResNet-18 visual stem through layer 3 (torchvision `ResNet18_Weights.IMAGENET1K_V1`; see [License and attribution](#license-and-attribution) for its provenance);
2. a width-preserving projection to 384 features;
3. a three-layer bidirectional GRU with hidden size 384;
4. a 196-class CTC head (195 characters plus blank).

Input images are converted to RGB, resized to height 64 while preserving aspect ratio, capped at width 2048 with centered white padding when required, and normalized with ImageNet mean/std. The output time dimension is approximately input width / 4. Decoding is greedy: repeated labels collapse and blank labels are removed. There is no lexicon or language-model beam search.

The ONNX graph fixes batch size to one and keeps image width/output time dynamic. Large Conv, GRU, and Linear initializers use symmetric per-channel int8 weight-only QDQ representation.

## Training data

The frozen manifest is published at `training-data/split_manifest.json`. Full source URLs, access dates, and licenses are in `training-data/SOURCES.md`; all synthetic-rendering fonts and their SIL OFL 1.1 records are in `training-data/FONTS.md`.

Real line data:

- ScaDS.AI German Line- and Word-Level Handwriting Dataset (v1.0) by Thomas Burghardt and Ahmad Alzin, Zenodo, doi:10.5281/zenodo.18301532, licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/);
- 5,843 lines from 385 pages (the collection ships line-level ground truth for all 775 pages but line crops for only 385, so the split covers those 385);
- page-disjoint split: 309/38/38 pages and 4,675/567/601 lines for train/validation/test;
- **"writer" means "page" throughout.** The corpus provides no writer identifiers, so the manifest uses `page_id` as the writer proxy (`writer_id_field`). Each page carries one hand, but nothing in the corpus rules out one person having written several pages, so writer-level disjointness is a documented assumption rather than a verified property. The only identifying metadata, `(gender, yearofbirth)`, is too coarse to check it: the 385 split pages fall into 90 such profiles and every profile in test also occurs in train;
- ten visually corrupt or unalignable training pairs were excluded from CTC loss only; the frozen manifest was not changed.

Synthetic data:

- the selected mixture sampled 50% real and 50% synthetic examples per epoch;
- text came from German Wikipedia (CC BY-SA 4.0) after alphabet sanitization;
- line images were rendered with twelve redistributed SIL OFL 1.1 handwriting fonts;
- real Tier-1 word crops supplied word-image composition examples under CC BY 4.0;
- synthetic examples never entered validation or test.

Training used PyTorch 2.11.0+cu128 on an NVIDIA RTX A5000, bfloat16 AMP, batch size 24, 12,000 samples per epoch, two frozen-backbone epochs, AdamW at 3e-4, and deterministic algorithms. Synthetic ratio was selected by validation CER only; seed 42 was selected for release by validation CER only.

## Evaluation protocol

All headline results use the immutable page-disjoint 601-line real test set (see the writer-proxy note under [Training data](#training-data)). CER and WER normalize Unicode to NFC and collapse whitespace without case or punctuation folding. Confidence intervals are 95% percentile intervals from 10,000 line-level bootstrap resamples with NumPy RNG seed 0. Metric and prediction files were written by committed evaluation scripts; values below are not manually transcribed from model output.

### Baselines on the frozen test set

| System | CER (95% CI) | WER (95% CI) | Size | Median latency/line |
|---|---:|---:|---:|---:|
| Tesseract 5.3.4 `deu`, `--psm 7` | 43.42% (41.74–45.13) | 94.50% (92.06–96.91) | 8.63 MB | 162 ms |
| `microsoft/trocr-base-handwritten`, zero-shot English | 26.46% (25.08–27.89) | 85.62% (82.62–88.62) | 2.67 GB | 2,130 ms |
| Qwen2.5-VL-7B-Instruct 4-bit, zero-shot | 9.44% (8.33–10.65) | 23.64% (21.55–25.82) | 5.64 GB | 2,909 ms |

Baseline latency was measured on an Apple M1; implementations and environments are recorded in each JSON result. These latencies are not measured on equal footing: Tesseract runs as a fresh subprocess per line, so its 162 ms includes process spawn and model reload, while the neural baselines amortize model load across the run. The Tesseract figure is an upper bound and the comparison flatters this model.

### CTC training runs

| Run | Validation CER | Test CER (95% CI) | Test WER (95% CI) |
|---|---:|---:|---:|
| 0% synthetic, seed 42 | 8.22% | 9.54% (8.70–10.45) | 36.88% (34.81–38.96) |
| **50% synthetic, seed 42 (selected)** | **6.92%** | **8.21% (7.40–9.10)** | **32.33% (30.27–34.41)** |
| 80% synthetic, seed 42 | 7.18% | 8.36% (7.54–9.25) | 33.93% (31.83–36.08) |
| 50% synthetic, seed 43 | 7.11% | 7.99% (7.17–8.90) | 31.83% (29.74–33.98) |
| 50% synthetic, seed 44 | 7.10% | 8.25% (7.45–9.13) | 32.62% (30.50–34.75) |
| TrOCR-small trained reference, seed 42 | 11.10% | 11.51% (10.61–12.45) | 34.17% (32.34–36.08) |

The three selected-ratio CTC seeds have mean test CER **8.15%** with sample standard deviation **0.14 percentage points** (`final_seeds.test_cer_std_sample` in `results/m2/summary.json`; the population figure, 0.117, is recorded alongside it). The trained encoder-decoder is an accuracy reference only and is not a release artifact.

Two runs reached their fixed epoch cap while validation CER was still improving, recorded in `convergence.runs` of the same file. The TrOCR-small reference peaked on its final epoch (12 of 12), so its 11.51% is an upper bound and the CTC advantage over it is an upper bound too — it measures one encoder-decoder under a fixed budget, not encoder-decoder architectures in general. The selected CTC seed also peaked on its final epoch (40 of 40), so the released model's own CER is not a converged value either; seeds 43 and 44 peaked at 38 and 36.

### ONNX and browser parity

| Runtime/artifact | Lines | CER | Delta | Median latency/line |
|---|---:|---:|---:|---:|
| PyTorch CPU checkpoint | 601 | 8.5035% | control | about 57 ms |
| ONNX fp32, Python ORT | 601 | 8.5035% | 0.0000 points | about 49 ms |
| ONNX int8, Python ORT | 601 | 8.5681% | +0.0646 points vs fp32 | about 72 ms |
| ONNX int8, Node ORT | 601 | 8.5681% | identical to Python ORT | about 70 ms |
| ONNX int8, browser WASM | 50 | 5.7937% | +0.0397 points vs Node on the same 50 lines (Node: 5.7540%) | 253 ms |

The browser row is **not comparable to the rows above it**: it covers only the first 50 test
lines, which happen to be an easier subset, so its 5.79% is lower than the 601-line 8.57%
for reasons that have nothing to do with the runtime. The runtime comparison is the delta
against Node on those same 50 lines. The figure that describes this artifact's accuracy
remains **8.5681%** on the full 601-line test set.

Two figures for the same checkpoint are both correct and describe different things. The M2 training result (8.21% for this seed, 8.15% as the three-seed mean) was measured on CUDA; the PyTorch CPU control above scores 8.5035% on the identical checkpoint and inputs. That 0.29-point gap comes from the CUDA and CPU GRU kernels, not from the export — which is why parity is measured against a same-backend CPU control. The number that describes the artifact published here is **8.5681%** (int8, CPU/WASM).

Browser evaluation used headless Chrome 151 on an Apple M1. The first 50 frozen test-manifest lines were evaluated; browser and Node differed on one prediction because browser Canvas and Pillow resizing are not bit-identical. The parity gate is an absolute CER delta no greater than 0.2 percentage points, matching the fp32 parity tolerance. Browser cold model load was 243 ms from a local static server. The verifier observed no HTTP request after readiness and no console/page errors. See `results/m4/browser.json` for exact measurements.

## Limitations and risks

- Accuracy is measured on one modern German handwriting collection; other writers, capture conditions, and document genres may shift performance.
- The held-out split separates pages, not verified writers. If a contributor wrote several pages in the corpus, some test hands may also appear in training, which would make the reported CER optimistic. The corpus ships no writer identifiers, so this cannot be checked.
- Greedy CTC decoding lacks lexical correction and can produce plausible-looking character errors.
- The model does not detect or crop lines. Supplying a full page is unsupported.
- The 195-character alphabet is intentionally fixed from training sources and a Latin-1 repertoire. Unsupported characters cannot be emitted.
- The browser uses Canvas resizing, which is not bit-identical to Pillow preprocessing.
- Handwriting can contain sensitive personal data. Browser inference keeps pixels local after load, but downstream applications remain responsible for storage, access control, and lawful use.
- Do not use this model as the sole basis for high-impact legal, medical, financial, or identity decisions.

## License and attribution

The released code and model artifacts are licensed under Apache-2.0. This permits commercial use. Training sources retain their original licenses; using this model does not relicense those source datasets. The release includes their required attribution and license records in `training-data/SOURCES.md` and `training-data/FONTS.md`.

The two ScaDS.AI Tier-1 datasets are CC BY 4.0, created by Thomas Burghardt and Ahmad Alzin (line- and word-level, doi:10.5281/zenodo.18301532) and by Till Nestler and Thomas Burghardt (full-page, doi:10.5281/zenodo.18283705). German Wikipedia source text is CC BY-SA 4.0; the ShareAlike condition attaches to that text, which is not redistributed in this model release. The bundled Patrick Hand demo font and all synthetic-rendering fonts are SIL OFL 1.1. ONNX Runtime Web is MIT licensed. See the published registers for record URLs, versions, and access dates.

### Pretrained-weight provenance

The visual stem is initialized from torchvision's ImageNet-1k pretrained ResNet-18 checkpoint, so those weights are carried inside `model.onnx`. That checkpoint is distributed by torchvision under BSD-3-Clause, which permits commercial redistribution and is the basis for including it in this Apache-2.0 release. The underlying ImageNet-1k *images* are governed by ImageNet's non-commercial research terms and are not redistributed here. Whether restrictions on training images propagate to weights trained on them is legally unsettled; this release follows the prevailing practice, and torchvision's own BSD-3-Clause distribution of these weights, in treating them as freely reusable. This is a weaker guarantee than the rest of the license register, where every input was selected to be unambiguously permissive, and it is stated here rather than left implicit. Training the stem from scratch (`pretrained=False`) removes the dependency at an unmeasured accuracy cost. Full detail is in `training-data/SOURCES.md`.