results/summary.md

# Benchmark summary

Passages: 75 · macro means over passages (micro CER in footnote)

| Model | CER ↓ | WER ↓ | Rel. CER reduction ↑ | Over-correction ↓ | Fix rate ↑ | Median s/passage | tok/s |
|---|---|---|---|---|---|---|---|
| OCR input (uncorrected) | 0.066 | 0.215 | — | — | — | — | — |
| DiffusionGemma 26B-A4B-it | 0.035 | 0.073 | 49.5% | 1.5% | 86.0% | 1.69 | 119.9 |
| Gemma-4-E4B-it | 0.042 | 0.107 | 45.9% | 0.4% | 61.5% | 15.33 | 12.9 |
| Gemma-4-26B-A4B-it (MoE) | 0.027 | 0.061 | 62.4% | 0.9% | 87.5% | 16.31 | 12.0 |

Micro (corpus-level) CER — input: 0.062, DiffusionGemma 26B-A4B-it: 0.032, Gemma-4-E4B-it: 0.038, Gemma-4-26B-A4B-it (MoE): 0.025.
Mean denoising steps, DiffusionGemma 26B-A4B-it: 9.5 (max 48).

## Config

```json
{
  "date": "2026-06-11",
  "dataset": "bln600",
  "n": 75,
  "seed": 42,
  "max_passage_tokens": 220,
  "prompt": "Correct the OCR errors in the following text from a 19th-century English newspaper.\nFix only recognition errors (wrong, missing, or extra characters). Do not modernise spelling, do not rephrase, and do not add or remove content. Preserve the original punctuation unless it is clearly an OCR error.\nOutput only the corrected text, with no commentary or preamble.\n\nOCR text:\n{ocr}",
  "transformers": "5.11.0",
  "torch": "2.9.1+cu128",
  "gpu": "NVIDIA A100-SXM4-80GB",
  "generation": {
    "diffusiongemma": "generation_config defaults (entropy sampler), max_new_tokens=256",
    "diffusiongemma_canvas": "as diffusiongemma, but first canvas seeded with the OCR text via decoder_input_ids (random tail padding, seed 0)",
    "gemma4": "do_sample=False (greedy), max_new_tokens=256",
    "gemma4_moe": "do_sample=False (greedy), max_new_tokens=256"
  }
}
```