parseq-b-rukopys

A PARSeq-B (permuted autoregressive sequence) line recognizer for single-line handwritten Ukrainian document text, fine-tuned on the Rukopys dataset.

The training data included handwritten, printed, annotation and table region crops from Rukopys.

TL;DR

	value
Architecture	PARSeq — ViT-B encoder + 1-layer permutation-LM decoder (ECCV 2022)
Parameters	~96M
Init from	Hukyl/parseq-b-cyrillic-handwritten (mixed pretrain)
Handles	handwritten, printed, annotation
Decode	AR greedy + 1 refinement iteration
Gold-val CER / WER	0.0953 / 0.2796
Input	a single cropped text line, BGR (numpy) — resized to 48×512
Output	the transcribed string (232-char frozen Cyrillic charset)

Intended use

Recognizing single cropped lines/regions of handwritten Ukrainian documents, downstream of a line/region detector or on already-segmented lines.

A companion ViT-S variant (~24M params) fine-tuned with the same recipe is available at Hukyl/parseq-s-rukopys. The two land in a statistical tie on the same held-out split: the ViT-S is marginally ahead on the dominant handwritten class, this model a little ahead on the low-support annotation/printed classes.

How to use

Note: the checkpoint is a plain torch.save archive, not transformers-compatible. The payload is a dict:

import torch
from huggingface_hub import hf_hub_download

path = hf_hub_download("Hukyl/parseq-b-rukopys", "best.pt")
payload = torch.load(path, map_location="cpu", weights_only=True)
# keys: "model_state" (state_dict), "charset" (str), "config" (dict),
#       "metrics" (dict), "epoch" (int)
print(payload["config"])
# {'img_height': 48, 'img_width': 512, 'patch_size': (4, 8), 'embed_dim': 768,
#  'enc_num_heads': 12, 'enc_mlp_ratio': 4, 'enc_depth': 12, 'dec_num_heads': 12,
#  'dec_mlp_ratio': 4, 'dec_depth': 1, 'max_label_length': 100, 'dropout': 0.1,
#  'decode_ar': True, 'refine_iters': 1, 'drop_path_rate': 0.05}

To run it: construct a PARSeq model (ViT encoder + 1-layer two-stream permutation decoder; see baudm/parseq, Apache-2.0) with the embedded config, build a tokenizer over the embedded charset (token order is [E] + the charset string + [B] + [P]; [E] is id 0), then model.load_state_dict(payload["model_state"]).

The input geometry ships in the checkpoint's config: each crop is resized unconditionally to 48×512 RGB (no aspect preservation; INTER_AREA on downscale — an INTER_LINEAR mismatch aliases on the heavy downscale and degrades accuracy) and normalized (x − 0.5) / 0.5. Output confidence is the mean per-step max-softmax over the decoded sequence.

Architecture

PARSeq, after Bautista & Atienza, "Scene Text Recognition with Permuted Autoregressive Sequence Models" (ECCV 2022, arXiv:2207.06966). Model code is derived from baudm/parseq (Apache-2.0). Geometry, decoder, and charset are identical to the ViT-S variant; only the encoder width and head count are scaled up.

component	value
encoder	ViT-B — 12 layers, dim 768, 12 heads, MLP ratio 4, patch 4×8
decoder	1-layer two-stream permutation-LM, 12 heads, MLP ratio 4
training	K=6 permutations (permuted AR sequence modeling)
decode	autoregressive greedy + 1 refinement iteration
dropout / drop-path	0.1 / 0.05
input	48×512 RGB, unconditional resize, (x−0.5)/0.5
max label length	100
parameters	~96M

Charset

A frozen 232-character Cyrillic charset (sha256 d1b9161e…3ff976). Token IDs follow string order; specials are [E] (id 0) first and [B], [P] last. Text is not NFKD-normalized. The set covers the full Ukrainian Cyrillic block (incl. і ї є ґ and the apostrophe), the Russian-only capitals Э Ё and letters ъ ы ё, digits, punctuation, and a small set of Greek/math symbols. Latin lookalikes that are visually identical to Cyrillic letters (a c e i o y; A B C E H K M O P T X) are deliberately excluded — the target output space is Cyrillic-normalized, so those are folded to their Cyrillic counterparts.

Training curriculum

Two stages, seed 42. The trainer filters samples to the frozen charset and drops labels longer than 100 characters before training.

1. Mixed pretrain — Hukyl/parseq-b-cyrillic-handwritten: general Cyrillic handwriting reader trained from scratch on a real + synthetic mix (~0.65M crops/epoch, synthetic:real ≈ 1.15:1). See the pretrain card for the full mix and recipe.
2. Rukopys gold fine-tune (15 ep, LR 5e-5 peak): warm-started from the stage-1 weights, fine-tuned on the human-labeled Rukopys gold data, with page-level validation split — 19,196 train / 3,202 val crops (handwritten 18,414 / annotation 450 / printed 240 / table 122 in train, before charset/length filtering). The published weights are the best-val-CER epoch (13 of 15).

Stage-2 hyperparameters (as launched)

hyperparameter	value
epochs	15
effective batch size	64 (micro-batch 64 × accum 1)
learning rate	5e-5 peak
schedule	OneCycleLR, warmup 5%
optimizer	AdamW, weight_decay 0.05
label smoothing	0.1
drop-path rate	0.05
permutations (K)	6
gradient clipping	20
precision	bf16 AMP
split	by-page (~185 held-out pages)
seed	42

Online data augmentation

Online augmentation was applied during training (default profile). Only training crops were augmented, and every train crop was augmented (identity_prob = 0.0), at the 48×512 working resolution (downscaled before augmenting).

Each crop was transformed once per epoch by one geometric + one or two photometric operations at random. Each class received a separate augmentation profile that was selected to minimize the distribution shift.

pool	handwritten / annotation	printed
geometric (pick 1)	margin pad 0.02–0.15 / trim 0.01–0.05, rotation ±1–5°, elastic distortion (α25 σ5), baseline warp (amp 2–8 px, freq 0.5–2)	margin pad 0.02–0.15 / trim 0.01–0.05, rotation ±1–3°
photometric (pick 1–2)	paper-colour shift, Gaussian noise σ5–15, JPEG q30–65, contrast 0.7–1.3 / gamma 0.6–1.5, morphological op (k=2)	paper-colour shift, Gaussian noise σ3–10, JPEG q40–70, contrast 0.8–1.2 / gamma 0.8–1.3, morphological op (k=2)

Results

Protocol: AR greedy + 1 refinement iteration, page-level gold validation split (val crops come from pages the model never saw in training), seed 42.

Overall

metric	training-time best	standalone predictor eval (n=3,202)
CER	0.0953	0.0958
WER	0.2796	0.2802
exact-match accuracy	—	0.4316

Per class

class	n	CER	WER	accuracy
handwritten	3,067	0.0849	0.2664	0.4356
annotation	73	0.4222	0.5882	0.4932
printed	48	0.3279	0.6527	0.2083
table	14	1.1667	1.4706	0.0000

We also acknowledge that printed/annotation/table n is quite small, so measuring CER against them is quite noisy. table (multi-line pipe-separated serialization) is not a target of this model; its CER above 1 is expected.

Limitations & biases

• Handwritten Ukrainian archival document material only; expect degradation on other scripts, languages, or modern born-digital text.
• The character set and normalization are tuned to the document-OCR metric — outputs are normalized text, not faithful transcription. Latin lookalikes are folded to their Cyrillic counterparts, so the model never emits the Latin forms.
• printed and annotation have small training support and high CER; table and formula are unsupported.
• Single seed and validation split — no across-run variance estimate.

Training data & attribution

dataset	source	license	role
Rukopys	UkrainianCatholicUniversity/rukopys	CC BY 4.0	gold fine-tune
UkrHandwritten	Kaggle annyhnatiuk/ukrainian-handwritten-text	CC BY-SA 4.0	pretrain (lineage)
Cyrillic Handwriting Dataset	Kaggle constantinwerner/cyrillic-handwriting-dataset	CC0	pretrain (lineage)
school_notebooks_RU	HF ai-forever/school_notebooks_RU	MIT	pretrain (lineage)
pumb-ai/synthetic-cyrillic-large	HF pumb-ai/synthetic-cyrillic-large	Apache-2.0	pretrain (lineage)
nastyboget/synthetic_cyrillic_large	HF nastyboget/synthetic_cyrillic_large	MIT	pretrain (lineage)

License & lineage

Model weights are released under CC BY-SA 4.0 — the most restrictive of the training inputs now that Rukopys is CC BY 4.0 (UkrHandwritten is share-alike, and shares this requirement down to the fine-tuned weights). The PARSeq model code is Apache-2.0 (baudm/parseq). The stage-1 pretrain checkpoint, which did not train on Rukopys, is released under CC BY-SA 4.0.