parseq-s-rukopys

A PARSeq-S (permuted autoregressive sequence) line recognizer for single-line handwritten Ukrainian document text, fine-tuned on the Rukopys dataset.

The training data included handwritten, printed, annotation and table region crops from Rukopys.

TL;DR

	value
Architecture	PARSeq — ViT-S encoder + 1-layer permutation-LM decoder (ECCV 2022)
Parameters	~24M
Init from	Hukyl/parseq-s-cyrillic-handwritten (mixed pretrain)
Handles	handwritten, printed, annotation
Trained on	nearly all the gold data
Decode	AR greedy + 1 refinement iteration
Representative gold-val CER / WER	0.0935 / 0.2713
Input	a single cropped text line, BGR (numpy) — resized to 48×512
Output	the transcribed string (232-char frozen Cyrillic charset)

Intended use

Recognizing single cropped lines/regions of handwritten Ukrainian documents, downstream of a line/region detector or on already-segmented lines.

A companion ViT-B variant fine-tuned with the same recipe is available at Hukyl/parseq-b-rukopys.

How to use

Note: the checkpoint is a plain torch.save archive, not transformers-compatible. The payload is a dict:

import torch
from huggingface_hub import hf_hub_download

path = hf_hub_download("Hukyl/parseq-s-rukopys", "best.pt")
payload = torch.load(path, map_location="cpu", weights_only=True)
# keys: "model_state" (state_dict), "charset" (str), "config" (dict),
#       "metrics" (dict), "epoch" (int)
print(payload["config"])
# {'img_height': 48, 'img_width': 512, 'patch_size': (4, 8), 'embed_dim': 384,
#  'enc_num_heads': 6, 'enc_mlp_ratio': 4, 'enc_depth': 12, 'dec_num_heads': 12,
#  'dec_mlp_ratio': 4, 'dec_depth': 1, 'max_label_length': 100, 'dropout': 0.1,
#  'decode_ar': True, 'refine_iters': 1, 'drop_path_rate': 0.05}

To run it: construct a PARSeq model (ViT encoder + 1-layer two-stream permutation decoder; see baudm/parseq, Apache-2.0) with the embedded config, build a tokenizer over the embedded charset (token order is [E] + the charset string + [B] + [P]; [E] is id 0), then model.load_state_dict(payload["model_state"]).

The input geometry ships in the checkpoint's config: each crop is resized unconditionally to 48×512 RGB (no aspect preservation; INTER_AREA on downscale — an INTER_LINEAR mismatch aliases on the heavy downscale and degrades accuracy) and normalized (x − 0.5) / 0.5. Output confidence is the mean per-step max-softmax over the decoded sequence.

Architecture

PARSeq, after Bautista & Atienza, "Scene Text Recognition with Permuted Autoregressive Sequence Models" (ECCV 2022, arXiv:2207.06966). Model code is derived from baudm/parseq (Apache-2.0). Geometry and charset are identical to the pretrain stage it is fine-tuned from.

component	value
encoder	ViT-S — 12 layers, dim 384, 6 heads, MLP ratio 4, patch 4×8
decoder	1-layer two-stream permutation-LM, 12 heads, MLP ratio 4
training	K=6 permutations (permuted AR sequence modeling)
decode	autoregressive greedy + 1 refinement iteration
dropout / drop-path	0.1 / 0.05
input	48×512 RGB, unconditional resize, (x−0.5)/0.5
max label length	100
parameters	~24M

Charset

A frozen 232-character Cyrillic charset (sha256 d1b9161e…3ff976). Token IDs follow string order; specials are [E] (id 0) first and [B], [P] last. Text is not NFKD-normalized. The set covers the full Ukrainian Cyrillic block (incl. і ї є ґ and the apostrophe), the Russian-only capitals Э Ё and letters ъ ы ё, digits, punctuation, and a small set of Greek/math symbols. Latin lookalikes that are visually identical to Cyrillic letters (a c e i o y; A B C E H K M O P T X) are deliberately excluded — the target output space is Cyrillic-normalized, so those are folded to their Cyrillic counterparts.

Training curriculum

Two stages, seed 42. The trainer filters samples to the frozen charset and drops labels longer than 100 characters before training.

1. Mixed pretrain — Hukyl/parseq-s-cyrillic-handwritten: general Cyrillic handwriting reader trained from scratch on a real + synthetic mix (~0.75M crops/epoch, synthetic:real ≈ 1.5:1). See the pretrain card for the full mix and recipe.
2. Rukopys gold fine-tune (15 ep, LR 5e-5 peak): warm-started from the stage-1 weights, fine-tuned on the human-labeled Rukopys gold data, with page-level validation split. Note: to incorporate as much of the Rukopys gold split as possible, this checkpoint had a near-zero validation set (~5 pages): 22,293 train / 105 val crops (handwritten 21,381 / annotation 519 / printed 288 / table 142, before charset/length filtering). The published weights are the best-val-CER epoch (11 of 15).

Stage-2 hyperparameters (as launched)

hyperparameter	value
epochs	15
effective batch size	64 (micro-batch 64 × accum 1)
learning rate	5e-5 peak
schedule	OneCycleLR, warmup 5%
optimizer	AdamW, weight_decay 0.05
label smoothing	0.1
drop-path rate	0.05
permutations (K)	6
gradient clipping	20
precision	bf16 AMP
split	by-page, near-full-gold (~5 held-out pages)
seed	42

Online data augmentation

Online augmentation was applied during training (default profile). Only training crops were augmented, and every train crop was augmented (identity_prob = 0.0), at the 48×512 working resolution (downscaled before augmenting).

Each crop was transformed once per epoch by one geometric + one or two photometric operations at random. Each class received a separate augmentation profile that was selected to minimize the distribution shift.

pool	handwritten / annotation	printed
geometric (pick 1)	margin pad 0.02–0.15 / trim 0.01–0.05, rotation ±1–5°, elastic distortion (α25 σ5), baseline warp (amp 2–8 px, freq 0.5–2)	margin pad 0.02–0.15 / trim 0.01–0.05, rotation ±1–3°
photometric (pick 1–2)	paper-colour shift, Gaussian noise σ5–15, JPEG q30–65, contrast 0.7–1.3 / gamma 0.6–1.5, morphological op (k=2)	paper-colour shift, Gaussian noise σ3–10, JPEG q40–70, contrast 0.8–1.2 / gamma 0.8–1.3, morphological op (k=2)

Results

Protocol: AR greedy + 1 refinement iteration, page-level gold validation split (val crops come from pages the model never saw in training), seed 42.

Why the numbers come from a sibling

Since this checkpoint included almost all of the gold split, measuring CER on the tiny val split would result in meaningless metrics. Instead, we report CER on the sibling model trained with the identical recipe (same warm-start, hyperparameters, augmentation, and seed) on a standard ~185-page held-out split. We suspect that this model is strictly equal or better than the reported numbers.

metric	training-time best	standalone predictor eval (n=3,202)
CER	0.0935	0.0954
WER	0.2713	0.2745
exact-match accuracy	—	0.4453

Per class

class	n	CER	WER	accuracy
handwritten	3,067	0.0792	0.2566	0.4522
annotation	73	0.5457	0.6555	0.4110
printed	48	0.4097	0.7783	0.1875
table	14	1.9864	1.4118	0.0000

We also acknowledge that printed/annotation/table n is quite small, so measuring CER against them is quite noisy. table (multi-line pipe-separated serialization) is not a target of this model; its CER above 1 is expected.

Limitations & biases

• Handwritten Ukrainian archival document material only; expect degradation on other scripts, languages, or modern born-digital text.
• The character set and normalization are tuned to the document-OCR metric — outputs are normalized text, not faithful transcription. Latin lookalikes are folded to their Cyrillic counterparts, so the model never emits the Latin forms.
• printed and annotation have small training support and high CER; table and formula are unsupported.
• Headline metrics are from an identical-recipe sibling, not this exact checkpoint (its own holdout is too small to score).
• Single seed — no across-run variance estimate.

Training data & attribution

dataset	source	license	role
Rukopys	UkrainianCatholicUniversity/rukopys	CC BY 4.0	gold fine-tune
UkrHandwritten	Kaggle annyhnatiuk/ukrainian-handwritten-text	CC BY-SA 4.0	pretrain (lineage)
Cyrillic Handwriting Dataset	Kaggle constantinwerner/cyrillic-handwriting-dataset	CC0	pretrain (lineage)
school_notebooks_RU	HF ai-forever/school_notebooks_RU	MIT	pretrain (lineage)
pumb-ai/synthetic-cyrillic-large	HF pumb-ai/synthetic-cyrillic-large	Apache-2.0	pretrain (lineage)
nastyboget/synthetic_cyrillic_large	HF nastyboget/synthetic_cyrillic_large	MIT	pretrain (lineage)

License & lineage

Model weights are released under CC BY-SA 4.0 — the most restrictive of the training inputs now that Rukopys is CC BY 4.0 (UkrHandwritten is share-alike, and shares this requirement down to the fine-tuned weights). The PARSeq model code is Apache-2.0 (baudm/parseq). The stage-1 pretrain checkpoint, which did not train on Rukopys, is released under CC BY-SA 4.0.