kaan84/sinhala-print-trocr

TrOCR Sinhala Printed Lines

Phase 1 in progress. Exploratory pilot weights are released; pre-registered confirmatory runs are under way. No real-document evaluation results exist yet.

This project investigates whether fine-tuning Ransaka/TrOCR-Sinhala on synthetic Sinhala text-line images improves printed, line-level Sinhala OCR performance on real documents.

Project status

Workstream	Status
Research scope	Locked in PROTOCOL.md v1.1
Dataset license	Confirmed MIT
Intermediate checkpoint license	Confirmed MIT (author, 2026-07-30)
Pilot run	Complete — exploratory, reported separately
Confirmatory training	Seed 42 complete; seeds 123 and 2026 pending
Real-document evaluation	Not started
Model weights	Pilot published; confirmatory weights pending

The complete model and training-data chain is MIT-licensed: microsoft/trocr-base-printed → Ransaka/TrOCR-Sinhala → Ransaka/sinhala_synthetic_ocr-large → this project's derived weights. Ransaka confirmed the intermediate checkpoint's MIT license and declared it in the repository metadata. The permanent confirmation record is linked in ATTRIBUTION_AND_LICENSING.md.

Exploratory pilot weights: kaan84/sinhala-print-trocr-pilot. Pilot results are never pooled with confirmatory results; see PROTOCOL.md Section 2.

Research question

Does synthetic-data fine-tuning of TrOCR improve printed, line-level Sinhala OCR on real documents?

Scope

In scope

• Printed Sinhala text
• Line-level text recognition
• Character Error Rate (CER)
• Word Error Rate (WER)
• Comparisons with the unmodified checkpoint and Tesseract 5.5
• Optional Surya comparison when its applicable terms are confirmed
• Period-stratified analysis where document-year metadata is available

Out of scope

• Handwriting
• Full-page layout detection or reading order
• Production or high-stakes deployment
• Broad Sinhala-English code mixing
• Claims of parity with page-level commercial or large vision-language systems

Data

Training and validation

Ransaka/sinhala_synthetic_ocr-large

• 6,969 synthetic Sinhala image-text pairs
• One published train split
• Five reported font families
• Dataset DOI: 10.57967/hf/1809
• Repository LICENSE: MIT
• Fixed project split: 90% train and 10% validation using seed 42
• Rows with null or empty reference text are removed before splitting (one row removed: 6,969 → 6,968)

Immutable dataset revisions and split-index checksums are recorded in the per-run experiment manifest.

Real-document evaluation

avishadilhara/sinhala-ocr-lk-acts-1010

• 1,010 page images from Sri Lankan legislative acts
• Published splits: train 707, test 202, evaluation 101
• License: CC BY 4.0
• Paper: arXiv:2606.29378

This dataset is listed in repository metadata for discoverability but is used for evaluation only and never for training.

TrOCR is a line-recognition model. The page dataset must therefore be converted into a documented, deterministic line-level benchmark before direct evaluation. Page-level benchmark figures are contextual references and must not be presented as directly comparable line-level results.

Pre-registered experiment

The hypotheses, fixed splits, seeds, metrics, normalization, hyperparameters, decision rules, and reporting requirements are defined in PROTOCOL.md. Material deviations must be documented before results are interpreted; the deviation log is Section 17.

Planned baselines

System	Role	Primary comparison
Ransaka/TrOCR-Sinhala	Starting checkpoint	Measures improvement from additional fine-tuning
Tesseract 5.5	Permissive classical baseline	CER and WER on the same line images
Surya OCR	Optional research baseline	CER and WER on the same evaluation unit

Published page-level results are contextual targets only:

System	CER	WER	Evaluation unit
Tesseract v5	10.69%	Not reported	Real pages
Surya OCR	8.84%	26.64%	Real pages
Google Document AI	2.06%	Not reported	Real pages
Fine-tuned LightOnOCR-2-1B	1.05%	5.63%	Real pages

These figures come from a June 2026 preprint and are not direct line-level acceptance thresholds.

Pre-registered training configuration

Parameter	Pre-registered value
Optimizer	AdamW
Learning rate	4e-5
Epochs	10
Effective batch size	8
Precision	FP16 when supported
Seeds	42, 123, 2026
Evaluation beams	4
Runs	3 independent runs

The machine-readable configuration is in configs/phase0.yaml.

Evaluation and reporting

CER is the primary metric and WER is secondary. Report:

• Results for every seed, plus mean and standard deviation
• Exact dataset revisions and split-index checksum
• Raw and normalized metrics
• 95% paired bootstrap confidence intervals for model differences
• Error examples grouped by degradation type
• Results by document period where supported
• Runtime and hardware details

Synthetic validation metrics decode predictions and references through the model tokenizer, so its spacing and ZWJ/ZWNJ losses cancel on both sides. Real-benchmark metrics are computed against raw ground-truth text, because external baselines do not pass through this tokenizer. See PROTOCOL.md Section 12.1.

Final results belong in results/metrics.json and the table below. Do not add zero-value placeholders to Hugging Face model-index metadata because they can be mistaken for real results.

System	Dataset/slice	CER mean ± SD	WER mean ± SD	Status
Starting checkpoint	Real Sinhala line benchmark	TBD	TBD	Not run
Fine-tuned model	Real Sinhala line benchmark	TBD	TBD	Not run
Tesseract 5.5	Real Sinhala line benchmark	TBD	TBD	Not run
Surya OCR	Real Sinhala line benchmark	TBD	TBD	Optional

Synthetic validation results to date (not a performance claim, and not comparable to the real-page figures above):

Run	Type	Seed	CER	WER	Exact match
Pilot	Exploratory	42	16.98%	32.11%	3.87%
Confirmatory	Pre-registered	42	14.39%	28.18%	6.89%
Confirmatory	Pre-registered	123	Pending	Pending	Pending
Confirmatory	Pre-registered	2026	Pending	Pending	Pending

The pilot used a different learning rate, epoch count, and split; its figures are a synthetic test split while the confirmatory figures are a synthetic validation split. The two rows are not comparable and must not be pooled.

Intended use

The eventual model is intended for research and prototyping involving printed Sinhala text lines, including OCR experiments, document digitization research, and accessibility prototypes. Human review is required for consequential use.

Limitations

• Synthetic training data may not represent real scanning noise, skew, bleed-through, compression, damaged print, or historical typography.
• Training fonts cover only a limited part of Sinhala typography.
• The inherited tokenizer does not preserve Zero Width Joiner and Zero Width Non-Joiner characters, which affects roughly 6.6% of training texts and places a ceiling on Sinhala conjunct output. See PROTOCOL.md Section 2.2.
• Accuracy may degrade on older documents, handwriting, mixed-language text, unusual symbols, curved lines, and full pages.
• Line segmentation quality can dominate end-to-end page performance.
• Results from legislative acts may not generalize to receipts, books, forms, signs, or contemporary digital screenshots.

Bias and responsible use

The training domain may overrepresent clean, computer-generated Sinhala and underrepresent historical fonts and real capture conditions. Evaluate new domains before use. Do not use unreviewed OCR output as authoritative evidence in legal, medical, financial, identity, or public-administration decisions.

Public legislative documents can contain names. Preserve source attribution, minimize unnecessary retention, and review extracted text before redistribution.

Inference

Confirmatory weights are not published in this repository yet. The pilot weights load as follows:

from PIL import Image
from transformers import TrOCRProcessor, VisionEncoderDecoderModel

repo_id = "kaan84/sinhala-print-trocr-pilot"
processor = TrOCRProcessor.from_pretrained(repo_id)
model = VisionEncoderDecoderModel.from_pretrained(repo_id)

image = Image.open("sinhala-line.png").convert("RGB")
pixel_values = processor(images=image, return_tensors="pt").pixel_values
generated_ids = model.generate(pixel_values, num_beams=4)
text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(text)

Input must be a cropped single line of printed Sinhala text. Auto-generated AutoModel or pipeline("image-to-text") snippets do not load this architecture correctly.

Repository structure

.
├── README.md
├── PROTOCOL.md
├── RISK_REGISTER.md
├── ATTRIBUTION_AND_LICENSING.md
├── REPRODUCIBILITY_CHECKLIST.md
├── requirements.txt
├── configs/
│   └── phase0.yaml
├── scripts/
│   └── README.md
├── assets/
│   └── README.md
├── runs/
│   └── README.md
└── results/
    ├── README.md
    └── metrics.template.json

Release gates

1. Obtain and archive explicit licensing clarification for Ransaka/TrOCR-Sinhala.
2. Record immutable dataset and model revisions.
3. Commit split indices and their checksum.
4. Complete three seeded runs.
5. Run every baseline on identical evaluation units.
6. Populate real metrics and add model-index metadata.
7. Complete the reproducibility checklist.
8. Publish model weights only after the licensing gate passes (pilot weights released under the confirmed MIT chain).

Gates 2 and 3 were not satisfied before the seed 42 confirmatory run; see PROTOCOL.md deviations D-2 and D-3.

Citation

If this research is published, add its final citation here. Upstream citations and licensing notes are maintained in ATTRIBUTION_AND_LICENSING.md.