Yigdzin 1 weights: elie_v8_coarse_grow26_ep2 (no-leakage, benchmark held out); update card + eval numbers

README.md

@@ -17,23 +17,27 @@ metrics:
 
 # Tibetan OCR — Yigdzin 1
 
-<!-- DRAFT model card for BDRC/tibetan-ocr, release tag v1 ("Yigdzin 1").
-     Numbers are PRELIMINARY (v1). Confirm display name sign-off before publishing. -->
+<!-- Model card for BDRC/tibetan-ocr ("Yigdzin 1").
+     Weights: checkpoint elie_v8_coarse_grow26_ep2 (bec_mixed_elie_v8 mix, evaluation
+     benchmark pages held out of training via images_exclude_from_train.csv). -->
 
 **A vision-first OCR model for Tibetan pecha** — modern publications, woodblock
 prints, and manuscripts (uchen and u-med), including pecha layout and orthographic
 shorthands. Built by the [Buddhist Digital Resource Center (BDRC)](https://bdrc.io)
-and trained on ~220k curated real + synthetic pages.
+and trained on the `bec_mixed_elie_v8` mix of curated real + synthetic pages, with
+the **evaluation-benchmark pages held out of training**.
 
-- **Repository:** `BDRC/tibetan-ocr` · release **`v1`** (display name **Yigdzin 1**)
+- **Repository:** `BDRC/tibetan-ocr` (display name **Yigdzin 1**)
 - **Base model:** [`PaddlePaddle/PaddleOCR-VL-1.6`](https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6) (Apache-2.0)
 - **Live demo:** [ocr.bdrc.io](https://ocr.bdrc.io) · **Desktop app:** [buda-base/tibetan-ocr-app](https://github.com/buda-base/tibetan-ocr-app)
 
 ## TL;DR
 
-On a 1,070-page hand-transcribed benchmark, typical-page character error rate is
-**~1.8% (median CER 0.0179)**; corpus micro-CER is **0.0756** (greedy). It is a
-*specialized* OCR model, not a general chat VLM.
+On a 1,070-page hand-transcribed benchmark (production serving: vLLM + sequential
+regime + DRY guard + temperature-retry), **median page CER is ~1.4%** and clean-page
+(non-catastrophic) mean CER is **~4.0%**, with **0 hard repetition loops**. The
+benchmark pages are held out of training. It is a *specialized* OCR model, not a
+general chat VLM.
 
 ## Model details
 
@@ -43,6 +47,8 @@ On a 1,070-page hand-transcribed benchmark, typical-page character error rate is
 | Parameters | ~810M (~413M vision · ~360M 26-layer decoder · ~4M embeddings/head) |
 | Vision share | ~51–59% of parameters ("big eyes, small mouth" — vision-first) |
 | Tokenizer | Tibetan **unicode-stack** tokenizer, vocab **3,560** (pruned + BoCorpus-warmed) |
+| Decoder growth | grow26: SOLAR-style depth-upscale 18L→26L, then annealed |
+| Position regime | **sequential** image-token M-RoPE (1-D); `max_pixels` 1280 budget |
 | Languages | Tibetan (`bo`); Tibetan script |
 | License | Apache-2.0 (derivative of PaddleOCR-VL-1.6; upstream `NOTICE` retained) |
 | Funder | Khyentse Foundation ("The BDRC Etext Corpus") |
@@ -136,11 +142,11 @@ export OCR_VLLM_IMAGE_TOKEN_POSITIONS=sequential   # 'grid' / unset = no-op
 Without it, structured (book/list) pages skip or merge lines and CER regresses badly.
 
 **2. DRY anti-loop guard + temperature retry (recommended).** Greedy decoding runs
-away into repetition on ~1.5% of pages. The surgical fix is the **DRY** ("Don't Repeat
-Yourself") penalty: it ties an n-gram ban on corpus CER, zeroes hard loops, and — unlike
-`repetition_penalty` / `no_repeat_ngram_size` — barely touches clean pages or legitimate
-repetitive scripture (mantras, litanies). It ships here as the self-contained
-`dry_logits_processor.py` (torch-only):
+away into repetition on a small fraction of pages. The surgical fix is the **DRY**
+("Don't Repeat Yourself") penalty: it ties an n-gram ban on corpus CER, zeroes hard
+loops, and — unlike `repetition_penalty` / `no_repeat_ngram_size` — barely touches
+clean pages or legitimate repetitive scripture (mantras, litanies). It ships here as
+the self-contained `dry_logits_processor.py` (torch-only):
 
 ```python
 from vllm import LLM, SamplingParams
@@ -152,15 +158,15 @@ params = SamplingParams(temperature=0, max_tokens=4096, extra_args={
 })
 ```
 
-For the ~2% of pages that still loop under greedy+DRY, **re-decode at temperature and
-keep the cleanest sample.** The strong signal is *how often DRY fired*: the measured knee
-is **≥ 100 fires** — re-decode only those pages at `temperature=0.4`, `n=3` (DRY still on)
-and pick the sample with the lowest leftover repetition. That is ~2% of pages / ~20 s
-extra on the 1,070-page benchmark and clears the residual hard loops.
-`dry_logits_processor.py` writes per-request fire counts (pass `dry_stats_id` /
-`dry_stats_path` in `extra_args`, read them back with `load_dry_stats_dir`);
-`deploy/fast_inference/bench.py` in the training repo is the reference implementation of
-the full greedy → fire-count gate → temperature-retry loop.
+For the pages that still loop under greedy+DRY, **re-decode at temperature and keep the
+cleanest sample.** The strong signal is *how often DRY fired*: the measured knee is
+**≥ 100 fires** — re-decode only those pages at `temperature=0.4`, `n=3` (DRY still on)
+and pick the sample with the lowest leftover repetition. On this benchmark that is ~2%
+of pages and clears the residual hard loops. `dry_logits_processor.py` writes
+per-request fire counts (pass `dry_stats_id` / `dry_stats_path` in `extra_args`, read
+them back with `load_dry_stats_dir`); `deploy/fast_inference/bench.py` in the training
+repo is the reference implementation of the full greedy → fire-count gate →
+temperature-retry loop.
 
 Do **not** reach for aggressive `repetition_penalty` / `no_repeat_ngram_size`: they
 "correct away" legitimate repeated scripture. DRY with the config above is the tested,
@@ -168,47 +174,49 @@ surgical choice.
 
 ## Training data
 
-~220k-page curated mix (`bec_mixed_elie_v6`), assembled from BDRC alignment
-collections + BoCorpus-rendered synthetic pages, with an easy→hard curriculum,
-u-med over-weighting, and repetition-aware filtering. Released component datasets:
+Curated mix (`bec_mixed_elie_v8`), assembled from BDRC alignment collections +
+BoCorpus-rendered synthetic pages, with an easy→hard curriculum, u-med over-weighting,
+and repetition-aware filtering. **The evaluation-benchmark pages are held out of
+training** (`images_exclude_from_train.csv`). Released component datasets:
 
-- Real transcriptions: [ALL-BDRC (ACIP Sungbum)](https://huggingface.co/datasets/BDRC/all-bdrc),
+- Real transcriptions: [ALL-BDRC (ACIP Sungbum)](https://huggingface.co/datasets/BDRC/ALL-BDRC-alignments),
   [PalriParkhang](https://huggingface.co/datasets/BDRC/palri-parkhang),
   [Berkeley](https://huggingface.co/datasets/BDRC/berkeley),
   [MonlamAI-transcriptions](https://huggingface.co/datasets/BDRC/monlamai-transcriptions),
   [MonlamAI-handwritten](https://huggingface.co/datasets/BDRC/monlamai-handwritten),
-  [Stok](https://huggingface.co/datasets/BDRC/stok), [TibSchol](https://huggingface.co/datasets/BDRC/tibschol) *(gated)*.
+  [Stok](https://huggingface.co/datasets/openpecha/stok), [TibSchol](https://huggingface.co/datasets/BDRC/tibschol) *(gated)*.
 - Synthetic: [Tibetan OCR synthetic v5](https://huggingface.co/datasets/BDRC/tibetan-ocr-synthetic).
 
-<!-- Fill final dataset repo links once created; see the release plan. -->
+## Evaluation
 
-## Evaluation *(preliminary, v1)*
+Evaluated on the BDRC hand-transcribed benchmark `20260315` (1,070 pages) in the
+**production configuration** (vLLM + `vllm_paddleocr_seqpos` sequential regime + DRY
+guard `mult=0.8 base=1.75 allowed_length=12` + temperature-retry `temp=0.4 n=3` on
+pages with ≥100 DRY fires). Metrics from the benchmark's `compute_cer.py` (botok
+normalization, whitespace stripping, tsheg folding, placeholder removal); pages with
+CER > 50% are counted as *catastrophic* and reported separately.
 
-Evaluated on the BDRC hand-transcribed benchmark `20260315` (1,070 pages), greedy
-decoding, `max_new_tokens=4096`, prompt policy `none`.
+| Metric (1,070 pages) | **Yigdzin 1** |
+|---|---:|
+| Median page CER | **0.0142** |
+| Mean page CER (all) | 0.0628 |
+| Catastrophic (>50%) share | 3.3% (35) |
+| Clean mean CER (non-catastrophic) | **0.0404** |
+| Clean mean SER | **0.0764** |
+| Hard repetition loops | **0** |
 
-| Setting | micro-CER | median CER | hard-loops |
-|---|---:|---:|---:|
-| **Yigdzin 1** (greedy) | **0.0756** | **0.0179** | **2** |
-| PaddleOCR-VL 18-layer baseline (ep1) | 0.1198 | 0.0211 | 16 |
-
-Production serving adds the **DRY anti-loop guard + temperature retry** described in
-[How to use](#how-to-use); that production configuration is reported on the leaderboard
-as **Yigdzin-v1**.
-
-CER = Levenshtein(hyp, ref)/len(ref) after botok normalization, whitespace
-stripping, tsheg folding, and placeholder removal. Full method + the multi-system
-leaderboard: **[BDRC Tibetan OCR benchmark](https://huggingface.co/datasets/BDRC/tibetan-ocr-benchmark)** and the **[leaderboard](https://huggingface.co/spaces/BDRC/tibetan-ocr-leaderboard)**.
+Full method + the multi-system leaderboard:
+**[BDRC Tibetan OCR benchmark](https://huggingface.co/datasets/BDRC/tibetan-ocr-benchmark)**
+and the **[leaderboard](https://huggingface.co/spaces/BDRC/tibetan-ocr-leaderboard)**.
 
 ## Limitations & recommendations
 
-- **Residual repetition loops** on a small fraction of pages (~1.5% greedy). Production
-  uses the **DRY** guard + temperature retry (see How to use), which clears them while
-  leaving legitimate repetitive scripture (mantras) intact; keep any anti-loop mitigation
-  this gentle.
-- **Line-break joins** on some dense (6-line) woodblock pecha (~1.9% of such pages).
+- **Residual repetition loops** on a small fraction of pages. Production uses the
+  **DRY** guard + temperature retry (see How to use), which clears them (0 hard loops
+  on the benchmark) while leaving legitimate repetitive scripture (mantras) intact;
+  keep any anti-loop mitigation this gentle.
+- **Line-break joins** on some dense (6-line) woodblock pecha.
 - Struggles on illustrations, non-Tibetan text, and illegible/damaged pages.
-- Metrics are **preliminary (v1)** and may change in later generations.
 
 ## Citation
 

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c54eb065005184a30fd17c25e1393e72a457b500045fe510a79221df0af9a34d
+oid sha256:2a698bc88308b27656c808c87c33778feb3873c54ebf4bf334f9e17185532514
 size 1628770560

processor_config.json

@@ -24,10 +24,10 @@
     "resample": 3,
     "rescale_factor": 0.00392156862745098,
     "size": {
-      "shortest_edge": 1024,
-      "longest_edge": 1003520
+      "longest_edge": 1003520,
+      "shortest_edge": 1024
     },
     "temporal_patch_size": 1
   },
   "processor_class": "PaddleOCRVLProcessor"
-}
+}

tokenizer_config.json

@@ -1040,14 +1040,17 @@
     "<|image_pad|>",
     "<|video_pad|>"
   ],
+  "image_token": "<|IMAGE_PLACEHOLDER|>",
   "is_local": true,
   "local_files_only": false,
   "mask_token": "<mask:1>",
   "model_max_length": 1000000000000000019884624838656,
+  "model_specific_special_tokens": {
+    "image_token": "<|IMAGE_PLACEHOLDER|>"
+  },
   "pad_token": "<unk>",
   "processor_class": "PaddleOCRVLProcessor",
   "sep_token": "<|end_of_sentence|>",
   "tokenizer_class": "TokenizersBackend",
-  "unk_token": "<unk>",
-  "image_token": "<|IMAGE_PLACEHOLDER|>"
-}
+  "unk_token": "<unk>"
+}