Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,243 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: rfdetr
|
| 4 |
+
pipeline_tag: object-detection
|
| 5 |
+
language:
|
| 6 |
+
- bo
|
| 7 |
+
tags:
|
| 8 |
+
- tibetan
|
| 9 |
+
- document-layout-analysis
|
| 10 |
+
- rf-detr
|
| 11 |
+
- rfdetr
|
| 12 |
+
- object-detection
|
| 13 |
+
- bounding-box
|
| 14 |
+
- BDRC
|
| 15 |
+
datasets:
|
| 16 |
+
- BDRC/TDLA-Training-Dataset-v2
|
| 17 |
+
metrics:
|
| 18 |
+
- name: canonical mean AP50 (test)
|
| 19 |
+
type: mAP
|
| 20 |
+
value: 0.960
|
| 21 |
+
model-index:
|
| 22 |
+
- name: Tibetan-Modern-Book-Layout-Detection-RFDETR
|
| 23 |
+
results:
|
| 24 |
+
- task:
|
| 25 |
+
type: object-detection
|
| 26 |
+
dataset:
|
| 27 |
+
name: TDLA-Training-Dataset-v2 (test)
|
| 28 |
+
type: BDRC/TDLA-Training-Dataset-v2
|
| 29 |
+
metrics:
|
| 30 |
+
- type: mAP
|
| 31 |
+
name: mAP@0.5 (native, test)
|
| 32 |
+
value: 0.996
|
| 33 |
+
- type: mAP
|
| 34 |
+
name: mAP@0.5:0.95 (native, test)
|
| 35 |
+
value: 0.813
|
| 36 |
+
---
|
| 37 |
+
|
| 38 |
+
# Tibetan Modern Book Layout Detection (RF-DETR-L)
|
| 39 |
+
|
| 40 |
+
An **RF-DETR-L** ([Roboflow](https://github.com/roboflow/rf-detr), DINOv2 backbone)
|
| 41 |
+
object detector that locates the four structural regions of a **modern Tibetan
|
| 42 |
+
book** page β **header**, **text-area**, **footnote**, **footer** β as a
|
| 43 |
+
preprocessing step for OCR and etext production.
|
| 44 |
+
|
| 45 |
+
- **Training code, recipes & write-up:** [buda-base/tibetan-book-layout-analysis](https://github.com/buda-base/tibetan-book-layout-analysis)
|
| 46 |
+
- **Dataset:** [BDRC/TDLA-Training-Dataset-v2](https://huggingface.co/datasets/BDRC/TDLA-Training-Dataset-v2) (gated, fair-use)
|
| 47 |
+
|
| 48 |
+
> This is one of several architectures BDRC fine-tuned on the same labels and
|
| 49 |
+
> recipe to test how much the choice of architecture matters (see the
|
| 50 |
+
> [blog post](https://github.com/buda-base/tibetan-book-layout-analysis/blob/main/BLOGPOST.md)).
|
| 51 |
+
> The primary, production release is the RT-DETR-l fine-tune at
|
| 52 |
+
> [BDRC/Tibetan-Modern-Book-Layout-Detection](https://huggingface.co/BDRC/Tibetan-Modern-Book-Layout-Detection).
|
| 53 |
+
> This RF-DETR-L checkpoint matches it almost exactly on every metric, and is
|
| 54 |
+
> published as a **permissively-licensed (Apache-2.0) alternative** for anyone
|
| 55 |
+
> who can't use the RT-DETR-l release's AGPL-derived training stack.
|
| 56 |
+
|
| 57 |
+
## Model description
|
| 58 |
+
|
| 59 |
+
This is an RF-DETR-L fine-tuned with the **`tam2col`** labelling scheme: text-area
|
| 60 |
+
boxes merged into one envelope per page, except on genuine two-column pages,
|
| 61 |
+
where it keeps one box per column. Header and footer are kept as separate
|
| 62 |
+
classes (they can be combined losslessly downstream).
|
| 63 |
+
|
| 64 |
+
| Property | Value |
|
| 65 |
+
| --- | --- |
|
| 66 |
+
| Architecture | RF-DETR-L (via [`rfdetr`](https://github.com/roboflow/rf-detr), DINOv2-small windowed backbone) |
|
| 67 |
+
| Task | Object detection |
|
| 68 |
+
| Base checkpoint | `rf-detr-large-2026.pth` (Roboflow, Apache-2.0) |
|
| 69 |
+
| Resolution | 1008 Γ 1008 |
|
| 70 |
+
| Number of classes | 4 (+ background) |
|
| 71 |
+
| Framework | `rfdetr` (`RFDETRLarge`) |
|
| 72 |
+
| Weights file | `rfdetr_tibetan_book_layout.pth` |
|
| 73 |
+
|
| 74 |
+
## Classes
|
| 75 |
+
|
| 76 |
+
Note: on the raw checkpoint, class id `0` is a reserved "none"/background
|
| 77 |
+
slot, so predicted class ids are shifted by one β see `infer.py`.
|
| 78 |
+
|
| 79 |
+
| Our class | Class name | Description |
|
| 80 |
+
| -- | --------- | --------------------- |
|
| 81 |
+
| 0 | header | running title / marginal text at top or side |
|
| 82 |
+
| 1 | text-area | main body text (one box per column) |
|
| 83 |
+
| 2 | footnote | notes below the text area |
|
| 84 |
+
| 3 | footer | folio numbers / marginal text at bottom or side |
|
| 85 |
+
|
| 86 |
+
## Recommended usage β per-class confidence thresholds
|
| 87 |
+
|
| 88 |
+
Like the primary RT-DETR-l release, this detector is recall-happy, so the best
|
| 89 |
+
operating point differs by class. These are each class's own max-F1 confidence
|
| 90 |
+
from a native per-class sweep on the test set:
|
| 91 |
+
|
| 92 |
+
| class | recommended conf |
|
| 93 |
+
| --- | --- |
|
| 94 |
+
| header (0) | **0.46** |
|
| 95 |
+
| text-area (1) | **0.32** |
|
| 96 |
+
| footnote (2) | **0.26** |
|
| 97 |
+
| footer (3) | **0.52** |
|
| 98 |
+
|
| 99 |
+
If you need a single global threshold, **0.30** is the best compromise (it is
|
| 100 |
+
also the operating point used for the cross-architecture comparison in the
|
| 101 |
+
blog post).
|
| 102 |
+
|
| 103 |
+
### Inference
|
| 104 |
+
|
| 105 |
+
```python
|
| 106 |
+
from rfdetr import RFDETRLarge
|
| 107 |
+
|
| 108 |
+
model = RFDETRLarge.from_checkpoint("rfdetr_tibetan_book_layout.pth")
|
| 109 |
+
CLASS_CONF = {0: 0.46, 1: 0.32, 2: 0.26, 3: 0.52} # header, text-area, footnote, footer
|
| 110 |
+
names = {0: "header", 1: "text-area", 2: "footnote", 3: "footer"}
|
| 111 |
+
|
| 112 |
+
det = model.predict("page.jpg", threshold=min(CLASS_CONF.values()), shape=(1024, 1024))
|
| 113 |
+
for box, cls_id, score in zip(det.xyxy, det.class_id, det.confidence):
|
| 114 |
+
cls = int(cls_id) - 1 # class 0 on the checkpoint is background
|
| 115 |
+
if cls < 0 or cls > 3 or score < CLASS_CONF[cls]:
|
| 116 |
+
continue
|
| 117 |
+
print(names[cls], round(float(score), 3), box.tolist())
|
| 118 |
+
```
|
| 119 |
+
|
| 120 |
+
A ready-made CLI (`infer.py`) with the thresholds baked in is included in this
|
| 121 |
+
repo.
|
| 122 |
+
|
| 123 |
+
### Downloading the weights
|
| 124 |
+
|
| 125 |
+
```python
|
| 126 |
+
from huggingface_hub import hf_hub_download
|
| 127 |
+
path = hf_hub_download("BDRC/Tibetan-Modern-Book-Layout-Detection-RFDETR",
|
| 128 |
+
"rfdetr_tibetan_book_layout.pth")
|
| 129 |
+
```
|
| 130 |
+
|
| 131 |
+
## Performance
|
| 132 |
+
|
| 133 |
+
Evaluated on the **held-out test split** (860 images) of
|
| 134 |
+
`BDRC/TDLA-Training-Dataset-v2`.
|
| 135 |
+
|
| 136 |
+
### Native 4-class metrics
|
| 137 |
+
|
| 138 |
+
| class | P | R | F1 | mAP@0.5 | mAP@0.5:0.95 |
|
| 139 |
+
| --- | --- | --- | --- | --- | --- |
|
| 140 |
+
| header | 0.977 | 0.960 | 0.968 | 0.986 | 0.757 |
|
| 141 |
+
| text-area | 0.986 | 0.994 | 0.990 | 0.996 | 0.982 |
|
| 142 |
+
| footnote | 0.913 | 0.933 | 0.923 | 0.973 | 0.816 |
|
| 143 |
+
| footer | 0.961 | 0.964 | 0.963 | 0.969 | 0.697 |
|
| 144 |
+
| **overall** | β | β | **0.961** | **0.981** | **0.813** |
|
| 145 |
+
|
| 146 |
+
The **mAP** columns are threshold-independent β they integrate over the full
|
| 147 |
+
precision/recall curve (every confidence), so they do not depend on any
|
| 148 |
+
operating threshold. The **P / R / F1** columns are reported at each class's
|
| 149 |
+
own **max-F1 confidence** (see the per-class thresholds above), *not* at a
|
| 150 |
+
fixed threshold.
|
| 151 |
+
|
| 152 |
+
### Canonical 3-class metrics
|
| 153 |
+
|
| 154 |
+
Header + footer are combined into one `header-footer` class (matched
|
| 155 |
+
individually), text-area is compared as one merged envelope, and footnote is
|
| 156 |
+
left as-is β a fair space in which every fine-tuned architecture in the blog
|
| 157 |
+
post was compared.
|
| 158 |
+
|
| 159 |
+
| class | best-F1 |
|
| 160 |
+
| --- | --- |
|
| 161 |
+
| header-footer | 0.963 |
|
| 162 |
+
| text-area | 0.994 |
|
| 163 |
+
| footnote | 0.923 |
|
| 164 |
+
| **mean F1** | **0.960** (@ conf 0.30) |
|
| 165 |
+
|
| 166 |
+
This matches BDRC's primary RT-DETR-l fine-tune (mean F1 0.960) almost exactly.
|
| 167 |
+
|
| 168 |
+
### Contamination (the metric that actually matters for OCR)
|
| 169 |
+
|
| 170 |
+
Of the ground-truth headers/footers and footnotes this model *misses*, the
|
| 171 |
+
share that get folded into its predicted text-area box (silently corrupting
|
| 172 |
+
downstream OCR) rather than dropped cleanly:
|
| 173 |
+
|
| 174 |
+
| region | detected | folded into text-area |
|
| 175 |
+
| --- | --- | --- |
|
| 176 |
+
| header/footer | 97% | 0.1% |
|
| 177 |
+
| footnote | 93% | 7% |
|
| 178 |
+
|
| 179 |
+
For comparison, off-the-shelf systems in the same evaluation ranged from 1.2%
|
| 180 |
+
to 56% on header/footer contamination alone β see the
|
| 181 |
+
[blog post](https://github.com/buda-base/tibetan-book-layout-analysis/blob/main/BLOGPOST.md)
|
| 182 |
+
for the full picture.
|
| 183 |
+
|
| 184 |
+
## Training details
|
| 185 |
+
|
| 186 |
+
| Parameter | Value |
|
| 187 |
+
| --- | --- |
|
| 188 |
+
| Base checkpoint | `rf-detr-large-2026.pth` (Roboflow, Apache-2.0) |
|
| 189 |
+
| Resolution | 1008 |
|
| 190 |
+
| Epochs | 100 planned, early-stopped ~epoch 59 (patience 20) |
|
| 191 |
+
| Batch size | 8 |
|
| 192 |
+
| GPU | single NVIDIA A10G (24 GB) |
|
| 193 |
+
|
| 194 |
+
- **Dataset:** [BDRC/TDLA-Training-Dataset-v2](https://huggingface.co/datasets/BDRC/TDLA-Training-Dataset-v2) β 8,325 images (6,751 train / 714 val / 860 test), volume-level leakage-free splits, augmented images confined to train.
|
| 195 |
+
- **Label variant (`tam2col`):** text-area boxes merged per page except on two-column pages; built with `data/build_curricula.py` in the GitHub repo.
|
| 196 |
+
|
| 197 |
+
## Intended use
|
| 198 |
+
|
| 199 |
+
Automatic layout detection of **modern Tibetan book** pages, as a
|
| 200 |
+
preprocessing step for OCR pipelines, document digitization, structured text
|
| 201 |
+
extraction, and digital-library indexing.
|
| 202 |
+
|
| 203 |
+
## Limitations
|
| 204 |
+
|
| 205 |
+
- Trained on **modern Tibetan books**; performance on traditional pecha,
|
| 206 |
+
manuscripts, or woodblock prints is not characterized and may be poor.
|
| 207 |
+
- Optimized for 1008β1024 px input; very high-resolution scans may benefit
|
| 208 |
+
from a higher inference resolution.
|
| 209 |
+
- The footnote class is rare in the source material (β1.4% of boxes); recall
|
| 210 |
+
is strong on the test set but the class remains the least-represented, and
|
| 211 |
+
this checkpoint's footnote contamination (7%) is higher than BDRC's primary
|
| 212 |
+
RT-DETR-l release (2%) despite a comparable footnote F1 β see the blog
|
| 213 |
+
post's discussion of why F1 and contamination aren't interchangeable.
|
| 214 |
+
- Header/footer boxes are small and easy to over-predict β use the
|
| 215 |
+
recommended per-class thresholds above.
|
| 216 |
+
|
| 217 |
+
## License
|
| 218 |
+
|
| 219 |
+
The model weights are released under the **Apache License 2.0**, matching the
|
| 220 |
+
license of the RF-DETR-L base checkpoint they were fine-tuned from. The **page
|
| 221 |
+
images used for training are not covered by any content license** β they are
|
| 222 |
+
BDRC library scans distributed on a fair-use basis. You are solely responsible
|
| 223 |
+
for your own copyright / rights analysis before use; BDRC accepts no liability
|
| 224 |
+
for misuse. See the
|
| 225 |
+
[dataset card](https://huggingface.co/datasets/BDRC/TDLA-Training-Dataset-v2)
|
| 226 |
+
for the full notice.
|
| 227 |
+
|
| 228 |
+
## Acknowledgements
|
| 229 |
+
|
| 230 |
+
Developed by the [Buddhist Digital Resource Center (BDRC)](https://www.bdrc.io)
|
| 231 |
+
for the BDRC Etext Corpus, with annotations produced and consolidated on the
|
| 232 |
+
Ultralytics platform.
|
| 233 |
+
|
| 234 |
+
## Citation
|
| 235 |
+
|
| 236 |
+
```bibtex
|
| 237 |
+
@software{bdrc_tibetan_book_layout_rfdetr_2026,
|
| 238 |
+
title = {Tibetan Modern Book Layout Detection (RF-DETR-L)},
|
| 239 |
+
author = {Buddhist Digital Resource Center (BDRC)},
|
| 240 |
+
year = {2026},
|
| 241 |
+
url = {https://huggingface.co/BDRC/Tibetan-Modern-Book-Layout-Detection-RFDETR}
|
| 242 |
+
}
|
| 243 |
+
```
|