Eroux commited on
Commit
3efb1b0
Β·
verified Β·
1 Parent(s): 1fc302b

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +243 -0
README.md ADDED
@@ -0,0 +1,243 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: rfdetr
4
+ pipeline_tag: object-detection
5
+ language:
6
+ - bo
7
+ tags:
8
+ - tibetan
9
+ - document-layout-analysis
10
+ - rf-detr
11
+ - rfdetr
12
+ - object-detection
13
+ - bounding-box
14
+ - BDRC
15
+ datasets:
16
+ - BDRC/TDLA-Training-Dataset-v2
17
+ metrics:
18
+ - name: canonical mean AP50 (test)
19
+ type: mAP
20
+ value: 0.960
21
+ model-index:
22
+ - name: Tibetan-Modern-Book-Layout-Detection-RFDETR
23
+ results:
24
+ - task:
25
+ type: object-detection
26
+ dataset:
27
+ name: TDLA-Training-Dataset-v2 (test)
28
+ type: BDRC/TDLA-Training-Dataset-v2
29
+ metrics:
30
+ - type: mAP
31
+ name: mAP@0.5 (native, test)
32
+ value: 0.996
33
+ - type: mAP
34
+ name: mAP@0.5:0.95 (native, test)
35
+ value: 0.813
36
+ ---
37
+
38
+ # Tibetan Modern Book Layout Detection (RF-DETR-L)
39
+
40
+ An **RF-DETR-L** ([Roboflow](https://github.com/roboflow/rf-detr), DINOv2 backbone)
41
+ object detector that locates the four structural regions of a **modern Tibetan
42
+ book** page β€” **header**, **text-area**, **footnote**, **footer** β€” as a
43
+ preprocessing step for OCR and etext production.
44
+
45
+ - **Training code, recipes & write-up:** [buda-base/tibetan-book-layout-analysis](https://github.com/buda-base/tibetan-book-layout-analysis)
46
+ - **Dataset:** [BDRC/TDLA-Training-Dataset-v2](https://huggingface.co/datasets/BDRC/TDLA-Training-Dataset-v2) (gated, fair-use)
47
+
48
+ > This is one of several architectures BDRC fine-tuned on the same labels and
49
+ > recipe to test how much the choice of architecture matters (see the
50
+ > [blog post](https://github.com/buda-base/tibetan-book-layout-analysis/blob/main/BLOGPOST.md)).
51
+ > The primary, production release is the RT-DETR-l fine-tune at
52
+ > [BDRC/Tibetan-Modern-Book-Layout-Detection](https://huggingface.co/BDRC/Tibetan-Modern-Book-Layout-Detection).
53
+ > This RF-DETR-L checkpoint matches it almost exactly on every metric, and is
54
+ > published as a **permissively-licensed (Apache-2.0) alternative** for anyone
55
+ > who can't use the RT-DETR-l release's AGPL-derived training stack.
56
+
57
+ ## Model description
58
+
59
+ This is an RF-DETR-L fine-tuned with the **`tam2col`** labelling scheme: text-area
60
+ boxes merged into one envelope per page, except on genuine two-column pages,
61
+ where it keeps one box per column. Header and footer are kept as separate
62
+ classes (they can be combined losslessly downstream).
63
+
64
+ | Property | Value |
65
+ | --- | --- |
66
+ | Architecture | RF-DETR-L (via [`rfdetr`](https://github.com/roboflow/rf-detr), DINOv2-small windowed backbone) |
67
+ | Task | Object detection |
68
+ | Base checkpoint | `rf-detr-large-2026.pth` (Roboflow, Apache-2.0) |
69
+ | Resolution | 1008 Γ— 1008 |
70
+ | Number of classes | 4 (+ background) |
71
+ | Framework | `rfdetr` (`RFDETRLarge`) |
72
+ | Weights file | `rfdetr_tibetan_book_layout.pth` |
73
+
74
+ ## Classes
75
+
76
+ Note: on the raw checkpoint, class id `0` is a reserved "none"/background
77
+ slot, so predicted class ids are shifted by one β€” see `infer.py`.
78
+
79
+ | Our class | Class name | Description |
80
+ | -- | --------- | --------------------- |
81
+ | 0 | header | running title / marginal text at top or side |
82
+ | 1 | text-area | main body text (one box per column) |
83
+ | 2 | footnote | notes below the text area |
84
+ | 3 | footer | folio numbers / marginal text at bottom or side |
85
+
86
+ ## Recommended usage β€” per-class confidence thresholds
87
+
88
+ Like the primary RT-DETR-l release, this detector is recall-happy, so the best
89
+ operating point differs by class. These are each class's own max-F1 confidence
90
+ from a native per-class sweep on the test set:
91
+
92
+ | class | recommended conf |
93
+ | --- | --- |
94
+ | header (0) | **0.46** |
95
+ | text-area (1) | **0.32** |
96
+ | footnote (2) | **0.26** |
97
+ | footer (3) | **0.52** |
98
+
99
+ If you need a single global threshold, **0.30** is the best compromise (it is
100
+ also the operating point used for the cross-architecture comparison in the
101
+ blog post).
102
+
103
+ ### Inference
104
+
105
+ ```python
106
+ from rfdetr import RFDETRLarge
107
+
108
+ model = RFDETRLarge.from_checkpoint("rfdetr_tibetan_book_layout.pth")
109
+ CLASS_CONF = {0: 0.46, 1: 0.32, 2: 0.26, 3: 0.52} # header, text-area, footnote, footer
110
+ names = {0: "header", 1: "text-area", 2: "footnote", 3: "footer"}
111
+
112
+ det = model.predict("page.jpg", threshold=min(CLASS_CONF.values()), shape=(1024, 1024))
113
+ for box, cls_id, score in zip(det.xyxy, det.class_id, det.confidence):
114
+ cls = int(cls_id) - 1 # class 0 on the checkpoint is background
115
+ if cls < 0 or cls > 3 or score < CLASS_CONF[cls]:
116
+ continue
117
+ print(names[cls], round(float(score), 3), box.tolist())
118
+ ```
119
+
120
+ A ready-made CLI (`infer.py`) with the thresholds baked in is included in this
121
+ repo.
122
+
123
+ ### Downloading the weights
124
+
125
+ ```python
126
+ from huggingface_hub import hf_hub_download
127
+ path = hf_hub_download("BDRC/Tibetan-Modern-Book-Layout-Detection-RFDETR",
128
+ "rfdetr_tibetan_book_layout.pth")
129
+ ```
130
+
131
+ ## Performance
132
+
133
+ Evaluated on the **held-out test split** (860 images) of
134
+ `BDRC/TDLA-Training-Dataset-v2`.
135
+
136
+ ### Native 4-class metrics
137
+
138
+ | class | P | R | F1 | mAP@0.5 | mAP@0.5:0.95 |
139
+ | --- | --- | --- | --- | --- | --- |
140
+ | header | 0.977 | 0.960 | 0.968 | 0.986 | 0.757 |
141
+ | text-area | 0.986 | 0.994 | 0.990 | 0.996 | 0.982 |
142
+ | footnote | 0.913 | 0.933 | 0.923 | 0.973 | 0.816 |
143
+ | footer | 0.961 | 0.964 | 0.963 | 0.969 | 0.697 |
144
+ | **overall** | β€” | β€” | **0.961** | **0.981** | **0.813** |
145
+
146
+ The **mAP** columns are threshold-independent β€” they integrate over the full
147
+ precision/recall curve (every confidence), so they do not depend on any
148
+ operating threshold. The **P / R / F1** columns are reported at each class's
149
+ own **max-F1 confidence** (see the per-class thresholds above), *not* at a
150
+ fixed threshold.
151
+
152
+ ### Canonical 3-class metrics
153
+
154
+ Header + footer are combined into one `header-footer` class (matched
155
+ individually), text-area is compared as one merged envelope, and footnote is
156
+ left as-is β€” a fair space in which every fine-tuned architecture in the blog
157
+ post was compared.
158
+
159
+ | class | best-F1 |
160
+ | --- | --- |
161
+ | header-footer | 0.963 |
162
+ | text-area | 0.994 |
163
+ | footnote | 0.923 |
164
+ | **mean F1** | **0.960** (@ conf 0.30) |
165
+
166
+ This matches BDRC's primary RT-DETR-l fine-tune (mean F1 0.960) almost exactly.
167
+
168
+ ### Contamination (the metric that actually matters for OCR)
169
+
170
+ Of the ground-truth headers/footers and footnotes this model *misses*, the
171
+ share that get folded into its predicted text-area box (silently corrupting
172
+ downstream OCR) rather than dropped cleanly:
173
+
174
+ | region | detected | folded into text-area |
175
+ | --- | --- | --- |
176
+ | header/footer | 97% | 0.1% |
177
+ | footnote | 93% | 7% |
178
+
179
+ For comparison, off-the-shelf systems in the same evaluation ranged from 1.2%
180
+ to 56% on header/footer contamination alone β€” see the
181
+ [blog post](https://github.com/buda-base/tibetan-book-layout-analysis/blob/main/BLOGPOST.md)
182
+ for the full picture.
183
+
184
+ ## Training details
185
+
186
+ | Parameter | Value |
187
+ | --- | --- |
188
+ | Base checkpoint | `rf-detr-large-2026.pth` (Roboflow, Apache-2.0) |
189
+ | Resolution | 1008 |
190
+ | Epochs | 100 planned, early-stopped ~epoch 59 (patience 20) |
191
+ | Batch size | 8 |
192
+ | GPU | single NVIDIA A10G (24 GB) |
193
+
194
+ - **Dataset:** [BDRC/TDLA-Training-Dataset-v2](https://huggingface.co/datasets/BDRC/TDLA-Training-Dataset-v2) β€” 8,325 images (6,751 train / 714 val / 860 test), volume-level leakage-free splits, augmented images confined to train.
195
+ - **Label variant (`tam2col`):** text-area boxes merged per page except on two-column pages; built with `data/build_curricula.py` in the GitHub repo.
196
+
197
+ ## Intended use
198
+
199
+ Automatic layout detection of **modern Tibetan book** pages, as a
200
+ preprocessing step for OCR pipelines, document digitization, structured text
201
+ extraction, and digital-library indexing.
202
+
203
+ ## Limitations
204
+
205
+ - Trained on **modern Tibetan books**; performance on traditional pecha,
206
+ manuscripts, or woodblock prints is not characterized and may be poor.
207
+ - Optimized for 1008–1024 px input; very high-resolution scans may benefit
208
+ from a higher inference resolution.
209
+ - The footnote class is rare in the source material (β‰ˆ1.4% of boxes); recall
210
+ is strong on the test set but the class remains the least-represented, and
211
+ this checkpoint's footnote contamination (7%) is higher than BDRC's primary
212
+ RT-DETR-l release (2%) despite a comparable footnote F1 β€” see the blog
213
+ post's discussion of why F1 and contamination aren't interchangeable.
214
+ - Header/footer boxes are small and easy to over-predict β€” use the
215
+ recommended per-class thresholds above.
216
+
217
+ ## License
218
+
219
+ The model weights are released under the **Apache License 2.0**, matching the
220
+ license of the RF-DETR-L base checkpoint they were fine-tuned from. The **page
221
+ images used for training are not covered by any content license** β€” they are
222
+ BDRC library scans distributed on a fair-use basis. You are solely responsible
223
+ for your own copyright / rights analysis before use; BDRC accepts no liability
224
+ for misuse. See the
225
+ [dataset card](https://huggingface.co/datasets/BDRC/TDLA-Training-Dataset-v2)
226
+ for the full notice.
227
+
228
+ ## Acknowledgements
229
+
230
+ Developed by the [Buddhist Digital Resource Center (BDRC)](https://www.bdrc.io)
231
+ for the BDRC Etext Corpus, with annotations produced and consolidated on the
232
+ Ultralytics platform.
233
+
234
+ ## Citation
235
+
236
+ ```bibtex
237
+ @software{bdrc_tibetan_book_layout_rfdetr_2026,
238
+ title = {Tibetan Modern Book Layout Detection (RF-DETR-L)},
239
+ author = {Buddhist Digital Resource Center (BDRC)},
240
+ year = {2026},
241
+ url = {https://huggingface.co/BDRC/Tibetan-Modern-Book-Layout-Detection-RFDETR}
242
+ }
243
+ ```