convnextv2-polyline-base-hebrew-samaritan

Hebrew + Samaritan fine-tune of convnextv2-polyline — ConvNeXtV2-Base backbone with an FPN neck and a D-FINE-style polyline decoder, predicting text-line baselines as polylines (control points + height) for medieval Hebrew and Samaritan manuscripts (PAGE-XML).

Hub repo: johnlockejrr/convnextv2-polyline-base-hebrew-samaritan
Primary file: best_cbad_f1.safetensors (~372 MB, FP32)
SHA256: b092a49b1cf578d569283e166e3d84e3e33d2fede097955b060b2d8f56e49b5c

Init chain: timm ConvNeXtV2-Base (fcmae_ft_in22k_in1k) + partial decoder load from johnlockejrr/dfine-det-large-baseline-stage0 → this fine-tune.

The Stage-0 checkpoint is a D-FINE Large / HGNetv2 polyline model. Only decoder tensors that match shape transfer into this ConvNeXt + FPN stack (backbone and FPN train from ImageNet / scratch). This release is not a continuation of the dfine-det Stage-1 Hebrew product weights.

Not for the ICDAR cBAD / Orli bake-off table. Scores below are on the Hebrew+Samaritan combined validation split used for early stopping.

---

Model summary

Architecture	ConvNeXtV2PolylineModel — ConvNeXtV2 → FPN → PolylineTransformer
Backbone	convnextv2_base.fcmae_ft_in22k_in1k (timm, ImageNet-22k→1k FCMAE fine-tune)
Neck	4-level FPN (out_channels=256); decoder consumes strides 8 / 16 / 32
Decoder	PolylineTransformer — 6 layers, hidden_dim=256, MSDeformAttn, contrastive denoising (num_denoising=100)
Parameters	97.2 M total (87.7 M backbone · ~2.3 M FPN · ~7.3 M decoder)
Queries	300
Geometry	(K=8) control points per line + height (chord-length densify at eval/export)
Canvas	(1280\times1280) letterbox
Init	ImageNet backbone + Stage-0 decoder load (strict=False; FPN not transferred)
Exported weights	EMA (ema_decay=0.9999, warm steps 2000) at monitor peak
Task	Document baseline / text-line detection → PAGE or ALTO export
Scripts	Hebrew (square / medieval) + Samaritan
Recommended conf	0.50 (val cbad_f1_max operating point)
Reading order (export)	Prefer --reading-order rtl for PAGE/ALTO serialize

Detection is set prediction of scored polylines. Polygonal line environments (BLLA) are applied only at PAGE/ALTO serialize time, not in the training objective.

---

Intended use

Use for

• Baseline detection on Hebrew and Samaritan manuscript pages (PAGE-XML workflows)
• Downstream HTR / layout pipelines that need text-line geometry without a D-FINE / HGNetv2 stack
• Research comparing ConvNeXtV2 + FPN vs HGNetv2 + HybridEncoder for the same polyline head and Arrow data

Not for

• Official ICDAR cBAD 2019 bake-offs (wrong track / domain)
• Production OCR transcription (detects lines; does not recognize text)
• Replacing layout region detectors (paragraphs, tables, etc.)
• Drop-in replacement of dfine-det Hebrew Stage-1 without re-validating conf and domain gap

---

Training pipeline

timm ConvNeXtV2-Base (ImageNet-22k → ImageNet-1k FCMAE FT)
        │
        ├─ backbone weights
        │
dfine-det Large Stage-0 polyline pretrain
  Hub: johnlockejrr/dfine-det-large-baseline-stage0
  (decoder tensors only → PolylineTransformer; FPN/backbone skipped)
        │
        ▼
Hebrew + Samaritan fine-tune  ← this release
  (1758 train / 261 val; monitor peak cbad_f1_max ≈ 0.8942 @ ep239, conf 0.50)

Config: configs/hebrew_samaritan_stage0_init_base.yaml in the convnextv2-polyline repository.

For reference, the same data + Stage-0 decoder init on ConvNeXtV2-Tiny peaked near cbad_f1_max ≈ 0.79. This Base run is the scaled backbone experiment.

---

Fine-tuning data

PAGE-XML baselines from:

Corpus	Role	Pages (train / val)
Hebrew_Medieval-seg	Square / medieval Hebrew	333 / 37
sam_44_mss_pango_additional	Samaritan (44+ MSS pack)	1425 / 224
Combined		1758 / 261

Split policy (seed 42; see dfine-det configs/hebrew_samaritan_combined_split.json):

• Samaritan: manuscript-level holdout (~12% pages target) — 43 train MSS / 7 val MSS (no page from a val MS appears in train).
• Hebrew: page-level split within Hebrew_Medieval-seg.

Compiled to Arrow (simplify_eps=0.01 → (K=8) control points) under dfine-det outputs/hebrew_samaritan_combined/arrow/ (schema-compatible with convnextv2-polyline).

---

Training recipe

Hyperparameter	Value
Config	configs/hebrew_samaritan_stage0_init_base.yaml
Backbone	convnextv2_base.fcmae_ft_in22k_in1k (pretrained_backbone: true)
Decoder init	Stage-0 best_cbad_f1.safetensors via load_polyline_decoder_pretrain
Optimizer	AdamW; base LR (5\times10^{-5}) (backbone (0.1\times))
Schedule	Linear warmup 2 epochs + cosine annealing (min_lr_ratio=0.1)
Backbone freeze	First 2 epochs, then unfreeze
Precision	bf16-mixed
Effective batch	8 (micro-batch 2 × accumulate_grad_batches 4)
Cap / early-stop	max 250 epochs; quit: early on cbad_f1_max; patience lag: 25; min_epochs 20; min_delta 0.0005
Outcome	Ran to epoch 248 (full budget); early stop did not fire
Best snapshot	epoch 239 (checkpoints/best-epoch=239.ckpt → EMA best_cbad_f1.safetensors)
Monitor peak	cbad_f1_max ≈ 0.8942 @ conf 0.50
Conf sweep (train)	{0.1, 0.2, 0.3, 0.4, 0.5, 0.6}; fast_conf_sweep: true
Losses / matcher	loss_class=8.0, loss_height=3.0, cost_y=3.0
Gradient clip	0.1
EMA	ema_decay=0.9999 (export uses EMA when warmed)
Augment	Enabled (augment: true)
Match distance	20 px on the 1280 canvas
Image size	1280 letterbox
Seed	42

Training was resumed once from checkpoints/last.ckpt after an interrupt (TensorBoard version_0 → version_1); metrics above are from the completed run through epoch 248.

---

Evaluation

Metric: cBAD-style line F1 — Hungarian matching of densified polylines with mean bidirectional Chamfer cost; TP if cost ≤ dist_thresh (20 px).

Hebrew + Samaritan combined val (early-stop split)

261 pages (hebrew_samaritan_combined val Arrow). Post-hoc test --conf-sweep --fast-conf-sweep on best_cbad_f1.safetensors (EMA export).

conf	cbad_f1	precision	recall	mean_chamfer
0.10	0.7289	0.5770	0.9894	3.48
0.20	0.8784	0.7991	0.9752	4.18
0.30	0.8910	0.8578	0.9268	6.68
0.40	0.8971	0.8994	0.8948	6.67
0.50	0.8978	0.9368	0.8619	5.47
0.60	0.8895	0.9731	0.8192	3.76

Headline (val @ 0.50): F1 0.898 · P 0.937 · R 0.862

Locked operating point: conf = 0.50. Conf 0.40 is within ~0.001 F1 (more balanced P/R). Train monitor peak was cbad_f1_max ≈ 0.8942 @ ep239 (same conf); post-hoc EMA eval is slightly higher. Fixed log tag cbad_f1 (conf 0.1) is much lower — always sweep or use 0.50.

Caveats

• These figures are in-domain val (same split used for early stopping). They are optimistic vs a fully held-out test set.
• An unseen Hebrew multi-export holdout (as used for dfine-det Stage-1) was not scored for this release; re-run test on that Arrow before comparing to Stage-1 unseen F1 (~0.90).
• Val F1 here (0.90) is not interchangeable with dfine-det Stage-1 val (0.95) — different backbone/neck and different init.
• Final epoch (248) train-time cbad_f1_max was 0.8828 (slightly below the ep239 peak).

---

How to use

Install

# From the convnextv2-polyline repository (PyTorch CUDA 12.6 recommended; see scripts/fresh_venv_cu126.sh)
uv sync
# or: pip install -e .

Inference (PAGE XML)

convnextv2-polyline -c configs/hebrew_samaritan_stage0_init_base.yaml -d cuda:0 infer \
  -w best_cbad_f1.safetensors \
  -o page.xml \
  --format page \
  --conf 0.5 \
  --reading-order rtl \
  page.jpg

Evaluate

convnextv2-polyline -c configs/hebrew_samaritan_stage0_init_base.yaml -d cuda:0 test \
  -w best_cbad_f1.safetensors \
  -e /path/to/val.arrow \
  --conf 0.50

Conf sweep:

convnextv2-polyline -c configs/hebrew_samaritan_stage0_init_base.yaml -d cuda:0 test \
  -w best_cbad_f1.safetensors \
  -e /path/to/val.arrow \
  --conf-sweep --fast-conf-sweep

---

Files in this release

File	Description
best_cbad_f1.safetensors	Recommended weights (EMA; monitor peak ≈ 0.8942 @ conf 0.50)
README.md	This model card

Upload the safetensors (+ this README) for Hub users; Lightning .ckpt files are optional (~1.1 GB each).

---

Limitations

• In-domain val F1 (~0.89) will overstate performance on unseen collections until a holdout is measured.
• (K=8) control points underfit strongly curved / damaged lines.
• Vertical “ghost” doubles can still appear on some layouts; cost_y / loss_height mitigate but do not eliminate them.
• FPN neck (vs D-FINE HybridEncoder) and partial Stage-0 transfer leave a gap vs the HGNetv2 Stage-1 product model on the same data.
• Private / institutional manuscript images used in training are not redistributed with the weights; respect each corpus license.
• Do not cite these scores as cBAD 2019 / Orli bake-off results.

---

Citation & credits

ConvNeXt V2 (backbone)

@inproceedings{woo2023convnextv2,
  title     = {ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders},
  author    = {Woo, Sanghyun and Debnath, Shoubhik and Hu, Ronghang and Chen, Xinlei
               and Liu, Zhuang and Kweon, In So and Xie, Saining},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2023}
}

• Weights via timm: convnextv2_base.fcmae_ft_in22k_in1k

D-FINE (polyline decoder lineage)

@inproceedings{peng2025dfine,
  title     = {D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement},
  author    = {Peng, Yansong and Li, Hebei and Wu, Peixi and Zhang, Yueyi and Sun, Xiaoyan and Wu, Feng},
  booktitle = {The Thirteenth International Conference on Learning Representations},
  year      = {2025},
  url       = {https://arxiv.org/abs/2410.13842}
}

• Code: github.com/Peterande/D-FINE
• Weights: huggingface.co/Peterande/D-FINE

Stage-0 polyline pretrain (decoder init)

@software{dfine_det_stage0_large,
  title  = {dfine-det Large Stage-0: Multiscript Polyline Baseline Pretrain},
  year   = {2026},
  url    = {https://huggingface.co/johnlockejrr/dfine-det-large-baseline-stage0}
}

This ConvNeXtV2-Base Hebrew/Samaritan fine-tune

@software{convnextv2_polyline_hebsam_base,
  title  = {ConvNeXtV2-Base Polyline: Hebrew and Samaritan Baseline Detection},
  author = {johnlockejrr},
  year   = {2026},
  note   = {ConvNeXtV2-Base + FPN + D-FINE polyline decoder; Stage-0 decoder init; operating conf 0.50},
  url    = {https://huggingface.co/johnlockejrr/convnextv2-polyline-base-hebrew-samaritan}
}

Additional notices

• Polyline head, matcher, losses, Arrow I/O, and PAGE/ALTO export are vendored from dfine-det (Apache-2.0), itself deriving from D-FINE (Apache-2.0, © 2024 The D-FINE Authors).
• PAGE polygonalization helpers derive from Kraken/BLLA (Apache-2.0, © Benjamin Kiessling) — used at export only.
• Training corpora: follow each dataset’s original license / institutional terms.

---

License

Apache License 2.0 for the convnextv2-polyline code and these weights, consistent with D-FINE’s Apache-2.0 release and the Apache-2.0 ImageNet fine-tuned ConvNeXtV2 checkpoints redistributed via timm. Downstream users must also comply with licenses of any datasets used in further fine-tuning.