HITIT-OCR v4 — Tüm Enhancement'larla Nihai Pipeline
Hedef: Hitit-only 198 sınıf, top-1 accuracy ≥94% (realistik stretch).
v3 baseline %90+ hedefine 16 ek enhancement ile çıkar.
Yeni eklenen modüller (hitit_ocr/src/enhancements/)
| # | Modül | Gain (realist) | Complexity |
|---|---|---|---|
| 1 | knn_retrieval.py — FAISS-based CHURRO kNN |
+0.8-1.5% | Düşük |
| 2 | supcon_auxiliary.py — SupCon + decoupled CL |
+0.5-1.0% | Orta |
| 3 | vlm_rerank.py — Qwen2.5-VL tiebreaker |
+0.3-0.8% | Orta |
| 4 | t3a_adaptation.py — gradient-free TTT |
+0.5-1.2% | Düşük |
| 5 | continual_ssl_full.py — 293K full SSL |
+0.5-1.0% | Orta |
| 6 | diffusion_aug.py — rare class synth |
+0.2-0.5% | Yüksek |
| 7 | sam_optimizer.py — SAM/ASAM |
+0.3-0.7% | Düşük |
| 8 | tablet_gnn.py — GAT spatial context |
+0.4-0.9% | Orta |
| 9 | hyperbolic_head.py — Poincaré ABZ hierarchy |
+0.2-0.5% | Orta |
| 10 | siglip2_text_align.py — text-image InfoNCE |
+0.3-0.7% | Orta |
| 12 | laplace_uncertainty.py — post-hoc calibration |
+0.1-0.3% rejection | Orta |
| 16 | curriculum_learning.py — difficulty-based |
+0.2-0.5% | Düşük |
| 19 | hw_optimizations.py — compile+bf16+flash |
0% (2× speed) | Düşük |
| 21 | confusion_pair_mlp.py + auto_confusion_pairs.py |
+0.8-1.5% | Düşük |
| 22 | period_moe.py — OH/MH/NH experts |
+0.4-1.0% | Orta |
| 23 | position_priors.py — line/word position |
+0.2-0.4% | Düşük |
Training pipeline
Phase 0: Data prep
├─ Label cleanlab noise detection
├─ Confusion pair candidate discovery (auto, post-training)
└─ Full 293K SSL pool build
Phase 1: Continual SSL (Hafta 1-3)
├─ DINOv3-L/14 on 293K unique images
├─ iBOT + DINO loss
├─ Gram anchoring
└─ 30 epochs, 4×A100 → ~700 GPU-h
Phase 2: Multi-backbone supervised (Hafta 4-8)
├─ Branch 2a: DINOv3-L + LoRA + SupCon + SAM + EMA+SWA (primary)
├─ Branch 2b: ConvNeXt-V2-L (secondary)
├─ Branch 2c: SigLIP2-SO400M (+ text alignment head)
└─ Hepsi 198-way Hitit, curriculum sampler, MoE head
Phase 3: Auxiliary components (Hafta 9)
├─ kNN embedding index (FAISS)
├─ Confusion pair binary MLPs (from top-50 confusion matrix)
├─ Tablet GNN fine-tune (on ensemble features)
└─ Period MoE gate calibration
Phase 4: LM training (Hafta 9-10)
├─ TLHdig Hittite 5-gram KenLM
├─ ByT5 transliteration fine-tune
└─ Viterbi rescorer
Phase 5: End-to-end tuning (Hafta 11)
├─ Ensemble weight optimization (Bayesian on val fold)
├─ kNN λ, T3A λ, LM λ tune
├─ Rejection threshold calibration (target 90% coverage)
└─ VLM trigger threshold
Phase 6: Evaluation (Hafta 12)
└─ Test fold 4 (LOCKED) — single run, report all metrics
Inference pipeline (10 aşama)
Image
│
▼
[1] Preprocessing (v1 pipeline)
│ CLAHE + gamma + letterbox 224 + norm
▼
[2] Multi-backbone ensemble forward
│ 3 backbone × 4 scale × 5 augment = 60 forward pass
│ Weighted softmax average (0.5/0.3/0.2)
▼
[3] EMA+SWA averaged weights applied
│
▼
[4] kNN retrieval fusion
│ FAISS top-50 from 70K curated pool
│ logits = 0.7*ensemble + 0.3*knn_votes
▼
[5] T3A template adjustment
│ Test batch templates update, cosine classify
│ logits = 0.8*prev + 0.2*t3a_sim
▼
[6] Confusion pair binary override
│ Top-1/Top-2 confusion pair ise, pair MLP karar verir
▼
[7] Period MoE gating
│ Feature → period probability → expert-weighted combine
▼
[8] Tablet GNN context (if batch=tablet)
│ GAT ile aynı tablet içi sign'lar arası message passing
▼
[9] KenLM sequence-level rescoring
│ Lattice top-5, Viterbi with 5-gram Hittite LM
▼
[10] VLM rerank (opt, uncertain only)
│ max_prob<0.6 → Qwen2.5-VL tiebreaker
▼
[11] Rejection @τ=0.5
│ Final < 0.5 → REJECT, else predict
▼
Output: (label, confidence, optional UNK)
Gain projection (198-class Hitit gold — dürüst tahmin)
v3 baseline (DINOv3-L Hitit-only CE): 82.0%
+ Label smoothing + EMA + SWA: +1.5% →83.5%
+ Multi-scale TTA: +3.0% →86.5%
+ ConvNeXt-V2 + SigLIP2 ensemble: +3.0% →89.5%
+ Label cleanlab: +1.0% →90.5% ⭐
+ kNN retrieval fusion: +1.0% →91.5%
+ T3A test-time adaptation: +0.7% →92.2%
+ Confusion pair disambiguation (top-50): +0.8% →93.0%
+ SupCon auxiliary head: +0.5% →93.5%
+ SAM optimizer: +0.3% →93.8%
+ Period MoE (OH/MH/NH): +0.4% →94.2%
+ KenLM LM rescoring (sequence): +1.5% →95.7% (tablet-level only)
+ VLM rerank (uncertain): +0.3% →94.5% (single sign)
+ Rejection (selective acc @ 90% cov): →96.5% selective
Final realistik tahmin:
- Single-sign top-1: %93-95 (kombine gain, overlap sonrası)
- Tablet-level (LM): %95-97
- Selective @ 90% coverage: %96-97
- Macro-F1 (tier-stratified): ~90%
Kaynak ihtiyacı
| Phase | GPU-h |
|---|---|
| Phase 1 (SSL) | 700 |
| Phase 2 (3 backbone) | 900 |
| Phase 3 (aux) | 150 |
| Phase 4 (LM + ByT5) | 150 |
| Phase 5-6 (tune + eval) | 200 |
| Total | ~2,100 |
v3'ten +20% maliyet, +3-4% accuracy gain.
Diminishing returns — dürüst uyarı
Literatürdeki cuneiform OCR çalışmaları:
- DeepScribe (141 sınıf Elamite): top-1 %77
- Stötzner 2023 (141 sınıf): top-1 %85.6
- ICDAR 2023 winner (72 sınıf): top-1 %91.4
- Bizim v3 (198 sınıf Hitit): top-1 hedef %90+
- Bizim v4 (+all enhancements): top-1 hedef %93-95
%95+ için literatür-dışı yollar gerekli:
- Expert paleographer annotation döngüsü (her yanlış için feedback)
- Tablet-level ensemble (sign-level değil)
- Yeni Hitit tablet fotoğrafı toplama (şu an 307 tablet)
Literatür teknikleriyle yapısal tavan ~%95 görünüyor. Bunu aşmak için domain-specific knowledge engineering kritik.
Enhancement kullanım kontrolü
Her enhancement opt-in/opt-out:
# hitit_ocr/configs/enhancements_v4.yaml
enabled:
- knn_retrieval # (+1.0%)
- t3a_adaptation # (+0.7%)
- confusion_pair_override # (+0.8%)
- supcon_auxiliary # (+0.5%)
- sam_optimizer # (+0.3%)
- period_moe # (+0.4%)
- curriculum_learning # (+0.3%)
- siglip2_text_align # (+0.5%)
- hw_optimizations # (speed only)
disabled_by_default:
- vlm_rerank # API cost, only if uncertain
- tablet_gnn # Needs batch=tablet (inference-time only)
- diffusion_aug # Risky, expert review gerekir
- hyperbolic_head # Marjinal, sadece hierarchy metrikleri varsa
- laplace_uncertainty # Rejection improvement only
conditional:
- position_priors: # transliterasyon-bazlı işlemlerde
use: if_stage_4
- continual_ssl_full:
use: if_before_finetune
Dosya haritası
hitit_ocr/
├── PIPELINE.md # v1 baseline
├── PIPELINE_V2_90PCT.md # v2 multi-source %90 hedef
├── PIPELINE_HITIT_ONLY.md # v3 Hitit-only
├── PIPELINE_V4.md # BU (v4 all enhancements)
├── PIPELINE_V4_SUMMARY.json # v4 structured summary
├── configs/
│ ├── classification_hitit_only.yaml (v3)
│ ├── ensemble_hitit_only.yaml (v3)
│ └── enhancements_v4.yaml (YENİ, config-driven toggle)
├── src/
│ ├── preprocessing/ (Stage 0)
│ ├── pipeline/ (Stage 3)
│ └── enhancements/ (YENİ — 16 modül)
│ ├── knn_retrieval.py
│ ├── supcon_auxiliary.py
│ ├── vlm_rerank.py
│ ├── t3a_adaptation.py
│ ├── continual_ssl_full.py
│ ├── diffusion_aug.py
│ ├── sam_optimizer.py
│ ├── tablet_gnn.py
│ ├── hyperbolic_head.py
│ ├── siglip2_text_align.py
│ ├── laplace_uncertainty.py
│ ├── curriculum_learning.py
│ ├── hw_optimizations.py
│ ├── confusion_pair_mlp.py
│ ├── auto_confusion_pairs.py
│ ├── period_moe.py
│ ├── position_priors.py
│ └── unified_trainer.py # tüm eklentilerin bir arada
└── runs/
├── ssl_dinov3_full/ # Phase 1
├── hitit_only_dinov3/ # Phase 2a
├── hitit_only_convnext/ # Phase 2b
├── hitit_only_siglip/ # Phase 2c
├── knn_store_hitit/ # Phase 3 FAISS index
├── confusion_pair_mlps/ # Phase 3 binary MLPs
├── hitit_kenlm_5gram.binary # Phase 4
├── hitit_ensemble_v4/ # Phase 5 final
└── eval_hitit_v4/ # Phase 6 results
Referanslar (yeni enhancements)
- kNN-LM (Khandelwal 2020); CHURRO 2024
- SupCon (Khosla 2020, arXiv:2004.11362); Decoupled CL (Yeh 2022)
- Qwen2.5-VL (Bai 2025, arXiv:2502.13923); InternVL3 (Chen 2025)
- T3A (Iwasawa NeurIPS 2021); SHOT (Liang 2020)
- SAM (Foret 2021, arXiv:2010.01412); ASAM (Kwon 2021)
- DA-Fusion (Trabucco 2023); DiffuseMix (CVPR 2024)
- GAT (Veličković 2018); LayoutGNN 2023
- Poincaré (Nickel & Kiela 2017); Hyperbolic Embeddings (Khrulkov CVPR 2020)
- SigLIP2 (Tschannen 2025, arXiv:2502.14786)
- Laplace Redux (Daxberger NeurIPS 2021)
- Curriculum (Bengio 2009); SuperLoss (Castells 2020)
- Shazeer MoE 2017
- Position-aware Cuneiform (Gordin 2020)
- FlashAttention-3 (Dao 2024); Liger Kernel (2024)