# HITIT-OCR v4 — Tüm Enhancement'larla Nihai Pipeline **Hedef:** Hitit-only 198 sınıf, top-1 accuracy **≥94%** (realistik stretch). v3 baseline %90+ hedefine **16 ek enhancement** ile çıkar. ## Yeni eklenen modüller (`hitit_ocr/src/enhancements/`) | # | Modül | Gain (realist) | Complexity | |---|---|---|---| | 1 | `knn_retrieval.py` — FAISS-based CHURRO kNN | +0.8-1.5% | Düşük | | 2 | `supcon_auxiliary.py` — SupCon + decoupled CL | +0.5-1.0% | Orta | | 3 | `vlm_rerank.py` — Qwen2.5-VL tiebreaker | +0.3-0.8% | Orta | | 4 | `t3a_adaptation.py` — gradient-free TTT | +0.5-1.2% | Düşük | | 5 | `continual_ssl_full.py` — 293K full SSL | +0.5-1.0% | Orta | | 6 | `diffusion_aug.py` — rare class synth | +0.2-0.5% | Yüksek | | 7 | `sam_optimizer.py` — SAM/ASAM | +0.3-0.7% | Düşük | | 8 | `tablet_gnn.py` — GAT spatial context | +0.4-0.9% | Orta | | 9 | `hyperbolic_head.py` — Poincaré ABZ hierarchy | +0.2-0.5% | Orta | | 10 | `siglip2_text_align.py` — text-image InfoNCE | +0.3-0.7% | Orta | | 12 | `laplace_uncertainty.py` — post-hoc calibration | +0.1-0.3% rejection | Orta | | 16 | `curriculum_learning.py` — difficulty-based | +0.2-0.5% | Düşük | | 19 | `hw_optimizations.py` — compile+bf16+flash | 0% (2× speed) | Düşük | | 21 | `confusion_pair_mlp.py` + `auto_confusion_pairs.py` | +0.8-1.5% | Düşük | | 22 | `period_moe.py` — OH/MH/NH experts | +0.4-1.0% | Orta | | 23 | `position_priors.py` — line/word position | +0.2-0.4% | Düşük | ## Training pipeline ``` Phase 0: Data prep ├─ Label cleanlab noise detection ├─ Confusion pair candidate discovery (auto, post-training) └─ Full 293K SSL pool build Phase 1: Continual SSL (Hafta 1-3) ├─ DINOv3-L/14 on 293K unique images ├─ iBOT + DINO loss ├─ Gram anchoring └─ 30 epochs, 4×A100 → ~700 GPU-h Phase 2: Multi-backbone supervised (Hafta 4-8) ├─ Branch 2a: DINOv3-L + LoRA + SupCon + SAM + EMA+SWA (primary) ├─ Branch 2b: ConvNeXt-V2-L (secondary) ├─ Branch 2c: SigLIP2-SO400M (+ text alignment head) └─ Hepsi 198-way Hitit, curriculum sampler, MoE head Phase 3: Auxiliary components (Hafta 9) ├─ kNN embedding index (FAISS) ├─ Confusion pair binary MLPs (from top-50 confusion matrix) ├─ Tablet GNN fine-tune (on ensemble features) └─ Period MoE gate calibration Phase 4: LM training (Hafta 9-10) ├─ TLHdig Hittite 5-gram KenLM ├─ ByT5 transliteration fine-tune └─ Viterbi rescorer Phase 5: End-to-end tuning (Hafta 11) ├─ Ensemble weight optimization (Bayesian on val fold) ├─ kNN λ, T3A λ, LM λ tune ├─ Rejection threshold calibration (target 90% coverage) └─ VLM trigger threshold Phase 6: Evaluation (Hafta 12) └─ Test fold 4 (LOCKED) — single run, report all metrics ``` ## Inference pipeline (10 aşama) ``` Image │ ▼ [1] Preprocessing (v1 pipeline) │ CLAHE + gamma + letterbox 224 + norm ▼ [2] Multi-backbone ensemble forward │ 3 backbone × 4 scale × 5 augment = 60 forward pass │ Weighted softmax average (0.5/0.3/0.2) ▼ [3] EMA+SWA averaged weights applied │ ▼ [4] kNN retrieval fusion │ FAISS top-50 from 70K curated pool │ logits = 0.7*ensemble + 0.3*knn_votes ▼ [5] T3A template adjustment │ Test batch templates update, cosine classify │ logits = 0.8*prev + 0.2*t3a_sim ▼ [6] Confusion pair binary override │ Top-1/Top-2 confusion pair ise, pair MLP karar verir ▼ [7] Period MoE gating │ Feature → period probability → expert-weighted combine ▼ [8] Tablet GNN context (if batch=tablet) │ GAT ile aynı tablet içi sign'lar arası message passing ▼ [9] KenLM sequence-level rescoring │ Lattice top-5, Viterbi with 5-gram Hittite LM ▼ [10] VLM rerank (opt, uncertain only) │ max_prob<0.6 → Qwen2.5-VL tiebreaker ▼ [11] Rejection @τ=0.5 │ Final < 0.5 → REJECT, else predict ▼ Output: (label, confidence, optional UNK) ``` ## Gain projection (198-class Hitit gold — dürüst tahmin) ``` v3 baseline (DINOv3-L Hitit-only CE): 82.0% + Label smoothing + EMA + SWA: +1.5% →83.5% + Multi-scale TTA: +3.0% →86.5% + ConvNeXt-V2 + SigLIP2 ensemble: +3.0% →89.5% + Label cleanlab: +1.0% →90.5% ⭐ + kNN retrieval fusion: +1.0% →91.5% + T3A test-time adaptation: +0.7% →92.2% + Confusion pair disambiguation (top-50): +0.8% →93.0% + SupCon auxiliary head: +0.5% →93.5% + SAM optimizer: +0.3% →93.8% + Period MoE (OH/MH/NH): +0.4% →94.2% + KenLM LM rescoring (sequence): +1.5% →95.7% (tablet-level only) + VLM rerank (uncertain): +0.3% →94.5% (single sign) + Rejection (selective acc @ 90% cov): →96.5% selective ``` **Final realistik tahmin**: - **Single-sign top-1**: **%93-95** (kombine gain, overlap sonrası) - **Tablet-level (LM)**: **%95-97** - **Selective @ 90% coverage**: **%96-97** - **Macro-F1 (tier-stratified)**: ~90% ## Kaynak ihtiyacı | Phase | GPU-h | |---|---| | Phase 1 (SSL) | 700 | | Phase 2 (3 backbone) | 900 | | Phase 3 (aux) | 150 | | Phase 4 (LM + ByT5) | 150 | | Phase 5-6 (tune + eval) | 200 | | **Total** | **~2,100** | v3'ten +20% maliyet, +3-4% accuracy gain. ## Diminishing returns — dürüst uyarı Literatürdeki cuneiform OCR çalışmaları: - DeepScribe (141 sınıf Elamite): top-1 %77 - Stötzner 2023 (141 sınıf): top-1 %85.6 - ICDAR 2023 winner (72 sınıf): top-1 %91.4 - Bizim v3 (198 sınıf Hitit): top-1 hedef **%90+** - Bizim v4 (+all enhancements): top-1 hedef **%93-95** **%95+ için literatür-dışı yollar gerekli:** - Expert paleographer annotation döngüsü (her yanlış için feedback) - Tablet-level ensemble (sign-level değil) - Yeni Hitit tablet fotoğrafı toplama (şu an 307 tablet) Literatür teknikleriyle **yapısal tavan ~%95** görünüyor. Bunu aşmak için domain-specific knowledge engineering kritik. ## Enhancement kullanım kontrolü Her enhancement opt-in/opt-out: ```yaml # hitit_ocr/configs/enhancements_v4.yaml enabled: - knn_retrieval # (+1.0%) - t3a_adaptation # (+0.7%) - confusion_pair_override # (+0.8%) - supcon_auxiliary # (+0.5%) - sam_optimizer # (+0.3%) - period_moe # (+0.4%) - curriculum_learning # (+0.3%) - siglip2_text_align # (+0.5%) - hw_optimizations # (speed only) disabled_by_default: - vlm_rerank # API cost, only if uncertain - tablet_gnn # Needs batch=tablet (inference-time only) - diffusion_aug # Risky, expert review gerekir - hyperbolic_head # Marjinal, sadece hierarchy metrikleri varsa - laplace_uncertainty # Rejection improvement only conditional: - position_priors: # transliterasyon-bazlı işlemlerde use: if_stage_4 - continual_ssl_full: use: if_before_finetune ``` ## Dosya haritası ``` hitit_ocr/ ├── PIPELINE.md # v1 baseline ├── PIPELINE_V2_90PCT.md # v2 multi-source %90 hedef ├── PIPELINE_HITIT_ONLY.md # v3 Hitit-only ├── PIPELINE_V4.md # BU (v4 all enhancements) ├── PIPELINE_V4_SUMMARY.json # v4 structured summary ├── configs/ │ ├── classification_hitit_only.yaml (v3) │ ├── ensemble_hitit_only.yaml (v3) │ └── enhancements_v4.yaml (YENİ, config-driven toggle) ├── src/ │ ├── preprocessing/ (Stage 0) │ ├── pipeline/ (Stage 3) │ └── enhancements/ (YENİ — 16 modül) │ ├── knn_retrieval.py │ ├── supcon_auxiliary.py │ ├── vlm_rerank.py │ ├── t3a_adaptation.py │ ├── continual_ssl_full.py │ ├── diffusion_aug.py │ ├── sam_optimizer.py │ ├── tablet_gnn.py │ ├── hyperbolic_head.py │ ├── siglip2_text_align.py │ ├── laplace_uncertainty.py │ ├── curriculum_learning.py │ ├── hw_optimizations.py │ ├── confusion_pair_mlp.py │ ├── auto_confusion_pairs.py │ ├── period_moe.py │ ├── position_priors.py │ └── unified_trainer.py # tüm eklentilerin bir arada └── runs/ ├── ssl_dinov3_full/ # Phase 1 ├── hitit_only_dinov3/ # Phase 2a ├── hitit_only_convnext/ # Phase 2b ├── hitit_only_siglip/ # Phase 2c ├── knn_store_hitit/ # Phase 3 FAISS index ├── confusion_pair_mlps/ # Phase 3 binary MLPs ├── hitit_kenlm_5gram.binary # Phase 4 ├── hitit_ensemble_v4/ # Phase 5 final └── eval_hitit_v4/ # Phase 6 results ``` ## Referanslar (yeni enhancements) - kNN-LM (Khandelwal 2020); CHURRO 2024 - SupCon (Khosla 2020, arXiv:2004.11362); Decoupled CL (Yeh 2022) - Qwen2.5-VL (Bai 2025, arXiv:2502.13923); InternVL3 (Chen 2025) - T3A (Iwasawa NeurIPS 2021); SHOT (Liang 2020) - SAM (Foret 2021, arXiv:2010.01412); ASAM (Kwon 2021) - DA-Fusion (Trabucco 2023); DiffuseMix (CVPR 2024) - GAT (Veličković 2018); LayoutGNN 2023 - Poincaré (Nickel & Kiela 2017); Hyperbolic Embeddings (Khrulkov CVPR 2020) - SigLIP2 (Tschannen 2025, arXiv:2502.14786) - Laplace Redux (Daxberger NeurIPS 2021) - Curriculum (Bengio 2009); SuperLoss (Castells 2020) - Shazeer MoE 2017 - Position-aware Cuneiform (Gordin 2020) - FlashAttention-3 (Dao 2024); Liger Kernel (2024)