code/PIPELINE_V4.md

# HITIT-OCR v4 — Tüm Enhancement'larla Nihai Pipeline

**Hedef:** Hitit-only 198 sınıf, top-1 accuracy **≥94%** (realistik stretch).

v3 baseline %90+ hedefine **16 ek enhancement** ile çıkar.

## Yeni eklenen modüller (`hitit_ocr/src/enhancements/`)

| # | Modül | Gain (realist) | Complexity |
|---|---|---|---|
| 1 | `knn_retrieval.py` — FAISS-based CHURRO kNN | +0.8-1.5% | Düşük |
| 2 | `supcon_auxiliary.py` — SupCon + decoupled CL | +0.5-1.0% | Orta |
| 3 | `vlm_rerank.py` — Qwen2.5-VL tiebreaker | +0.3-0.8% | Orta |
| 4 | `t3a_adaptation.py` — gradient-free TTT | +0.5-1.2% | Düşük |
| 5 | `continual_ssl_full.py` — 293K full SSL | +0.5-1.0% | Orta |
| 6 | `diffusion_aug.py` — rare class synth | +0.2-0.5% | Yüksek |
| 7 | `sam_optimizer.py` — SAM/ASAM | +0.3-0.7% | Düşük |
| 8 | `tablet_gnn.py` — GAT spatial context | +0.4-0.9% | Orta |
| 9 | `hyperbolic_head.py` — Poincaré ABZ hierarchy | +0.2-0.5% | Orta |
| 10 | `siglip2_text_align.py` — text-image InfoNCE | +0.3-0.7% | Orta |
| 12 | `laplace_uncertainty.py` — post-hoc calibration | +0.1-0.3% rejection | Orta |
| 16 | `curriculum_learning.py` — difficulty-based | +0.2-0.5% | Düşük |
| 19 | `hw_optimizations.py` — compile+bf16+flash | 0% (2× speed) | Düşük |
| 21 | `confusion_pair_mlp.py` + `auto_confusion_pairs.py` | +0.8-1.5% | Düşük |
| 22 | `period_moe.py` — OH/MH/NH experts | +0.4-1.0% | Orta |
| 23 | `position_priors.py` — line/word position | +0.2-0.4% | Düşük |

## Training pipeline

```
Phase 0: Data prep
  ├─ Label cleanlab noise detection
  ├─ Confusion pair candidate discovery (auto, post-training)
  └─ Full 293K SSL pool build

Phase 1: Continual SSL (Hafta 1-3)
  ├─ DINOv3-L/14 on 293K unique images
  ├─ iBOT + DINO loss
  ├─ Gram anchoring
  └─ 30 epochs, 4×A100 → ~700 GPU-h

Phase 2: Multi-backbone supervised (Hafta 4-8)
  ├─ Branch 2a: DINOv3-L + LoRA + SupCon + SAM + EMA+SWA (primary)
  ├─ Branch 2b: ConvNeXt-V2-L (secondary)
  ├─ Branch 2c: SigLIP2-SO400M (+ text alignment head)
  └─ Hepsi 198-way Hitit, curriculum sampler, MoE head
  
Phase 3: Auxiliary components (Hafta 9)
  ├─ kNN embedding index (FAISS)
  ├─ Confusion pair binary MLPs (from top-50 confusion matrix)
  ├─ Tablet GNN fine-tune (on ensemble features)
  └─ Period MoE gate calibration

Phase 4: LM training (Hafta 9-10)
  ├─ TLHdig Hittite 5-gram KenLM
  ├─ ByT5 transliteration fine-tune
  └─ Viterbi rescorer

Phase 5: End-to-end tuning (Hafta 11)
  ├─ Ensemble weight optimization (Bayesian on val fold)
  ├─ kNN λ, T3A λ, LM λ tune
  ├─ Rejection threshold calibration (target 90% coverage)
  └─ VLM trigger threshold

Phase 6: Evaluation (Hafta 12)
  └─ Test fold 4 (LOCKED) — single run, report all metrics
```

## Inference pipeline (10 aşama)

```
Image
  │
  ▼
[1] Preprocessing (v1 pipeline)
  │   CLAHE + gamma + letterbox 224 + norm
  ▼
[2] Multi-backbone ensemble forward
  │   3 backbone × 4 scale × 5 augment = 60 forward pass
  │   Weighted softmax average (0.5/0.3/0.2)
  ▼
[3] EMA+SWA averaged weights applied
  │
  ▼
[4] kNN retrieval fusion
  │   FAISS top-50 from 70K curated pool
  │   logits = 0.7*ensemble + 0.3*knn_votes
  ▼
[5] T3A template adjustment
  │   Test batch templates update, cosine classify
  │   logits = 0.8*prev + 0.2*t3a_sim
  ▼
[6] Confusion pair binary override
  │   Top-1/Top-2 confusion pair ise, pair MLP karar verir
  ▼
[7] Period MoE gating
  │   Feature → period probability → expert-weighted combine
  ▼
[8] Tablet GNN context (if batch=tablet)
  │   GAT ile aynı tablet içi sign'lar arası message passing
  ▼
[9] KenLM sequence-level rescoring
  │   Lattice top-5, Viterbi with 5-gram Hittite LM
  ▼
[10] VLM rerank (opt, uncertain only)
  │   max_prob<0.6 → Qwen2.5-VL tiebreaker
  ▼
[11] Rejection @τ=0.5
  │   Final < 0.5 → REJECT, else predict
  ▼
Output: (label, confidence, optional UNK)
```

## Gain projection (198-class Hitit gold — dürüst tahmin)

```
v3 baseline (DINOv3-L Hitit-only CE):              82.0%
+ Label smoothing + EMA + SWA:             +1.5%  →83.5%
+ Multi-scale TTA:                         +3.0%  →86.5%
+ ConvNeXt-V2 + SigLIP2 ensemble:          +3.0%  →89.5%
+ Label cleanlab:                          +1.0%  →90.5% ⭐
+ kNN retrieval fusion:                    +1.0%  →91.5%
+ T3A test-time adaptation:                +0.7%  →92.2%
+ Confusion pair disambiguation (top-50):  +0.8%  →93.0%
+ SupCon auxiliary head:                   +0.5%  →93.5%
+ SAM optimizer:                           +0.3%  →93.8%
+ Period MoE (OH/MH/NH):                   +0.4%  →94.2%
+ KenLM LM rescoring (sequence):           +1.5%  →95.7% (tablet-level only)
+ VLM rerank (uncertain):                  +0.3%  →94.5% (single sign)
+ Rejection (selective acc @ 90% cov):     →96.5% selective
```

**Final realistik tahmin**:
- **Single-sign top-1**: **%93-95** (kombine gain, overlap sonrası)
- **Tablet-level (LM)**: **%95-97**
- **Selective @ 90% coverage**: **%96-97**
- **Macro-F1 (tier-stratified)**: ~90%

## Kaynak ihtiyacı

| Phase | GPU-h |
|---|---|
| Phase 1 (SSL) | 700 |
| Phase 2 (3 backbone) | 900 |
| Phase 3 (aux) | 150 |
| Phase 4 (LM + ByT5) | 150 |
| Phase 5-6 (tune + eval) | 200 |
| **Total** | **~2,100** |

v3'ten +20% maliyet, +3-4% accuracy gain.

## Diminishing returns — dürüst uyarı

Literatürdeki cuneiform OCR çalışmaları:
- DeepScribe (141 sınıf Elamite): top-1 %77
- Stötzner 2023 (141 sınıf): top-1 %85.6
- ICDAR 2023 winner (72 sınıf): top-1 %91.4
- Bizim v3 (198 sınıf Hitit): top-1 hedef **%90+**
- Bizim v4 (+all enhancements): top-1 hedef **%93-95**

**%95+ için literatür-dışı yollar gerekli:**
- Expert paleographer annotation döngüsü (her yanlış için feedback)
- Tablet-level ensemble (sign-level değil)
- Yeni Hitit tablet fotoğrafı toplama (şu an 307 tablet)

Literatür teknikleriyle **yapısal tavan ~%95** görünüyor. Bunu aşmak için domain-specific knowledge engineering kritik.

## Enhancement kullanım kontrolü

Her enhancement opt-in/opt-out:

```yaml
# hitit_ocr/configs/enhancements_v4.yaml
enabled:
  - knn_retrieval               # (+1.0%)
  - t3a_adaptation              # (+0.7%)
  - confusion_pair_override     # (+0.8%)
  - supcon_auxiliary            # (+0.5%)
  - sam_optimizer               # (+0.3%)
  - period_moe                  # (+0.4%)
  - curriculum_learning         # (+0.3%)
  - siglip2_text_align          # (+0.5%)
  - hw_optimizations            # (speed only)
  
disabled_by_default:
  - vlm_rerank                  # API cost, only if uncertain
  - tablet_gnn                  # Needs batch=tablet (inference-time only)
  - diffusion_aug               # Risky, expert review gerekir
  - hyperbolic_head             # Marjinal, sadece hierarchy metrikleri varsa
  - laplace_uncertainty         # Rejection improvement only

conditional:
  - position_priors:            # transliterasyon-bazlı işlemlerde
      use: if_stage_4
  - continual_ssl_full:
      use: if_before_finetune
```

## Dosya haritası

```
hitit_ocr/
├── PIPELINE.md                    # v1 baseline
├── PIPELINE_V2_90PCT.md           # v2 multi-source %90 hedef
├── PIPELINE_HITIT_ONLY.md         # v3 Hitit-only
├── PIPELINE_V4.md                 # BU (v4 all enhancements)
├── PIPELINE_V4_SUMMARY.json       # v4 structured summary
├── configs/
│   ├── classification_hitit_only.yaml     (v3)
│   ├── ensemble_hitit_only.yaml            (v3)
│   └── enhancements_v4.yaml                (YENİ, config-driven toggle)
├── src/
│   ├── preprocessing/                      (Stage 0)
│   ├── pipeline/                           (Stage 3)
│   └── enhancements/                       (YENİ — 16 modül)
│       ├── knn_retrieval.py
│       ├── supcon_auxiliary.py
│       ├── vlm_rerank.py
│       ├── t3a_adaptation.py
│       ├── continual_ssl_full.py
│       ├── diffusion_aug.py
│       ├── sam_optimizer.py
│       ├── tablet_gnn.py
│       ├── hyperbolic_head.py
│       ├── siglip2_text_align.py
│       ├── laplace_uncertainty.py
│       ├── curriculum_learning.py
│       ├── hw_optimizations.py
│       ├── confusion_pair_mlp.py
│       ├── auto_confusion_pairs.py
│       ├── period_moe.py
│       ├── position_priors.py
│       └── unified_trainer.py               # tüm eklentilerin bir arada
└── runs/
    ├── ssl_dinov3_full/                    # Phase 1
    ├── hitit_only_dinov3/                  # Phase 2a
    ├── hitit_only_convnext/                # Phase 2b
    ├── hitit_only_siglip/                  # Phase 2c
    ├── knn_store_hitit/                    # Phase 3 FAISS index
    ├── confusion_pair_mlps/                # Phase 3 binary MLPs
    ├── hitit_kenlm_5gram.binary            # Phase 4
    ├── hitit_ensemble_v4/                  # Phase 5 final
    └── eval_hitit_v4/                      # Phase 6 results
```

## Referanslar (yeni enhancements)

- kNN-LM (Khandelwal 2020); CHURRO 2024
- SupCon (Khosla 2020, arXiv:2004.11362); Decoupled CL (Yeh 2022)
- Qwen2.5-VL (Bai 2025, arXiv:2502.13923); InternVL3 (Chen 2025)
- T3A (Iwasawa NeurIPS 2021); SHOT (Liang 2020)
- SAM (Foret 2021, arXiv:2010.01412); ASAM (Kwon 2021)
- DA-Fusion (Trabucco 2023); DiffuseMix (CVPR 2024)
- GAT (Veličković 2018); LayoutGNN 2023
- Poincaré (Nickel & Kiela 2017); Hyperbolic Embeddings (Khrulkov CVPR 2020)
- SigLIP2 (Tschannen 2025, arXiv:2502.14786)
- Laplace Redux (Daxberger NeurIPS 2021)
- Curriculum (Bengio 2009); SuperLoss (Castells 2020)
- Shazeer MoE 2017
- Position-aware Cuneiform (Gordin 2020)
- FlashAttention-3 (Dao 2024); Liger Kernel (2024)