Add all paper figures to model card with arXiv:2606.22546

README.md

@@ -17,6 +17,7 @@ library_name: pytorch
 datasets:
 - refcoco
 - refcocog
+arxiv: 2606.22546
 ---
 
 # Venice-H1: Failure-Aware Query Re-Ranking for Referring Image Segmentation
@@ -26,13 +27,42 @@ datasets:
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
 **Nicolò Savioli, Ph.D.** — OdaxAI Research  
-nicolo.savioli@odaxai.com
+nicolo.savioli@odaxai.com · [odaxai.com](https://odaxai.com)
+
+---
+
+## Architecture Overview
+
+![Architecture Overview](figures/architecture_overview.png)
+
+*Venice-H1 pipeline. A frozen DeRIS backbone generates N=10 candidate masks. Multi-scale grid signatures encode spatial quality. The Failure Re-Ranker gates intervention: it only overrides Query-0 when confident the default choice is wrong.*
+
+---
+
+## The Failure-Case Bottleneck
+
+| | |
+|---|---|
+| ![Error Budget](figures/error_budget.png) | ![IoU Scatter](figures/iou_scatter_analysis.png) |
+| *7–18% of samples generate 40–68% of total error* | *Failure cases form a "triangle of opportunity"* |
+
+---
+
+## Multi-Scale Grid Signatures
+
+![Grid Signatures](figures/grid_signature_vis.png)
+
+*Compact 675-dim spatial descriptors pooled at 4×4, 8×8, 16×16 grids per candidate mask.*
+
+![Grid Cells](figures/grid_cells_detail_v2.png)
+
+*Multi-scale grid cells inspired by entorhinal cortex representations.*
 
 ---
 
 ## Model Description
 
-Venice-H1 is a lightweight, backbone-decoupled re-ranking module for Referring Image Segmentation (RIS). Operating on top of a frozen DeRIS-L backbone, it detects when the default query selection fails and selects a better alternative using:
+Venice-H1 is a lightweight, backbone-decoupled re-ranking module for Referring Image Segmentation (RIS). It detects when the default query selection fails and selects a better alternative using:
 
 - **Multi-Scale Grid Signatures**: 4×4, 8×8, 16×16 spatial pooling → 675-dim descriptors
 - **Failure Gate**: binary classifier predicting whether Query 0 is suboptimal
@@ -44,37 +74,74 @@ Venice-H1 is a lightweight, backbone-decoupled re-ranking module for Referring I
 
 ---
 
-## External Dependencies
-
-Venice-H1 operates as a post-hoc re-ranker on top of:
+## Results
 
-| Component | Model | Paper |
-|-----------|-------|-------|
-| Backbone | DeRIS-L | Dai et al. (2025) |
-| Visual Encoder | Swin-Large | Liu et al. (2021) |
-| Language Encoder | BEiT-3 | Wang et al. (2023) |
-| Mask Generator | Mask2Former | Cheng et al. (2022) |
+### On failure cases (where Venice-H1 intervenes)
 
-Venice-H1 does **not** include these weights. You need a running DeRIS-L instance to extract features.
+![Per Split Improvement](figures/per_split_improvement.png)
 
----
-
-## Checkpoint: `venice_h1_deris_l.pt`
-
-Trained on RefCOCO/RefCOCO+/RefCOCOg using DeRIS-L features. Evaluated on RefCOCO val split.
+*Positive Δ across all 8 evaluation splits.*
 
 | Metric | Value |
 |--------|-------|
 | Parameters | **11,296,258** |
-| Backbone | DeRIS-L |
-| Epoch | 15 |
-| τ (threshold) | 0.90 |
 | **Δ_fail (mIoU on failures)** | **+1.824** |
 | **AUC (failure detection)** | **0.778** |
 | **Δ_full (overall mIoU)** | **+0.039** |
 | Q0 mIoU | 86.469 |
 | Selected mIoU | 86.509 |
 | Oracle mIoU | 89.691 |
+| Harmful-switch rate | < 0.6% |
+
+### Failure Gate Analysis
+
+| | |
+|---|---|
+| ![ROC Curves](figures/roc_curves.png) | ![Coverage Risk](figures/coverage_risk.png) |
+| *ROC curves across splits. AUC 0.78–0.82* | *Coverage-risk trade-off at different τ* |
+
+---
+
+## Qualitative Results
+
+![Qualitative Examples](figures/qualitative_examples.jpg)
+
+*Re-ranking on RefCOCO val. Each row: input, ground truth, default query (red, fails), Venice-H1 corrected selection (blue). Venice-H1 recovers IoU > 84% in all cases.*
+
+---
+
+## Ablation Study
+
+![Ablation Study](figures/ablation_study.png)
+
+| Configuration | Δ_fail | Gate AUC |
+|---|---|---|
+| BASE only (no grid) | +1.01 | 0.812 |
+| 4×4 only | +1.01 | 0.821 |
+| 8×8 only | +0.87 | 0.790 |
+| 16×16 only | +1.00 | 0.828 |
+| **BASE + all grids (ours)** | **+1.22** | **0.807** |
+
+---
+
+## Medical Cross-Domain Transfer
+
+![Medical Transfer](figures/medical_cross_domain.png)
+
+*Zero-shot transfer to MS-CXR (+1.16 mIoU) and M3D-RefSeg-2D (+0.51 mIoU) without fine-tuning.*
+
+---
+
+## External Dependencies
+
+| Component | Model | Paper |
+|-----------|-------|-------|
+| Backbone | DeRIS-L | Dai et al. (2025) |
+| Visual Encoder | Swin-Large | Liu et al. (2021) |
+| Language Encoder | BEiT-3 | Wang et al. (2023) |
+| Mask Generator | Mask2Former | Cheng et al. (2022) |
+
+Venice-H1 does **not** include these weights. You need a running DeRIS-L instance to extract features.
 
 ---
 
@@ -84,7 +151,6 @@ Trained on RefCOCO/RefCOCO+/RefCOCOg using DeRIS-L features. Evaluated on RefCOC
 import torch
 from huggingface_hub import hf_hub_download
 
-# Download checkpoint
 ckpt_path = hf_hub_download(repo_id="OdaxAI/venice-h1", filename="venice_h1_deris_l.pt")
 ckpt = torch.load(ckpt_path, map_location="cpu", weights_only=False)
 
@@ -93,51 +159,22 @@ print("Metrics:", ckpt["metrics"])
 print("Parameters:", sum(v.numel() for v in ckpt["model"].values() if hasattr(v, "numel")))
 ```
 
-**Reproduce paper results** (no dataset needed — verifies architecture + metrics):
+**Reproduce paper results** (no dataset needed):
 
 ```bash
 git clone https://github.com/odaxai/Venice-H1.git
-cd Venice-H1 && pip install -r requirements.txt
+cd Venice-H1 && pip install -r requirements.txt && pip install -e .
 python reproduce_results.py --verify_only
 ```
 
-Expected output:
-```
-── Architecture Verification ──────────────────────────────
-  Parameters     : 11,296,258  ✓ MATCH
-
-── Paper Cross-Check (RefCOCO val) ─────────────────────────
-  ✓ delta_fail   : 1.8244  (paper: 1.824)
-  ✓ auc_fail     : 0.7776  (paper: 0.778)
-  ✓ delta_full   : 0.0392  (paper: 0.039)
 ```
+── Architecture Verification ───────────────────
+  Parameters : 11,296,258  ✓ MATCH
 
-**Full inference pipeline** (requires DeRIS-L features — see [GitHub README](https://github.com/odaxai/Venice-H1)):
-
-```python
-from venice_h1.model.reranker import VeniceH1Reranker
-import torch
-from huggingface_hub import hf_hub_download
-
-ckpt_path = hf_hub_download(repo_id="OdaxAI/venice-h1", filename="venice_h1_deris_l.pt")
-ckpt = torch.load(ckpt_path, map_location="cpu", weights_only=False)
-cfg  = ckpt["config"]
-
-model = VeniceH1Reranker(
-    query_feat_dim=cfg["query_feat_dim"],
-    hidden_dim=cfg["hidden_dim"],
-    n_layers=cfg["n_layers"],
-    n_heads=cfg["n_heads"],
-    tau=cfg["tau"],
-)
-model.load_state_dict(ckpt["model"], strict=False)
-model.eval()
-
-# features: (B, N=10, 936) — from scripts/extract_features.py
-with torch.no_grad():
-    out      = model(features, det_scores, mask_means)
-    p_fail   = out["p_fail"]    # (B,) failure probability
-    selected = model.rerank(features, det_scores, mask_means)  # (B,) best query
+── Paper Cross-Check (RefCOCO val) ─────────────
+  ✓ delta_fail : 1.8244  (paper: 1.824)
+  ✓ auc_fail   : 0.7776  (paper: 0.778)
+  ✓ delta_full : 0.0392  (paper: 0.039)
 ```
 
 ---