saademad/medico2026-gutcheck

CVG-IBA_MMRG — MediaEval Medico 2026 (Subtasks 1 & 2)

A training-free reliability layer around a frozen microsoft/Florence-2-base + LoRA adapter (peeache/FL2_VQA_MIX_128_256). No backbone fine-tuning was used for this submission.

Subtask 1 — VQA (submission_task1.py)

• Backbone: Florence-2-base + the LoRA adapter, prompt "<MedVQA> {question}", beam search (3 beams).
• We emit the raw verbose generation (no concise normalizer): Kvasir-VQA-x1 answers are verbose natural language, so raw output is the correct target for BLEU/ROUGE/METEOR.
• Generation is per-example (Florence-2 merges image tokens into the encoder, so a padded text-only attention mask is mismatched in batched generation).

Subtask 2 — Explainable & safe reasoning (submission_task2.jsonl, visuals/)

Each val_id carries:

• answer — the same verbose <MedVQA> prediction as Subtask 1.
• textual_explanation — a self-probed, faithful clinician-style report. For a present localizable finding we query the model's own attribute questions (location / colour / size / morphology) and synthesize them with tentative phrasing, gated to exclude negatives / instruments / landmarks (no hallucinated findings by construction).
• visual_explanation — a <REFERRING_EXPRESSION_SEGMENTATION> overlay for localizable entities (official polygon decoding → binary mask at visuals/{val_id}_{img_id}_mask.png). 230 cases grounded.
• confidence_score — a calibrated probability from a per-(aspect, answer) reliability table fit on a held-out, image-disjoint slice (so val confidence is never in-sample).

Why these choices (results we report in the working notes)

• Confidence calibration: per-(aspect, answer) reliability predicts answer correctness at AUROC ≈ 0.87, far above the model's own generative confidence (≈ 0.53) and MSP (≈ 0.60).
• Risk-controlled abstention: a distribution-free Clopper–Pearson conformal rule guarantees asserted-answer error ≤ α (e.g. ~55% coverage at a verified ≤ 5% error), unlike a fixed threshold.
• Faithful explanation: measured by three independent metrics (structured claim-support, NLI entailment, LLM-judge). Faithful-by-construction templates dominate LLM narration; a lexical faithfulness gate is insufficient (it passes ~95% of rewrites that still carry semantic drift).
• Honest negative results ("base-rate dominance"): instance-level uncertainty, visual retrieval, and segmentation-grounding all fail to predict correctness on the frozen VLM — correctness is governed by output-conditioned base rates, which is why the calibrated reliability layer is the right, safe design.

Post-processing

• Segmentation masks: official Florence-2 polygon post-processing → filled binary PNG (no extra morphology; fill-holes / largest-CC were evaluated and found to be no-ops on these clean polygons).
• Confidence → safety policy: assert ≥ 0.80, hedge 0.50–0.80, abstain < 0.50 (interpretable because the score is a true probability).