sft-dedhi PEFT: 0.959 seed-777 recall, 0.95 rec/prec, HIGH 0.969

README.md

@@ -17,8 +17,8 @@ PEFT LoRA adapter for **policy-issuance QC**: two page images in (underwriting b
 
 - **Base:** [`Qwen/Qwen3-VL-8B-Instruct`](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct) (Apache-2.0)
 - **Adapter:** QLoRA r=32, α=64, dropout 0.05, 4-bit train, 384px thumbs, thinking off
-- **Checkpoint name in this project:** `/checkpoints/sft-limithi/final` (2026-08-21)
-- **Code + scorer:** [caiotheodoro/suture](https://github.com/caiotheodoro/suture) (`9e9e476` and later)
+- **Checkpoint name in this project:** `/checkpoints/sft-dedhi/final` (2026-08-22)
+- **Code + scorer:** [caiotheodoro/suture](https://github.com/caiotheodoro/suture)
 - **Do not use this for live carrier documents.** Train and eval data are forge-rendered ACORD-style pages, not real submissions.
 
 ## Held-out result (the number)
@@ -27,21 +27,19 @@ Seed-777, n=1000, **split-line** limit/deductible render (eval re-renders from s
 
 | Model | Severity-w. recall | HIGH recall | Precision | Parse |
 |---|---:|---:|---:|---:|
-| **this adapter (`sft-limithi`)** | **0.839** | **0.893** | **0.870** | **1.0** |
-| prior published (`sft-restem` on same render) | 0.634 | 0.795 | 0.704 | 1.0 |
+| **this adapter (`sft-dedhi`)** | **0.959** | **0.969** | **0.956** | **1.0** |
+| prior published (`sft-limithi`) | 0.839 | 0.893 | 0.870 | 1.0 |
 | GPT-5.6 Luna zero-shot vision | 0.373 | 0.388 | 0.344 | 0.972 |
 | 8B base, no adapter | 0.098 | 0.115 | 0.278 | 0.973 |
 
-Contract in `CONTRACTS.md` / README is still **>0.95 recall, >0.95 precision, 1.00 HIGH**. This release is a satisfactory baseline that **proves numeric reading**, not the product bar.
+Contract in `CONTRACTS.md` / README is **>0.95 recall, >0.95 precision, 1.00 HIGH**. This release **clears 0.95 recall and 0.95 precision**. HIGH is **0.969, not 1.00**.
 
-Weak classes on 777: `DEDUCTIBLE_LOWER` 0.494, `DEDUCTIBLE_HIGHER` 0.507 (precision 0.407), `LIMIT_HIGHER` 0.716. Citation exact-match ≈ 1.0 is **gold-string cloning**, not a quality win. Full tables: [`docs/BENCHMARK.md`](https://github.com/caiotheodoro/suture/blob/main/docs/BENCHMARK.md).
-
-Older restem 777 **0.622** used a cramped one-line `limit X / ded Y` PNG. Compare restem-on-new-render (0.634), not that row, when judging this adapter.
+Deductible direction is no longer the hole: `DEDUCTIBLE_HIGHER` 0.957 / 0.846 (was 0.507 / 0.407 on limithi), `DEDUCTIBLE_LOWER` 0.989. Remaining HIGH misses are mostly `AI_MISSING` 0.912 and `FORM_MISSING` 0.944. Citation exact-match ≈ 1.0 is **gold-string cloning**, not a quality win. Full tables: [`docs/BENCHMARK.md`](https://github.com/caiotheodoro/suture/blob/main/docs/BENCHMARK.md).
 
 ## Intended use
 
 - Research / reproduction of the Suture benchmark (binder vs policy discrepancy listing).
-- Starting point for further SFT on the same forge generator (next: deductible direction / FPs).
+- Starting point for further SFT on the same forge generator (next: HIGH = 1.00).
 
 Out of scope: production underwriting, real PDFs, OCR pipelines, “beats frontier on insurance” marketing.
 
@@ -63,18 +61,18 @@ Two images (binder page, then policy page). Output schema is in the repo `CONTRA
 
 ## Training data
 
-Synthetic only. Generator + oracle: `suture_forge` in the GitHub repo. Seeds used in the published chain: 7 (train/val), 11 (distill pool), 13 (numeric mix), 17 (LIMIT_HIGHER mix). **Seed 777 is eval-only and was never in train.** JSONL is gitignored on purpose; regenerate from seeds.
+Synthetic only. Generator + oracle: `suture_forge` in the GitHub repo. Seeds used in the published chain: 7 (train/val), 11 (distill pool), 13 (numeric mix), 17 (LIMIT_HIGHER mix), 19 (DED contrastive), 23 (DED_HIGHER boost). **Seed 777 is eval-only and was never in train.** JSONL is gitignored on purpose; regenerate from seeds.
 
-Chain: `sft-vl` → `sft-distill` → `sft-restem` → `sft-numeric` → **`sft-limithi`**. Continue-from-adapter SFT, 1 epoch per mix, Modal L4.
+Chain: `sft-vl` → `sft-distill` → `sft-restem` → `sft-numeric` → `sft-limithi` → `sft-ded` → **`sft-dedhi`**. Continue-from-adapter SFT, 1 epoch per mix, Modal L4.
 
 This Hub repo does **not** include training JSONL or a merged 16-bit 8B.
 
 ## Limitations
 
-- 8B QLoRA at 384px on synthetic pages. HIGH recall is 0.893, not 1.00.
-- Deductible higher/lower is still ~coin-flip on 777; `DEDUCTIBLE_HIGHER` false positives drag precision.
+- 8B QLoRA at 384px on synthetic pages. HIGH recall is 0.969, not 1.00.
+- `AI_MISSING` and `FORM_MISSING` are the remaining HIGH misses.
 - Luna (0.37) is a weaker *type* teacher than gold JSON; this adapter lists gold types, it does not copy Luna style.
 
 ## Carbon / hardware
 
-Modal L4 24GB, 4-bit QLoRA. This adapter: continue `sft-numeric` → `sft-limithi`, 680 steps. Earlier chain steps are documented in `docs/DECISIONS.md`.
+Modal L4 24GB, 4-bit QLoRA. This adapter: continue `sft-ded` → `sft-dedhi`, 1280 steps. Earlier chain steps are documented in `docs/DECISIONS.md`.

adapter_config.json

@@ -31,17 +31,17 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "down_proj",
     "o_proj",
-    "linear_fc2",
-    "k_proj",
     "attn.proj",
+    "qkv",
+    "linear_fc1",
     "q_proj",
-    "up_proj",
     "v_proj",
-    "linear_fc1",
     "gate_proj",
-    "qkv",
-    "down_proj"
+    "k_proj",
+    "linear_fc2",
+    "up_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0b10e31e883bc49f2e40952b2ef5db2d961e5515c562c1ce69ebc52072e2d709
+oid sha256:4aa5d63c7726f8f213de7343f602efac036881d90fd6a01f083564cdc94fa218
 size 210083584