Update model card: 7 rewards, OOD coverbench, pretty-print helper, fix max-len

README.md

@@ -17,12 +17,13 @@ tags:
 
 # DecomposeRL-7B
 
-**DecomposeRL-7B** is a fact-verification model that learns to decompose claims into atomic sub-questions, iteratively answer them from an evidence document, and produce a final `Supported` / `Refuted` judgment. It is trained from `Qwen2.5-7B-Instruct` with **GRPO + LoRA** using rewards over format validity, sub-claim coverage, answer necessity, and question diversity.
+**DecomposeRL-7B** is a fact-verification model that learns to *decompose* a claim into atomic sub-questions, iteratively answer them from an evidence document, and produce a final `Supported` / `Refuted` judgment. It is trained from `Qwen2.5-7B-Instruct` with **GRPO + LoRA** under a stack of **seven complementary rewards** that shape the reward landscape around three axes: structural correctness, per-question quality, and set-level sufficiency.
 
 ## Highlights
 
 - **84.5% micro-average balanced accuracy** across 9 in-domain claim-verification benchmarks (sample-weighted)
 - **84.6% macro-average balanced accuracy** across the same 9 benchmarks
+- **60.2% balanced accuracy on Coverbench** (out-of-domain, long-evidence)
 - Strong on long-form evidence: 87% on Ex-FEVER, 92% on FEVEROUS, 76% on HoVer
 - Reasoning is **fully transparent** — the model emits its sub-claim checklist, every question it asked, every quote from evidence, and a final label
 
@@ -35,7 +36,7 @@ tags:
 | **Parameters** | 7B |
 | **Training** | GRPO + LoRA (r=64, α=128) |
 | **LoRA Targets** | q, k, v, o, gate, up, down projections |
-| **Context Length** | 4,096 tokens |
+| **Max Sequence Length** | 16,016 tokens (training-time) |
 | **Language** | English |
 
 ## Method
@@ -47,15 +48,32 @@ DecomposeRL trains the policy to follow a **decompose-question-answer-verify** l
 3. **Sufficiency check** (`<think>`): track which sub-claims are resolved; loop until every sub-claim is addressed.
 4. **Final verdict** (`<verification>`): `Supported` or `Refuted`.
 
-### Training Rewards
+### Reward Stack — seven complementary signals
 
-GRPO is supervised with a composite reward over generated trajectories:
+GRPO is supervised with a sum of seven rewards $R(\tau) = \sum_k R_k(\tau)$, grouped into three families:
 
-- **Format reward** — well-formed `<think>`/`<question>`/`<answer>`/`<verification>` structure
-- **Verification reward** — correct final label
-- **Necessity reward** — generated sub-questions are necessary to verify the claim
-- **Diversity reward** — sub-questions cover distinct aspects (MMR-based)
-- **Coverage reward** — sub-questions jointly cover the claim
+**Programmatic anchors** (no judge call)
+
+1. **Format** ($R_\text{fmt}$) — fraction of structural conditions satisfied: well-formed XML, `<question>`→`<answer>` alternation, valid final verification label. Treated as a gating prerequisite.
+2. **Question count** ($R_\text{qc}$) — triangular kernel on $r = n/n^\star$ (predicted vs silver decomposition length): $\max(0, 1-|r-1|)$, peaking at $r=1$, vanishing for $r\ge 2$.
+3. **Diversity** ($R_\text{div}$) — MMR-style penalty for redundancy across $\{q_1,\dots,q_n\}$, computed over `Qwen3-Embedding-8B` embeddings as $-\tfrac{1}{n}\sum_{i=2}^{n}\max_{j<i}\cos(q_i, q_j)$.
+
+**Set-level signals**
+
+4. **Coverage** ($R_\text{cov}$) — judge predicts the verdict from the answers alone (without the evidence document) and is rewarded for matching the gold label. Cleanest test of whether the decomposition is *collectively sufficient*.
+5. **Verification** ($R_\text{ver}$) — sparse but unambiguous outcome anchor: does the final `<verification>` label match the gold label?
+
+**Leave-one-out and per-question composites**
+
+6. **Necessity** ($R_\text{nec}$) — for each $q_i$, run the coverage judge with and without it. Score on a four-state matrix (necessary $+1$ / redundant $+\tfrac{1}{2}$ / neutral $0$ / harmful $-1$) and aggregate via the minimum. The harmful $-1$ case is the only negative signal in the stack — it lets the policy *remove* misleading questions.
+7. **Joint multiplicative quality** ($R_\text{joint}$) — three judge-based sub-signals composed multiplicatively per question:
+   - **(7a) Answerability** ($R_\text{ans}^{(i)}$) — is $q_i$ answerable from the evidence alone?
+   - **(7b) Atomicity** ($R_\text{atom}^{(i)}$) — five-criterion checklist: a real question, single-focus, no compound conjunctions, verifiable, claim-grounded.
+   - **(7c) Answer correctness** ($R_\text{corr}^{(i)}$) — is $a_i$ faithful to the document (no contradictions, no extrinsic info)?
+
+  $$R_\text{joint} = \tfrac{1}{n}\sum_{i=1}^{n} R_\text{ans}^{(i)} \cdot R_\text{atom}^{(i)} \cdot R_\text{corr}^{(i)}$$
+
+  A failure on any single axis drives that question's term to zero. For honest abstentions (`a_i = "I don't know"`) the undefined $R_\text{corr}$ factor is dropped so a calibrated abstention is rewarded by question quality alone.
 
 ## Quickstart
 
@@ -108,33 +126,99 @@ messages = [{"role": "user", "content": user_prompt}]
 text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
 inputs = tokenizer([text], return_tensors="pt").to(model.device)
 
-out = model.generate(**inputs, max_new_tokens=2048, temperature=0.7, do_sample=True)
+# max_new_tokens matches training-time max_completion_length
+out = model.generate(**inputs, max_new_tokens=4500, temperature=0.7, do_sample=True)
 response = tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
 print(response)
 ```
 
 The full training-time prompt template (with extended instructions, a worked example, and sub-claim classification guidance) lives in `decomposer/prompts.py` of the source repo and is what gives the strongest performance.
 
-### Parsing the Output
+### Pretty-print the trace
 
-The final label is between the last `<verification>` and `</verification>` tags:
+The model produces an iterative `<think>` / `<question>` / `<answer>` / `<verification>` trace. The helper below parses it into a structured form and prints it as a readable conversation:
 
 ```python
 import re
 
-match = re.search(r"<verification>\s*(Supported|Refuted)\s*</verification>", response)
-label = match.group(1) if match else None
+TAG_RE = re.compile(r"<(think|question|answer|verification)>(.*?)</\1>", re.DOTALL)
+
+def parse_trace(text: str):
+    """Return a list of (tag, content) tuples in the order they appear."""
+    return [(tag, body.strip()) for tag, body in TAG_RE.findall(text)]
+
+def pretty_print(text: str) -> None:
+    cycle_idx = 0
+    pending_q = None
+    for tag, body in parse_trace(text):
+        if tag == "think":
+            print("─" * 78)
+            print("🧠  THINK")
+            print("─" * 78)
+            print(body)
+            print()
+        elif tag == "question":
+            cycle_idx += 1
+            pending_q = body
+        elif tag == "answer":
+            print(f"🔸  Q{cycle_idx}: {pending_q}")
+            print(f"💬  A{cycle_idx}: {body}")
+            print()
+            pending_q = None
+        elif tag == "verification":
+            print("=" * 78)
+            print(f"✅  VERIFICATION: {body}")
+            print("=" * 78)
+
+# usage:
+pretty_print(response)
+
+# extract just the final label:
+label_match = re.search(r"<verification>\s*(Supported|Refuted)\s*</verification>", response)
+label = label_match.group(1) if label_match else None
+print("Label:", label)
+```
+
+**Example output:**
+
+```
+──────────────────────────────────────────────────────────────────────────────
+🧠  THINK
+──────────────────────────────────────────────────────────────────────────────
+The claim has two atomic sub-claims:
+  1. The Eiffel Tower was completed in 1887 (temporal, independently falsifiable)
+  2. The Eiffel Tower stands 330 metres tall (quantitative)
+I will verify each in turn against the evidence document.
+
+🔸  Q1: When was the Eiffel Tower completed?
+💬  A1: The evidence states the tower was built "from 1887 to 1889", so it was
+        completed in 1889, not 1887.
+
+🔸  Q2: What is the height of the Eiffel Tower?
+💬  A2: The evidence states "330 metres (1,083 ft) tall." — 330 m confirmed.
+
+──────────────────────────────────────────────────────────────────────────────
+🧠  THINK
+──────────────────────────────────────────────────────────────────────────────
+Sub-claim 1 is refuted (1889 ≠ 1887). Since it is independently falsifiable,
+the overall claim is refuted regardless of sub-claim 2.
+
+==============================================================================
+✅  VERIFICATION: Refuted
+==============================================================================
 ```
 
 ### Using vLLM
 
 ```bash
-vllm serve dipta007/decomposeRL-7b --max-model-len 4096
+vllm serve dipta007/decomposeRL-7b --max-model-len 16016
 ```
 
+The `--max-model-len` matches the training-time `max_seq_length=16016` (with `max_prompt_length=11500` + `max_completion_length=4500`).
+
 ## Performance
 
-Balanced accuracy on 9 in-domain claim-verification benchmarks (best checkpoint, `step 5400`):
+### In-domain — balanced accuracy on 9 claim-verification benchmarks
 
 | Dataset | # Examples | Balanced Acc |
 |---|---:|---:|
@@ -150,6 +234,14 @@ Balanced accuracy on 9 in-domain claim-verification benchmarks (best checkpoint,
 | **Micro-average** (sample-weighted) | 14,503 | **0.8445** |
 | **Macro-average** | — | **0.8417** |
 
+### Out-of-domain — Coverbench
+
+| Dataset | # Examples | Balanced Acc | Accuracy | F1 |
+|---|---:|---:|---:|---:|
+| Coverbench (long-evidence, OOD) | 728 | **0.6021** | 0.5989 | 0.6086 |
+
+Coverbench evaluates claims paired with substantially longer evidence than the training distribution, so it is a stress test of how well the decomposition behavior transfers when the document grows.
+
 ## Intended Use
 
 - **In-scope**: verifying factual claims against a *provided* evidence document (open-book fact verification, retrieval-augmented fact-checking pipelines).