Trim example to 2 iterations with per-iter think blocks

README.md

@@ -17,13 +17,25 @@ tags:
 
 # DecomposeRL-7B
 
+<p align="center">
+  <a href="https://arxiv.org/abs/0000.00000">
+    <img src="https://img.shields.io/badge/%F0%9F%93%84_Paper-Coming_Soon-b12a00?style=for-the-badge&labelColor=ffb300" alt="Paper Coming Soon">
+  </a>
+</p>
+
+[![Paper](https://img.shields.io/badge/arXiv-coming--soon-red)](https://arxiv.org/abs/0000.00000)
+[![Project Page](https://img.shields.io/badge/Project-Page-green)](https://dipta007.github.io/DecomposeRL/)
+[![Dataset](https://img.shields.io/badge/HuggingFace-Dataset-yellow)](https://huggingface.co/datasets/dipta007/decomposeRL)
+[![Models](https://img.shields.io/badge/HuggingFace-Models-orange)](https://huggingface.co/collections/dipta007/decomposerl)
+[![GitHub](https://img.shields.io/badge/GitHub-Code-blue)](https://github.com/dipta007/DecomposeRL)
+
 **DecomposeRL-7B** is a fact-verification model that learns to *decompose* a claim into atomic sub-questions, iteratively answer them from an evidence document, and produce a final `Supported` / `Refuted` judgment. It is trained from `Qwen2.5-7B-Instruct` with **GRPO + LoRA** under a stack of **seven complementary rewards** that shape the reward landscape around three axes: structural correctness, per-question quality, and set-level sufficiency.
 
 ## Highlights
 
 - **84.5% micro-average balanced accuracy** across 9 in-domain claim-verification benchmarks (sample-weighted)
 - **84.6% macro-average balanced accuracy** across the same 9 benchmarks
-- **60.2% balanced accuracy on Coverbench** (out-of-domain, long-evidence)
+- Out-of-domain: **60.2% balanced accuracy on Coverbench**, **77.0% on LLM-AggreFact**
 - Strong on long-form evidence: 87% on Ex-FEVER, 92% on FEVEROUS, 76% on HoVer
 - Reasoning is **fully transparent** — the model emits its sub-claim checklist, every question it asked, every quote from evidence, and a final label
 
@@ -50,30 +62,26 @@ DecomposeRL trains the policy to follow a **decompose-question-answer-verify** l
 
 ### Reward Stack — seven complementary signals
 
-GRPO is supervised with a sum of seven rewards $R(\tau) = \sum_k R_k(\tau)$, grouped into three families:
+GRPO is supervised with a sum of seven rewards, grouped into three families:
 
 **Programmatic anchors** (no judge call)
 
-1. **Format** ($R_\text{fmt}$) — fraction of structural conditions satisfied: well-formed XML, `<question>`→`<answer>` alternation, valid final verification label. Treated as a gating prerequisite.
-2. **Question count** ($R_\text{qc}$) — triangular kernel on $r = n/n^\star$ (predicted vs silver decomposition length): $\max(0, 1-|r-1|)$, peaking at $r=1$, vanishing for $r\ge 2$.
-3. **Diversity** ($R_\text{div}$) — MMR-style penalty for redundancy across $\{q_1,\dots,q_n\}$, computed over `Qwen3-Embedding-8B` embeddings as $-\tfrac{1}{n}\sum_{i=2}^{n}\max_{j<i}\cos(q_i, q_j)$.
+1. **Format** — ensures the trace is parseable; a gating prerequisite without which no other reward can be computed.
+2. **Question count** — discourages collapsing the decomposition into one mega-question or padding it with filler.
+3. **Diversity** — penalizes redundant questions so the policy covers distinct sub-claims instead of rewording the same one.
 
 **Set-level signals**
 
-4. **Coverage** ($R_\text{cov}$) — judge predicts the verdict from the answers alone (without the evidence document) and is rewarded for matching the gold label. Cleanest test of whether the decomposition is *collectively sufficient*.
-5. **Verification** ($R_\text{ver}$) — sparse but unambiguous outcome anchor: does the final `<verification>` label match the gold label?
+4. **Coverage** — checks whether the verdict can be recovered from the answers alone; tests if the decomposition is *collectively sufficient*.
+5. **Verification** — direct outcome anchor: did the final label match the gold label?
 
 **Leave-one-out and per-question composites**
 
-6. **Necessity** ($R_\text{nec}$) — for each $q_i$, run the coverage judge with and without it. Score on a four-state matrix (necessary $+1$ / redundant $+\tfrac{1}{2}$ / neutral $0$ / harmful $-1$) and aggregate via the minimum. The harmful $-1$ case is the only negative signal in the stack — it lets the policy *remove* misleading questions.
-7. **Joint multiplicative quality** ($R_\text{joint}$) — three judge-based sub-signals composed multiplicatively per question:
-   - **(7a) Answerability** ($R_\text{ans}^{(i)}$) — is $q_i$ answerable from the evidence alone?
-   - **(7b) Atomicity** ($R_\text{atom}^{(i)}$) — five-criterion checklist: a real question, single-focus, no compound conjunctions, verifiable, claim-grounded.
-   - **(7c) Answer correctness** ($R_\text{corr}^{(i)}$) — is $a_i$ faithful to the document (no contradictions, no extrinsic info)?
-
-  $$R_\text{joint} = \tfrac{1}{n}\sum_{i=1}^{n} R_\text{ans}^{(i)} \cdot R_\text{atom}^{(i)} \cdot R_\text{corr}^{(i)}$$
-
-  A failure on any single axis drives that question's term to zero. For honest abstentions (`a_i = "I don't know"`) the undefined $R_\text{corr}$ factor is dropped so a calibrated abstention is rewarded by question quality alone.
+6. **Necessity (leave-one-out)** — the only signal that can push the policy to *remove* misleading questions; a question is necessary iff its removal would change the verdict.
+7. **Joint multiplicative quality** — composes three per-question sub-signals so a question must clear *all* of them simultaneously rather than scoring partial credit:
+   - **(7a) Answerability** — is the question answerable from the evidence?
+   - **(7b) Atomicity** — is it a single-focus, verifiable question grounded in the claim?
+   - **(7c) Answer correctness** — is the answer faithful to the document (no contradictions, no extrinsic info)?
 
 ## Quickstart
 
@@ -132,8 +140,6 @@ response = tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tok
 print(response)
 ```
 
-The full training-time prompt template (with extended instructions, a worked example, and sub-claim classification guidance) lives in `decomposer/prompts.py` of the source repo and is what gives the strongest performance.
-
 ### Pretty-print the trace
 
 The model produces an iterative `<think>` / `<question>` / `<answer>` / `<verification>` trace. The helper below parses it into a structured form and prints it as a readable conversation:
@@ -170,7 +176,6 @@ def pretty_print(text: str) -> None:
             print(f"✅  VERIFICATION: {body}")
             print("=" * 78)
 
-# usage:
 pretty_print(response)
 
 # extract just the final label:
@@ -179,29 +184,44 @@ label = label_match.group(1) if label_match else None
 print("Label:", label)
 ```
 
-**Example output:**
+**Example output on a real trace** (claim: *"New reports show Kevin Nicholson made over \$1 million advising companies that shut down Wisconsin plants and laid off 1,900 people."*):
 
 ```
 ──────────────────────────────────────────────────────────────────────────────
 🧠  THINK
 ──────────────────────────────────────────────────────────────────────────────
-The claim has two atomic sub-claims:
-  1. The Eiffel Tower was completed in 1887 (temporal, independently falsifiable)
-  2. The Eiffel Tower stands 330 metres tall (quantitative)
+The claim can be broken down into several sub-claims:
+1. Nicholson made over $1 million
+2. The companies were in Wisconsin
+3. The companies shut down plants and laid off 1,900 people
+4. Nicholson's work is causally tied to those layoffs and shutdowns
 I will verify each in turn against the evidence document.
 
-🔸  Q1: When was the Eiffel Tower completed?
-💬  A1: The evidence states the tower was built "from 1887 to 1889", so it was
-        completed in 1889, not 1887.
+🔸  Q1: What does the evidence document say about Nicholson's earnings?
+💬  A1: The evidence document states that Nicholson received more than $1 million
+       for his consulting work between January 2016 and November 2017. However,
+       it does not say how much he was paid for consulting for the companies
+       that did layoffs and shutdowns.
 
-🔸  Q2: What is the height of the Eiffel Tower?
-💬  A2: The evidence states "330 metres (1,083 ft) tall." — 330 m confirmed.
+──────────────────────────────────────────────────────────────────────────────
+🧠  THINK
+──────────────────────────────────────────────────────────────────────────────
+Sub-claim 1 ($1 million) is partially supported, but the amount is not tied to
+the specific companies. Next, verify the companies and the layoffs.
+
+🔸  Q2: What does the evidence document say about the companies and layoffs?
+💬  A2: The evidence document states that Nicholson worked as a consultant for
+       companies that laid off nearly 1,900 people since 2015, shutting down
+       plants in Wisconsin and other states. But it also says Baldwin cites no
+       evidence that Nicholson's work caused the layoffs and shutdowns — only
+       some element of truth, our definition of Mostly False.
 
 ──────────────────────────────────────────────────────────────────────────────
 🧠  THINK
 ──────────────────────────────────────────────────────────────────────────────
-Sub-claim 1 is refuted (1889 ≠ 1887). Since it is independently falsifiable,
-the overall claim is refuted regardless of sub-claim 2.
+The causal link between Nicholson's consulting and the layoffs is unsupported.
+The document explicitly rates the claim Mostly False, so the overall claim is
+refuted.
 
 ==============================================================================
 ✅  VERIFICATION: Refuted
@@ -234,13 +254,12 @@ The `--max-model-len` matches the training-time `max_seq_length=16016` (with `ma
 | **Micro-average** (sample-weighted) | 14,503 | **0.8445** |
 | **Macro-average** | — | **0.8417** |
 
-### Out-of-domain — Coverbench
+### Out-of-domain
 
 | Dataset | # Examples | Balanced Acc | Accuracy | F1 |
 |---|---:|---:|---:|---:|
-| Coverbench (long-evidence, OOD) | 728 | **0.6021** | 0.5989 | 0.6086 |
-
-Coverbench evaluates claims paired with substantially longer evidence than the training distribution, so it is a stress test of how well the decomposition behavior transfers when the document grows.
+| Coverbench | 728 | **0.6021** | 0.5989 | 0.6086 |
+| LLM-AggreFact | 29,320 | **0.7695** | 0.8510 | 0.9054 |
 
 ## Intended Use
 
@@ -249,6 +268,12 @@ Coverbench evaluates claims paired with substantially longer evidence than the t
 
 The model is trained to say *"I don't know"* when the evidence document is silent — please respect that signal in downstream systems instead of forcing a label.
 
+## Citation
+
+```bibtex
+
+```
+
 ## License
 
 Released under the Apache 2.0 License.