Update README.md

README.md

@@ -18,35 +18,44 @@ model_name: TreeRPO-Qwen2.5-Math-1.5B
 
 # TreeRPO-Qwen2.5-Math-1.5B
 
-**Short summary:** A 1.5B parameter math reasoning model fine-tuned with *TreeRPO*, a hierarchical extension of GRPO that assigns rewards to “thought” nodes instead of whole sequences—achieving higher GSM8K accuracy with **~10K total** supervised + RL examples and **no reward model**.
+**Summary:**  
+A 1.5B parameter math reasoning model fine-tuned with **TreeRPO**, a hierarchical extension of GRPO that assigns rewards to “thought” nodes (not just full completions). Achieves higher GSM8K accuracy with just ~10K supervised + RL examples and **no reward model**.
 
 🔎 **Full write-up (method, math, analysis):**  
-https://omrisapir.substack.com/publish/post/167273414
+[TreeRPO: Hierarchical Credit Assignment for Data-Efficient Math Reasoning](https://omrisapir.substack.com/publish/post/167273414)
+
+---
 
 ## Model Details
-- **Base model:** `Qwen/Qwen2.5-Math-1.5B`
-- **Method:** TreeRPO
-- **Reward signal:** Deterministic exact-match checker (binary). Interior node reward = average of descendant leaf rewards.
-- **Intended domain:** Grade-school & intermediate math word problems (GSM8K style)
+- **Base model:** [`Qwen/Qwen2.5-Math-1.5B`](https://huggingface.co/Qwen/Qwen2.5-Math-1.5B)
+- **Method:** TreeRPO (tree-structured GRPO; up to depth 7; branching by entropy & length)
+- **Reward signal:** Deterministic exact-match checker (binary). Interior node rewards = mean descendant leaf rewards.
+- **Domain:** Grade-school and intermediate math word problems (GSM8K style)
 
 ## Intended Use
-Research on hierarchical RL for reasoning; math tutoring prototypes with human oversight; experimentation in deterministic pass/fail domains (e.g., potential extension to code with unit tests).
+Research on hierarchical RL for reasoning; math tutoring (with human oversight); or as a research baseline for deterministic pass/fail domains (potential to extend to code with unit tests).
 
-**Not intended for:** Open-ended unsafe dialogue, factual QA outside math, high‑stakes decision making.
- single 48GB GPU (~18h RL phase).
+**Not intended for:**  
+Open-ended or unsafe dialog, general factual QA, or high-stakes applications.
+
+---
 
 ## Evaluation (GSM8K Test Set, 1,319 problems)
 
-| Model | Greedy (%) | Maj@8 (%) | Notes |
-|-------|------------|-----------|-------|
-| Qwen2.5-Math-1.5B-Instruct | 84.8 | 89.5 | Reported settings |
-| **Qwen2.5-Math-1.5B-TreeRPO** | **86.4** | **89.6** | Same decoding (temp 0 / (0.7, top-p 0.8)) |
+| Model                          | Greedy (%) | Maj@8 (%) | Notes                                |
+|---------------------------------|------------|-----------|--------------------------------------|
+| Qwen2.5-Math-1.5B-Instruct      | 84.8       | 89.5      | Reported settings                    |
+| **TreeRPO-Qwen2.5-Math-1.5B**   | **86.4**   | **89.6**  | Same decoding (temp 0 / (0.7, 0.8))  |
 
-- **Greedy** = temperature 0 deterministic decoding.  
-- **Maj@8** = 8 sampled completions (temp 0.7, top-p 0.8) majority vote on final boxed answer. Ties / missing boxed answer → incorrect. Single-run numbers (no multi-seed variance).
+- **Greedy:** temperature = 0 (deterministic)
+- **Maj@8:** 8 completions (temperature 0.7, top-p 0.8); majority vote on final boxed answer
+
+---
 
 ## How to Use
 
+If your Transformers version supports chat templates (≥4.38), use:
+
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
 import torch
@@ -69,4 +78,3 @@ prompt_text = tok.apply_chat_template(
 inputs = tok(prompt_text, return_tensors="pt").to(model.device)
 outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.0)
 print(tok.decode(outputs[0], skip_special_tokens=True))
-