Upload README.md with huggingface_hub

README.md

@@ -10,11 +10,13 @@ GRPO RL fine-tune of the Marin **Delphi-25B** base model (`marin-community/delph
 This is the **step-160** checkpoint (20 epochs). Training reward on the hard-math distribution rose from ~4% to ~29%.
 
 ## Recipe
-- GRPO, rule-based boxed-answer reward (`math_verify`), `kl_loss_coef=1e-3`, `rollout.n=8`, lr 5e-7.
-- **Rollout temperature 0.5** (not the recipe default of 1.0 — a temperature sweep showed the reward signal collapses above ~0.5 for this weak base on hard math).
+- GRPO, rule-based boxed-answer reward (`math_verify`), `kl_loss_coef=1e-3`, `rollout.n=8`, lr 5e-7, top_p 1.0.
+- **Rollout temperature 0.5** (not the recipe default of 1.0 — a sweep showed the reward signal collapses above ~0.5 for this weak base on hard math).
 - Seeded with a **3-shot prompt** (a base model needs demonstrations, not a bare instruction). The model expects this exact prompt format and stops at `<|im_end|>`.
 
-## Prompt format (3-shot prefix, then your problem)
+> Note: the Delphi tokenizer is Llama-3 based, so `<|im_start|>`/`<|im_end|>` are **literal text, not special tokens**. Feed them as raw strings and **stop generation at the `<|im_end|>` string**.
+
+## Prompt format (fixed 3-shot prefix, then your problem)
 ```
 <|im_start|>system
 You are a helpful assistant.<|im_end|>
@@ -41,4 +43,16 @@ Please reason step by step, and put your final answer within \boxed{}.<|im_end|>
 <|im_start|>assistant
 ```
 
-Research checkpoint; inherits the base model license.
+## Example problems
+Append each to the 3-shot prefix above. Sampling: greedy, stop=`<|im_end|>`.
+
+| Problem | Answer |
+|---|---|
+| What is the remainder when $2^{10}$ is divided by 7? | 2 |
+| A right square pyramid has base edge 6 and lateral edge 5. Find the total area of its four triangular faces. | 48 |
+| A right triangle has legs 9 and 12. What is the length of the hypotenuse? | 15 |
+| What is the largest prime factor of 2024? | 23 |
+| How many positive divisors does 36 have? | 9 |
+
+## Capability notes (honest)
+RL gave a weak base **genuine but narrow** competence. It reliably executes **short, single-thread computations** (modular cycles, one Pythagorean step, small factorizations) and reasons in the correct shape (steps -> single \boxed{} answer -> clean stop). It is **unreliable on problems requiring multi-term bookkeeping** (combining several intermediate quantities), where it drops terms or slips on arithmetic. ~29% on hard (level 3-5) MATH reflects this. It is a demonstration that the SimpleRL recipe works mechanically on a weak base; it is **not** a strong general math model.