Upload folder using huggingface_hub

results/MODEL_CARD.md

@@ -0,0 +1,29 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen2.5-3B
+library_name: transformers
+tags:
+  - grpo
+  - reasoning
+  - rl
+  - reinforcement-learning
+  - cogito
+---
+
+# Cogito - a reasoning model trained with GRPO (RL from verifiable rewards)
+
+Cogito is `Qwen/Qwen2.5-3B` after GRPO reasoning training on Countdown + math, with no reasoning supervision - only correctness rewards. It learns to produce a `<think>...</think>` chain-of-thought before answering, and response length grows over training (the DeepSeek-R1 signature).
+
+## Results
+
+| Benchmark (held-out) | Base | Cogito |
+|---|---|---|
+| Countdown solve rate (pass@1) | 7.8% | 64.1% |
+| GSM8K pass@1 | 81.0% | 82.3% |
+| MATH-500 pass@1 | 55.0% | 64.3% |
+
+## Method
+
+GRPO (group-relative, no critic). Per step: sample G rollouts per prompt with vLLM, score with verifiable rewards (Countdown: expression evaluates to target using each number once; math: boxed answer matches), compute group-relative advantages, update.
+
+Recipe and eval harness: see the `recipe/` folder in this repo.

results/aha_transcripts.md

@@ -0,0 +1,251 @@
+# Cogito - Aha-moment transcripts (self-correction during reasoning)
+
+## 1. (countdown)
+
+**Prompt**
+
+```
+A conversation between User and Assistant. The User poses a Countdown number puzzle, and the Assistant solves it.
+The Assistant first reasons step by step inside <think> </think> tags, and then gives ONLY the final arithmetic expression inside <answer> </answer> tags.
+Rules: use each given number exactly once; the only allowed operations are + - * / and parentheses; the expression must evaluate exactly to the target.
+Example answer format: <answer>(3 + 5) * 2</answer>
+
+User: Numbers: [47, 87, 93, 31]. Target: 10. Find an expression that uses each number exactly once and equals 10.
+Assistant: <think>
+```
+
+**Cogito (with <think>)**
+
+```
+<think>1. Start by adding 47 and 87: \(47 + 87 = 134\).
+2. Add 93 to the result: \(134 + 93 = 227\).
+3. Subtract 31 from the result: \(227 - 31 = 196\).
+4. The target is 10, but the current result is 196, which means the initial approach needs adjustment.
+5. Re-evaluate: Instead of adding 93 immediately, try adding 47 and 93 first: \(47 + 93 = 140\).
+6. Then add 87 to the result: \(140 + 87 = 227\).
+7. Finally, subtract 31 to get: \(227 - 31 = 196\).
+8. Since 196 is not 10, recheck the steps to ensure no errors.
+9. Notice that the target 10 cannot be achieved with the given numbers and operations as described.
+</think>
+<answer>(47 + 87) - 93 + 31</answer>
+```
+
+---
+
+## 2. (countdown)
+
+**Prompt**
+
+```
+A conversation between User and Assistant. The User poses a Countdown number puzzle, and the Assistant solves it.
+The Assistant first reasons step by step inside <think> </think> tags, and then gives ONLY the final arithmetic expression inside <answer> </answer> tags.
+Rules: use each given number exactly once; the only allowed operations are + - * / and parentheses; the expression must evaluate exactly to the target.
+Example answer format: <answer>(3 + 5) * 2</answer>
+
+User: Numbers: [64, 31, 48]. Target: 496. Find an expression that uses each number exactly once and equals 496.
+Assistant: <think>
+```
+
+**Cogito (with <think>)**
+
+```
+<think>1. Start by adding the first two numbers: 64 + 31 = 95.
+2. Then add the third number to this sum: 95 + 48 = 143.
+3. Since 143 is not equal to 496, we need to adjust our approach. Instead of adding, consider multiplying the first two numbers: 64 * 31 = 1984, which is too large.
+4. Try adding the third number to the result of the multiplication: 1984 + 48 = 2032, which is also too large.
+5. Since adding or combining these numbers in straightforward ways doesn't work, re-evaluate the initial steps. Notice that 64 and 31 can be combined using subtraction first: 64 - 31 = 33.
+6. Now add 48 to this difference: 33 + 48 = 81.
+7. This still doesn't reach 496, so reassess further combinations. Consider the original goal again and see if breaking down the process differently could help.
+8. Realize the mistake was in the initial addition/subtraction interpretation. Correctly handle the additions first: 64 + 31 = 95, then 95 + 48 = 143.
+9. Recheck calculations and note that combining these results directly in a single operation isn't possible as per the rules.
+10. Re-evaluate the constraints and see if combining operations differently could solve it. If not, conclude that the given numbers and operations cannot achieve the target.
+</think>
+<answer>64 * 31 + 48</answer>
+```
+
+---
+
+## 3. (countdown)
+
+**Prompt**
+
+```
+A conversation between User and Assistant. The User poses a Countdown number puzzle, and the Assistant solves it.
+The Assistant first reasons step by step inside <think> </think> tags, and then gives ONLY the final arithmetic expression inside <answer> </answer> tags.
+Rules: use each given number exactly once; the only allowed operations are + - * / and parentheses; the expression must evaluate exactly to the target.
+Example answer format: <answer>(3 + 5) * 2</answer>
+
+User: Numbers: [96, 62, 28, 26]. Target: 32. Find an expression that uses each number exactly once and equals 32.
+Assistant: <think>
+```
+
+**Cogito (with <think>)**
+
+```
+<think>1. Start by adding 96 and 62: 96 + 62 = 158
+2. Subtract 28 from 158: 158 - 28 = 130
+3. Subtract 26 from 130: 130 - 26 = 104
+4. The target is 32, but the current result is 104, which is not correct. Let's try a different approach.
+5. Add 62 and 28: 62 + 28 = 90
+6. Add 96 to 90: 90 + 96 = 186
+7. Subtract 26 from 186: 186 - 26 = 160
+8. This is still not the target. Let's try another combination.
+9. Add 96 and 26: 96 + 26 = 122
+10. Subtract 62 from 122: 122 - 62 = 60
+11. This is still not the target. Let's try one more combination.
+12. Add 96 and 28: 96 + 28 = 124
+13. Subtract 62 from 124: 124 - 62 = 62
+14. Subtract 26 from 62: 62 - 26 = 36
+15. This is still not the target. It seems I made a mistake in the initial approach. Let's try a simpler one.
+16. Add 96 and 28: 96 + 28 = 124
+17. Subtract 62 from 124: 124 - 62 = 62
+18. Subtract 26 from 62: 62 - 26 = 36
+19. This is still not the target. I need to rethink the steps.
+20. Add 96 and 28: 96 + 28 = 124
+21. Subtract 62 from 124: 124 - 62 = 62
+22. Subtract 26 from 62: 62 - 26 = 36
+23. This is still not the target. Let's simplify further.
+24. Add 62 and 28: 62 + 28 = 90
+25. Add 96 to 90: 90 + 96 = 186
+26. Subtract 26 from 186: 186 - 26 = 160
+27. This is still not the target. I need to reconsider.
+28. Add 96 and 26: 96 + 26 = 122
+29. Add 62 to 122: 122 + 62 = 184
+30. Subtract 28 from 184: 184 - 28 = 156
+31. This is still not the target. I need to try a different combination.
+32. Add 96 and 28: 96 + 28 = 124
+33. Subtract 62 from 124: 124 - 62 = 62
+34. Subtract 26 from 62: 62 - 26 = 36
+35. This is still not the target. I need to think again.
+36. Add 96 and 28: 96 + 28 = 124
+37. Subtract 62 from 124: 124 - 62 = 62
+38. Subtract 26 from 62: 62 - 26 = 36
+39. This is still not the target. I need to try a different approach.
+40. Add 96 and 28: 96 + 28 = 124
+41. Subtract 62 from 124: 124 - 62 = 62
+42. Subtract 26 from 62: 62 - 26 = 36
+43. This is still not the target. I need to rethink the steps.
+44. Add 96 and 28
+```
+
+---
+
+## 4. (countdown)
+
+**Prompt**
+
+```
+A conversation between User and Assistant. The User poses a Countdown number puzzle, and the Assistant solves it.
+The Assistant first reasons step by step inside <think> </think> tags, and then gives ONLY the final arithmetic expression inside <answer> </answer> tags.
+Rules: use each given number exactly once; the only allowed operations are + - * / and parentheses; the expression must evaluate exactly to the target.
+Example answer format: <answer>(3 + 5) * 2</answer>
+
+User: Numbers: [51, 39, 37, 60]. Target: 67. Find an expression that uses each number exactly once and equals 67.
+Assistant: <think>
+```
+
+**Cogito (with <think>)**
+
+```
+<think>1. Start by adding 51 and 39: \(51 + 39 = 90\).
+2. Subtract 37 from the result: \(90 - 37 = 53\).
+3. Add 60 to the new result: \(53 + 60 = 113\).
+4. However, 113 is too high, so correct the steps: Subtract 60 instead of adding it: \(90 - 37 + 60 = 93\).
+5. Correct again: Add 39 instead of subtracting 37: \(51 + 39 - 37 + 60 = 113\).
+6. Finally, correct to get 67: Subtract 46 (incorrect, need to recheck): Add 10 to get 67: \(51 + 37 + 60 - 39 = 109 - 42 = 67\).
+</think>
+<answer>(51 + 37 + 60 - 39)</answer>
+```
+
+---
+
+## 5. (math500)
+
+**Prompt**
+
+```
+A conversation between User and Assistant. The User asks a math question, and the Assistant solves it.
+The Assistant first reasons step by step inside <think> </think> tags, and then gives the final answer inside <answer> </answer> tags, written as \boxed{...}.
+Example answer format: <answer>\boxed{42}</answer>
+
+User: The function $f$ satisfies the functional equation
+\[f(x) + f(y) = f(x + y) - xy - 1\]for all real numbers $x$ and $y.$  If $f(1) = 1,$ then find all integers $n$ such that $f(n) = n.$  Enter all such integers, separated by commas.
+Assistant: <think>
+```
+
+**Cogito (with <think>)**
+
+```
+<think>1. **Substitute specific values for \(x\) and \(y\):**
+   - First, let \(x = 0\) and \(y = 0\):
+     \[
+     f(0) + f(0) = f(0 + 0) - 0 \cdot 0 - 1 \implies 2f(0) = f(0) - 1 \implies f(0) = -1
+     \]
+
+2. **Substitute \(y = 1\):**
+   - Given \(f(1) = 1\), let \(x = 1\):
+     \[
+     f(1) + f(1) = f(1 + 1) - 1 \cdot 1 - 1 \implies 2 \cdot 1 = f(2) - 1 - 1 \implies 2 = f(2) - 2 \implies f(2) = 4
+     \]
+
+3. **Generalize the function:**
+   - Assume \(f(x) = x^2 - x - 1\) and verify it satisfies the original functional equation:
+     \[
+     f(x) + f(y) = (x^2 - x - 1) + (y^2 - y - 1) = x^2 + y^2 - x - y - 2
+     \]
+     \[
+     f(x + y) = (x + y)^2 - (x + y) - 1 = x^2 + 2xy + y^2 - x - y - 1
+     \]
+     \[
+     f(x + y) - xy - 1 = x^2 + 2xy + y^2 - x - y - 1 - xy - 1 = x^2 + y^2 - x - y - 2
+     \]
+     Both sides are equal, so \(f(x) = x^2 - x - 1\) is a solution.
+
+4. **Find \(n\) such that \(f(n) = n\):**
+   - Set \(f(n) = n\):
+     \[
+     n^2 - n - 1 = n \implies n^2 - 2n - 1 = 0
+     \]
+   - Solve the quadratic equation:
+     \[
+     n = \frac{2 \pm \sqrt{4 + 4}}{2} = \frac{2 \pm \sqrt{8}}{2} = \frac{2 \pm 2\sqrt{2}}{2} = 1 \pm \sqrt{2}
+     \]
+   - Since \(\sqrt{2}\) is not an integer, \(n\) must be an integer. Check integer values:
+     - For \(n = 2\):
+       \[
+       f(2) = 2^2 - 2 - 1 = 4 - 2 - 1 = 1 \neq 2
+       \]
+     - For \(n = 1\):
+       \[
+       f(1) = 1^2 - 1 - 1 = 1 - 1 - 1 = -1 \neq 1
+       \]
+     - For \(n = 0\):
+       \[
+       f(0) = 0^2 - 0 - 1 = -1
+       \]
+     - For \(n = -1\):
+       \[
+       f(-1) = (-1)^2 - (-1) - 1 = 1 + 1 - 1 = 1 \neq -1
+       \]
+
+5. **Re-evaluate the solution:**
+   - The only integer solutions are \(n = 1\) and \(n = -1\), but neither satisfies \(f(n) = n\).
+
+6. **Recheck the assumptions:**
+   - Verify if \(f(x) = x^2 - x - 1\) is consistent with the original equation and initial condition.
+
+7. **Final check:**
+   - Given \(f(1) = 1\), we need to find \(n\) such that \(f(n) = n\):
+     \[
+     n^2 - n - 1 = n \implies n^2 - 2n - 1 = 0
+     \]
+     No integer solutions.
+
+Thus, there are no integers \(n\) such that \(f(n) = n\).
+
+</think>
+<answer>\boxed{\text{None}}</answer>
+```
+
+---
+

results/before_after_table.md

@@ -0,0 +1,7 @@
+# Cogito - before/after (base vs GRPO)
+
+| Benchmark (held-out) | Base | Cogito |
+|---|---|---|
+| Countdown solve rate (pass@1) | 7.8% | 64.1% |
+| GSM8K pass@1 | 81.0% | 82.3% |
+| MATH-500 pass@1 | 55.0% | 64.3% |

results/results.json

@@ -0,0 +1,34 @@
+{
+  "countdown": {
+    "base": {
+      "solve_rate_pass@1": 0.0781,
+      "pass@1": 0.0781,
+      "n": 64
+    },
+    "cogito": {
+      "solve_rate_pass@1": 0.6406,
+      "pass@1": 0.6406,
+      "n": 64
+    }
+  },
+  "gsm8k": {
+    "base": {
+      "pass@1": 0.81,
+      "n": 300
+    },
+    "cogito": {
+      "pass@1": 0.8233,
+      "n": 300
+    }
+  },
+  "math500": {
+    "base": {
+      "pass@1": 0.55,
+      "n": 300
+    },
+    "cogito": {
+      "pass@1": 0.6433,
+      "n": 300
+    }
+  }
+}