Update README.md
Browse files
README.md
CHANGED
|
@@ -2,16 +2,32 @@
|
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
Playbox RFT Checkpoint (Generalist Base)
|
|
|
|
|
|
|
| 5 |
Model Name: Playbox_RFT_Checkpoint
|
|
|
|
| 6 |
Base Architecture: Qwen3.5-0.8B-Base (24 transformer layers, mixed linear/full attention, hidden dimension 1024, grouped-query attention).
|
|
|
|
| 7 |
Parameters: ~0.8B
|
|
|
|
| 8 |
Developers: Epoch AI Labs
|
|
|
|
| 9 |
Intended Use: General mathematical reasoning, chain-of-thought generation, and pattern completion. Serves as the default, untouched path for standard mathematical queries (e.g., GSM8K) in the Routed Hammer ensemble.
|
|
|
|
| 10 |
Training Pipeline:
|
| 11 |
Stage 1 (Full-Parameter SFT): Supervised fine-tuning on GSM8K training data to establish reasoning chains and formatting.
|
|
|
|
| 12 |
Stage 2A (Periphery-Only GRPO): Group Relative Policy Optimization restricted to the final 8 transformer layers to preserve lower-layer capabilities while improving math output. Utilized strict math gating and frontier filtering.
|
|
|
|
| 13 |
Stage 2B (Identity GRPO Alignment): Final 6 layers trained on behavioral objectives (identity, anti-sycophancy, ambiguity handling) with no math training.
|
|
|
|
|
|
|
| 14 |
Performance:
|
|
|
|
| 15 |
GSM8K: 50.3% accuracy, 94.8% format compliance (outperforming comparators up to 9× its size like Qwen2.5-Math-7B-Instruct).
|
|
|
|
| 16 |
Adversarial OOD: 22.0% accuracy (without routing).
|
|
|
|
|
|
|
|
|
|
| 17 |
Limitations: Vulnerable to catastrophic forgetting. If fine-tuned further on tool-integrated reasoning (TIR), it completely loses general mathematical ability (dropping to 3.5% on out-of-distribution math). Behavioral alignment (identity) remains unsolved (0% preservation).
|
|
|
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
Playbox RFT Checkpoint (Generalist Base)
|
| 5 |
+
|
| 6 |
+
|
| 7 |
Model Name: Playbox_RFT_Checkpoint
|
| 8 |
+
|
| 9 |
Base Architecture: Qwen3.5-0.8B-Base (24 transformer layers, mixed linear/full attention, hidden dimension 1024, grouped-query attention).
|
| 10 |
+
|
| 11 |
Parameters: ~0.8B
|
| 12 |
+
|
| 13 |
Developers: Epoch AI Labs
|
| 14 |
+
|
| 15 |
Intended Use: General mathematical reasoning, chain-of-thought generation, and pattern completion. Serves as the default, untouched path for standard mathematical queries (e.g., GSM8K) in the Routed Hammer ensemble.
|
| 16 |
+
|
| 17 |
Training Pipeline:
|
| 18 |
Stage 1 (Full-Parameter SFT): Supervised fine-tuning on GSM8K training data to establish reasoning chains and formatting.
|
| 19 |
+
|
| 20 |
Stage 2A (Periphery-Only GRPO): Group Relative Policy Optimization restricted to the final 8 transformer layers to preserve lower-layer capabilities while improving math output. Utilized strict math gating and frontier filtering.
|
| 21 |
+
|
| 22 |
Stage 2B (Identity GRPO Alignment): Final 6 layers trained on behavioral objectives (identity, anti-sycophancy, ambiguity handling) with no math training.
|
| 23 |
+
|
| 24 |
+
|
| 25 |
Performance:
|
| 26 |
+
|
| 27 |
GSM8K: 50.3% accuracy, 94.8% format compliance (outperforming comparators up to 9× its size like Qwen2.5-Math-7B-Instruct).
|
| 28 |
+
|
| 29 |
Adversarial OOD: 22.0% accuracy (without routing).
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
|
| 33 |
Limitations: Vulnerable to catastrophic forgetting. If fine-tuned further on tool-integrated reasoning (TIR), it completely loses general mathematical ability (dropping to 3.5% on out-of-distribution math). Behavioral alignment (identity) remains unsolved (0% preservation).
|