| library_name: pytorch | |
| tags: | |
| - causal-lm | |
| - reinforcement-learning | |
| - human-feedback | |
| - testgeniy | |
| - custom-architecture | |
| language: | |
| - en | |
| - ru | |
| base_model: Asilarkness/testgeniy | |
| # TestGeniy RL v2 - HelpSteer2/OASST real-RL step 20 | |
| Experimental post-training checkpoint for the custom TestGeniy architecture. | |
| This is a separate candidate folder; it does not replace the primary dialogue_sft_v6 checkpoint. | |
| ## Training | |
| - Base: testgeniy_v6_clean/checkpoints/dialogue_sft.pt | |
| - Algorithm: online group-relative policy gradient with a KL anchor to v6 | |
| - Human reward: reward head trained on human HelpSteer2 ratings and validated on OASST preferences | |
| - Verifier rewards: real GSM8K, MATH, ARC-Challenge and FOLIO labels | |
| - Prompts: real-only; no synthetic prompt dataset | |
| - MTP: disabled | |
| - Updates: 20 | |
| - Group size: 4 | |
| - Learning rate: 5e-8 | |
| ## Evaluation | |
| Same fixed 100-example suite and evaluator used for the v6 comparison: | |
| | Checkpoint | GSM8K | MATH-500 | ARC | FOLIO | Composite | | |
| |---|---:|---:|---:|---:|---:| | |
| | v6 Dialogue SFT | 24 | 7 | 26 | 29 | 21.50 | | |
| | RL v2 step 20 | 24 | 8 | 26 | 29 | 21.75 | | |
| Human holdouts: OASST validation 55% for both; HelpSteer2-derived 500-pair holdout 41.6% for both. | |
| This checkpoint is an experimental candidate. Validate before replacing the primary model. | |