| license: other | |
| base_model: Qwen/Qwen3-4B-Instruct-2507 | |
| tags: | |
| - grpo | |
| - verl | |
| - math-reasoning | |
| - harpm | |
| # Hint_Model_GRPO_V1 | |
| GRPO-trained **Qwen3-4B-Instruct-2507** with **HARPM** (Hard-problem Adaptive Reference-Prompt Matching) hint injection. | |
| ## Method | |
| - Phase 1: 1 epoch plain GRPO on quarter training set → identify 8/8-fail hard problems (1709 found). | |
| - Phase 2: annotate 6-dim features with local Qwen → nearest-neighbor match against a hard reference set → inject reference problem+solution into prompts (597 problems hinted). | |
| - Phase 3: 14 epochs GRPO on the hinted dataset. | |
| - Total 15 epochs, matched budget/config with the plain baseline. | |
| ## Result (hardset validation) | |
| | metric | untrained | baseline02 (15ep plain) | **HINT (this model)** | | |
| |---|---|---|---| | |
| | acc mean@4 | 0.026 | 0.067 | **0.0865** | | |
| | acc best@4 | 0.046 | — | **0.122** | | |
| Equal-budget improvement of **+29%** mean@4 over the plain baseline; validation accuracy increased monotonically over training. | |
| ## Training config | |
| - 2 nodes x 8 GPU, TP=1, GRPO, lr=1e-6 (constant), train_batch=128, n=8, temperature=1.5, repetition_penalty=1.05 | |
| - max_prompt_len=2048, max_response_len=16384, attn=sdpa | |
| Note: single-seed run; multi-seed variance not yet measured. | |