smjain/qwen-tool-grpo

Qwen2.5-1.5B-Instruct — GRPO tool-use (RL only, no SFT)

LoRA adapters trained with GRPO (no SFT) to use tools agentically in a real executing sandbox. The policy calls a tool, the tool actually runs (read/write/ edit/bash/grep/glob/ls), the result is fed back, and reward comes from the true task outcome (test passes / file correct / answer found) — a verifiable program, not a reward model.

• Base: Qwen/Qwen2.5-1.5B-Instruct
• Method: Group Relative Policy Optimization; G trajectories per task, group- normalized advantage, KL to the frozen base (adapter-off).
• Reward: verifiable outcome over 4 task families (fix_bug, locate, make_file, edit_config).
• No SFT: RL only, on top of the Instruct checkpoint.

Checkpoints are under step_*/ and final/. Load with PEFT on top of the base.