Qwen3-1.7B-RL-final

Qwen3-1.7B-RL-final is a reinforcement learning (RL) post-trained version of Qwen/Qwen3-1.7B. It is optimized for multilingual reasoning, mathematics, and algorithmic problem-solving.

This model was trained using the veRL framework with GRPO (Group Relative Policy Optimization) on procedural datasets generated by the Reasoning Gym (r-gym) framework.

Training Methodology

• Base Model: Qwen/Qwen3-1.7B
• RL Framework: veRL
• RL Algorithm: GRPO (Group Relative Policy Optimization)
• Reward Function: Algorithmic verification via Reasoning Gym (rg_reward.py:compute_score)
• Dataset: Procedural dataset generated via reasoning-gym containing multiple algorithmic and logic tasks across multiple languages, available at MauroPello/reasoning-gym-verl-datasets. The training set was dynamically interleaved across tasks and languages to maintain a balanced data distribution.
• Hyperparameters:
  • Learning Rate: 1e-6
  • Batch Size: 256
  • Max Prompt Length: 4096
  • Max Response Length: 8192
  • PPO Epochs: 2
  • KL Coefficient: 0.001

Evaluations & Performance

We evaluated the model against the base Qwen/Qwen3-1.7B across several benchmarks, showing substantial improvements in reasoning accuracy, particularly on the Reasoning Gym validation split and held-out validation tasks/languages.

Benchmark / Dataset	Base model Accuracy	RL model Accuracy	Absolute Improvement
Reasoning Gym Validation Split (val.parquet)	35.29%	50.51%	+15.22%
Held Out Languages (Dutch nl, Turkish tr)	30.20%	36.60%	+6.40%
Held Out Tasks	37.88%	40.29%	+2.41%
MMLU Pro Lite (Multilingual subset)	38.48%	38.97%	+0.49%
Polymath (Multilingual Mathematics)	33.24%	33.44%	+0.20%

Key Takeaways

1. Massive Validation Gains: Accuracy on the Reasoning Gym validation set jumped by 15.22% (from 35.29% to 50.51%).
2. Strong Language Generalization: On held-out languages (Dutch and Turkish) not specifically targeted during translation tuning, generalization accuracy improved by 6.40%.
3. Robust Task Generalization: Accuracy on held-out tasks increased by 2.41%, proving that GRPO training generalizes to unseen rule-based verification tasks.