metadata
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
library_name: transformers
tags:
- reinforcement-learning
- rlve
- grpo
- program-trees
RL-Base-R1-Distill-Q1.5B
DeepSeek-R1-Distill-Qwen-1.5B continued with reinforcement learning on random program trees built from string-operation primitives. The trees have depth 1–2 and up to two children at each node. Training used 20,000 sampled rollouts (2,500 updates), one prompt and eight rollouts per update, four GPUs, and 1,024 initial generation tokens plus answer forcing with up to 128 more tokens. This is the final model checkpoint at update 2,499.
The training run and its in-distribution evaluation history are on Weights & Biases. The optimizer and sampler state are retained in the original training checkpoint; this repository provides model weights and tokenizer files for inference or further fine-tuning.