Upload completed program-tree base RL model

a01bb85 verified 2 days ago

950 Bytes

---
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
library_name: transformers
tags:
- reinforcement-learning
- rlve
- grpo
- program-trees
---
# RL-Base-R1-Distill-Q1.5B

DeepSeek-R1-Distill-Qwen-1.5B continued with reinforcement learning on
random program trees built from string-operation primitives. The trees have
depth 1–2 and up to two children at each node. Training used 20,000 sampled
rollouts (2,500 updates), one prompt and eight rollouts per update, four GPUs,
and 1,024 initial generation tokens plus answer forcing with up to 128 more
tokens. This is the final model checkpoint at update 2,499.

The training run and its in-distribution evaluation history are on
[Weights & Biases](https://wandb.ai/david-heineman/rl-trees-base-rl/runs/f75d51ca). The optimizer and sampler state are retained
in the original training checkpoint; this repository provides model weights
and tokenizer files for inference or further fine-tuning.

	---
	base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
	library_name: transformers
	tags:
	- reinforcement-learning
	- rlve
	- grpo
	- program-trees
	---

	# RL-Base-R1-Distill-Q1.5B

	DeepSeek-R1-Distill-Qwen-1.5B continued with reinforcement learning on
	random program trees built from string-operation primitives. The trees have
	depth 1–2 and up to two children at each node. Training used 20,000 sampled
	rollouts (2,500 updates), one prompt and eight rollouts per update, four GPUs,
	and 1,024 initial generation tokens plus answer forcing with up to 128 more
	tokens. This is the final model checkpoint at update 2,499.

	The training run and its in-distribution evaluation history are on
	[Weights & Biases](https://wandb.ai/david-heineman/rl-trees-base-rl/runs/f75d51ca). The optimizer and sampler state are retained
	in the original training checkpoint; this repository provides model weights
	and tokenizer files for inference or further fine-tuning.