---
base_model:
- deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
datasets:
- Miaow-Lab/RLVR-Linearity-Dataset
license: apache-2.0
pipeline_tag: text-generation
library_name: transformers
tags:
- reasoning
- grpo
- reinforcement-learning
---

# Model Card

## 1. Model Details
This model is a fine-tuned checkpoint described in the paper **"Not All Steps are Informative: On the Linearity of LLMs’ RLVR Training"**. It was trained using Reinforcement Learning (RL) to investigate the phenomenon of linear evolution in model weights and output log-probabilities during RLVR training.

- **Paper:** [Not All Steps are Informative: On the Linearity of LLMs' RLVR Training](https://huggingface.co/papers/2601.04537)
- **Code:** [GitHub Repository](https://github.com/Miaow-Lab/RLVR-Linearity)
- **Base Model:** [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B)
- **Training Method:** GRPO (using the `verl` framework)


## 2. Training Details

- **Hyperparameters:**
  - Learning Rate: `1e-6`
  - Train Batch Size: `128`
  - PPO Mini Batch Size: `64`
  - RL Algorithm: `GRPO`
  - Rollout Temperature: 1.0
  - Group Size: 16
- **Compute:** Trained on `32 x H100` GPUs for about `150` hours.

For full training configurations, please refer to the `config.json` or the training scripts in the official [GitHub repository](https://github.com/Miaow-Lab/RLVR-Linearity).

## 3. Citation

If you use this model in your research, please cite our paper:

```bibtex
@misc{wang2026stepsinformativelinearityllms,
      title={Not All Steps are Informative: On the Linearity of LLMs' RLVR Training}, 
      author={Tianle Wang and Zhongyuan Wu and Shenghao Jin and Hao Xu and Wei Chen and Ning Miao},
      year={2026},
      eprint={2601.04537},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2601.04537}, 
}
```

> [!TIP]
> **Motivation for this Model**
> This checkpoint is released primarily as a research artifact to facilitate the analysis of linearity in model outputs and weight updates during RLVR fine‑tuning.