| --- |
| license: mit |
| library_name: transformers |
| tags: |
| - reinforcement-learning |
| - grpo |
| - math |
| - trajectories |
| --- |
| |
| # ES RL Model and Trajectories |
|
|
| This repository contains the step-20 Hugging Face export for |
| `qwen35-4b-math-grpo-dapo400-e15-seed1`, the corresponding RL trajectories, |
| and the latest evaluation outputs. |
|
|
| ## Contents |
|
|
| - `models/qwen35-4b-math-grpo-dapo400-e15-seed1-step20/`: Hugging Face model export with 4 safetensors shards, tokenizer files, and generation/config metadata. |
| - `models/qwen35-4b-docvqa-grpo-four-gpu-from-step40-seed42-step120/`: DocVQA Hugging Face model export with 4 safetensors shards, tokenizer files, and generation/config metadata. |
| - `trajectories/qwen35-4b-math-grpo-dapo400-e15-seed1/raw/`: raw train and validation trajectory JSONL files. |
| - `trajectories/qwen35-4b-math-grpo-dapo400-e15-seed1/exports/attempt2_step001_020/`: exported train/validation trajectory JSONL files for steps 1-20. |
| - `trajectories/qwen35-4b-docvqa-grpo-four-gpu-from-step40-seed42/trace2skill_collection_step80/`: DocVQA trace2skill trajectory collection outputs, including RL rollouts and sample-only trajectories. |
| - `eval/qwen35-4b-math-grpo-dapo400-e15-seed1/step20_eval_50x4096_rerun_20260805/`: evaluation summary and per-sample outputs. |
| - `eval/qwen35-4b-docvqa-grpo-four-gpu-from-step40-seed42/`: DocVQA post-eval outputs, trace2skill sample-only eval outputs, and RL curve CSV. |
|
|
| ## Evaluation |
|
|
| Evaluation was run with 4 samples per item, temperature 1.0, top-p 1.0, |
| top-k 40, presence penalty 2.0, and `paper_react_cli` mode. |
|
|
| | Dataset | Records | Mean Score | Max@4 | |
| | --- | ---: | ---: | ---: | |
| | DAPO100 | 400 | 0.6875 | 0.8300 | |
| | AIME2026 | 120 | 0.5833 | 0.7667 | |
| | DocVQA post-eval | 400 | 0.4522 | 0.5147 | |
|
|
| DocVQA trace2skill sample-only evaluation used 8 samples per item on 50 evolve |
| items and reached mean score 0.4338 with max@8 0.7585. |
|
|
| The full optimizer state, Megatron/FSDP training checkpoint files, tool |
| workspaces, server logs, and cache directories are not included in this repo. |
|
|
