File size: 2,530 Bytes
4d69456 d7a75b5 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 | ---
license: apache-2.0
base_model: Qwen/Qwen3-Coder-30B-A3B-Instruct
tags: [reinforcement-learning, rlvr, skyrl, tasktrove, qwen3-moe]
---
# tasktrove-dq-unitsyn-python (step 20)
RL checkpoint from the TaskTrove data-quality sweep, trained with
[SkyRL](https://github.com/marin-community/MarinSkyRL) from `Qwen/Qwen3-Coder-30B-A3B-Instruct` using RLOO over agentic
software-engineering tasks executed by OpenCode in sandboxed environments.
- Base model: `Qwen/Qwen3-Coder-30B-A3B-Instruct` (Qwen3 MoE, 48 layers)
- Checkpoint: `global_step_20`
- Source run: `rl-tasktrove-dq-sweep-30b-terminus2-qwen-20260725-163115-1ae770`
- Weights: 16 safetensors shards, 61.1 GB
## What this is for
The sweep measures **dataset quality**, not model quality. Each arm trains the same base model on a
different TaskTrove source so the sources can be compared. These checkpoints are research artifacts
for that comparison. None has been evaluated as a general-purpose model, and no benchmark numbers
are claimed here.
## Training configuration
RLOO (`advantage_estimator: rloo_n`) with megatron backend, tensor-parallel 4, pipeline-parallel 2,
expert-parallel 4, across 32 H100s. The objective is deliberately unregularized: `use_kl_loss:
false`, `use_entropy_loss: false`, and `policy_update_steps: 1`, which leaves the PPO clip ratio
inert at 0.0. That choice makes entropy dynamics the primary failure mode across the sweep, and it
is why several arms ended early.
## Provenance
Exported from a `torch.distributed.checkpoint` megatron checkpoint by re-running the trainer's own
export path (`bridge.save_hf_weights`) at the checkpoint's own step, so no offline conversion was
involved. Shard count, index `total_size` and `weight_map` completeness were verified against the
object store before upload.
## Training Traces
Training-time OpenCode/Harbor rollouts for this run are published as a companion dataset:
**[laion/terminal_bench_2_tasktrove_dq_unitsyn_python_step20_30b_a3b_20260730_014827](https://huggingface.co/datasets/laion/terminal_bench_2_tasktrove_dq_unitsyn_python_step20_30b_a3b_20260730_014827)**
The dataset contains the `last` episode of each trial (per
`make_and_upload_trace_dataset --episodes last`) -- the rollouts the policy was trained on.
## Training Logs
`training_logs/` holds the parsed metric surface for this run -- per-step training metrics,
vLLM engine metrics, a summary report, and the reward-vs-steps plot -- alongside the raw
trainer log. Capability tokens have been redacted from the logs.
|