| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3-Coder-30B-A3B-Instruct |
| tags: [reinforcement-learning, rlvr, skyrl, tasktrove, qwen3-moe] |
| --- |
| |
| # tasktrove-dq-unitsyn-python (step 20) |
|
|
| RL checkpoint from the TaskTrove data-quality sweep, trained with |
| [SkyRL](https://github.com/marin-community/MarinSkyRL) from `Qwen/Qwen3-Coder-30B-A3B-Instruct` using RLOO over agentic |
| software-engineering tasks executed by OpenCode in sandboxed environments. |
|
|
| - Base model: `Qwen/Qwen3-Coder-30B-A3B-Instruct` (Qwen3 MoE, 48 layers) |
| - Checkpoint: `global_step_20` |
| - Source run: `rl-tasktrove-dq-sweep-30b-terminus2-qwen-20260725-163115-1ae770` |
| - Weights: 16 safetensors shards, 61.1 GB |
|
|
| ## What this is for |
|
|
| The sweep measures **dataset quality**, not model quality. Each arm trains the same base model on a |
| different TaskTrove source so the sources can be compared. These checkpoints are research artifacts |
| for that comparison. None has been evaluated as a general-purpose model, and no benchmark numbers |
| are claimed here. |
|
|
| ## Training configuration |
|
|
| RLOO (`advantage_estimator: rloo_n`) with megatron backend, tensor-parallel 4, pipeline-parallel 2, |
| expert-parallel 4, across 32 H100s. The objective is deliberately unregularized: `use_kl_loss: |
| false`, `use_entropy_loss: false`, and `policy_update_steps: 1`, which leaves the PPO clip ratio |
| inert at 0.0. That choice makes entropy dynamics the primary failure mode across the sweep, and it |
| is why several arms ended early. |
|
|
| ## Provenance |
|
|
| Exported from a `torch.distributed.checkpoint` megatron checkpoint by re-running the trainer's own |
| export path (`bridge.save_hf_weights`) at the checkpoint's own step, so no offline conversion was |
| involved. Shard count, index `total_size` and `weight_map` completeness were verified against the |
| object store before upload. |
|
|
| ## Training Traces |
|
|
| Training-time OpenCode/Harbor rollouts for this run are published as a companion dataset: |
| **[laion/terminal_bench_2_tasktrove_dq_unitsyn_python_step20_30b_a3b_20260730_014827](https://huggingface.co/datasets/laion/terminal_bench_2_tasktrove_dq_unitsyn_python_step20_30b_a3b_20260730_014827)** |
|
|
| The dataset contains the `last` episode of each trial (per |
| `make_and_upload_trace_dataset --episodes last`) -- the rollouts the policy was trained on. |
|
|
| ## Training Logs |
|
|
| `training_logs/` holds the parsed metric surface for this run -- per-step training metrics, |
| vLLM engine metrics, a summary report, and the reward-vs-steps plot -- alongside the raw |
| trainer log. Capability tokens have been redacted from the logs. |
|
|
