Instructions to use davidan03/openwebrl-4b-iterpool-schedule with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use davidan03/openwebrl-4b-iterpool-schedule with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
OpenWebRL-4B: pooled-data vs split-schedule RPO (LoRA adapters)
Two LoRA adapters (r 16, alpha 32, all attention + MLP projections) trained from OpenWebRL-4B-SFT with Round 1's RPO recipe (DPO beta 0.1 + SFT term weight 1.0 on successful-episode pairs, reference = OpenWebRL-4B-SFT, lr peak 5e-5, warmup 512 examples, half cosine to 2.5e-5, batch 32, 2 H200). Both see the same 11,554 Sol pairs (half A = Round 1 data, 5,304 pairs; half B = data collected by the Round 1 model, 6,250 pairs) twice: 723 updates.
| folder | experiment | data order |
|---|---|---|
pooled/ |
Experiment 1: accumulate both rounds' data, train once from SFT | A + B shuffled together, 2 epochs |
seqsched/ |
Experiment 2: two rounds as ONE LR schedule (no re-warmup) | 2 epochs of A, then 2 epochs of B; LR 4.41e-5 at the switch (update 332) |
Each folder: update-000723/ = final adapter; earlier checkpoints update-000181/362/542 (seqsched/ also update-000332,
the end of half A); recipe/ = frozen config, metrics, held-out validation, the training sbatch.
Load: PeftModel.from_pretrained(<OpenWebRL-4B-SFT>, "<repo>", subfolder="pooled/update-000723").
adapter_config.json names the base by its local cluster path; point it at your copy of OpenWebRL-4B-SFT.
Not yet evaluated.
- Downloads last month
- -