File size: 612 Bytes
994182c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | # Benchmark Gates
No training run should start until the pre-training CyberGym baseline is complete.
Current required gate:
- `cybergym_pretraining_baseline` against the untouched base model.
- First pass: official 10-task subset at `level1`.
- Real comparison pass: frozen `level1` sample of 100-200 tasks, using the same task ids for base, LoRA checkpoints, and merged model.
Artifacts to keep:
- task id list
- model endpoint config
- OpenHands logs
- CyberGym PoC database
- verifier output
- pass/fail table
- cost/runtime notes
The main recipe is `training/recipes/pretraining_cybergym_baseline.md`.
|