File size: 2,416 Bytes
2abcc30 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | # Local challenger train (SFT then RL)
Offline loop for an Albedo SN97 Qwen3.6-35B-A3B challenger. Starts from genesis, packs official gold turns with the live rewritten-submit protocol, LoRA-SFT, exports with **byte-identical** genesis metadata, local-duels, then light GRPO.
`proxy_score` is a gate, not the GLM checklist. Do not RL a model that submits without editing.
## Commands
From the albedo repo root:
```bash
source /workspace/.venv/bin/activate
export PYTHONPATH=/workspace/gpt/albedo/src:/workspace/gpt/albedo
pip install -e '.[train]' # once: trl, peft, accelerate, deepspeed
# 1. pack official parquet (smoke first)
python -m local_train pack --smoke
python -m local_train inspect --pack /workspace/data/sft-packs/sft-64-smoke.jsonl
python -m local_train pack --max-examples 20000
# 2. LoRA SFT from genesis
python -m local_train sft --smoke
python -m local_train sft --pack /workspace/data/sft-packs/sft-20000-sft-pack.jsonl
# 3. merge + copy genesis metadata
python -m local_train export --adapter /workspace/models/albedo-challenger-sft-adapter \
--out-dir /workspace/models/albedo-challenger-sft
# 4. hash + albedo check-model (does not run OpenSearch dedup)
python -m local_train check --path /workspace/models/albedo-challenger-sft
# 5. live sanity chain (must pass before any on-chain commit — 1 miner-fault left bans)
python -m local_train chain-pack --mix-pack /workspace/data/sft-packs/sft-4000-sft-pack.jsonl
python -m local_train chain --challenger /workspace/models/albedo-challenger-rl --samples 3 --turns 32
# 6. go/no-go duel (8×12, then 32×12)
python -m local_train duel --challenger /workspace/models/albedo-challenger-sft --samples 8 --turns 12
python -m local_train duel --challenger /workspace/models/albedo-challenger-sft --samples 32 --turns 12
# 7. GRPO only after go (edit rate ≥ 0.5 and submit ≫ 0)
python -m local_train rl --smoke
python -m local_train export --adapter /workspace/models/albedo-challenger-rl-adapter \
--base /workspace/models/albedo-challenger-sft \
--out-dir /workspace/models/albedo-challenger-rl
python -m local_train check --path /workspace/models/albedo-challenger-rl
```
## Reward (RL)
Hard 0 on truncate / loop / no bash / leak / two submits with no edit between them (the live chain ban). Then edit-on-gold-path, exact submit, submit-before-edit penalty, dummy `newfile.py` / `filename.py` penalty. Not `proxy_score`.
|