Local challenger train (SFT then RL)
Offline loop for an Albedo SN97 Qwen3.6-35B-A3B challenger. Starts from genesis, packs official gold turns with the live rewritten-submit protocol, LoRA-SFT, exports with byte-identical genesis metadata, local-duels, then light GRPO.
proxy_score is a gate, not the GLM checklist. Do not RL a model that submits without editing.
Commands
From the albedo repo root:
source /workspace/.venv/bin/activate
export PYTHONPATH=/workspace/gpt/albedo/src:/workspace/gpt/albedo
pip install -e '.[train]' # once: trl, peft, accelerate, deepspeed
# 1. pack official parquet (smoke first)
python -m local_train pack --smoke
python -m local_train inspect --pack /workspace/data/sft-packs/sft-64-smoke.jsonl
python -m local_train pack --max-examples 20000
# 2. LoRA SFT from genesis
python -m local_train sft --smoke
python -m local_train sft --pack /workspace/data/sft-packs/sft-20000-sft-pack.jsonl
# 3. merge + copy genesis metadata
python -m local_train export --adapter /workspace/models/albedo-challenger-sft-adapter \
--out-dir /workspace/models/albedo-challenger-sft
# 4. hash + albedo check-model (does not run OpenSearch dedup)
python -m local_train check --path /workspace/models/albedo-challenger-sft
# 5. live sanity chain (must pass before any on-chain commit — 1 miner-fault left bans)
python -m local_train chain-pack --mix-pack /workspace/data/sft-packs/sft-4000-sft-pack.jsonl
python -m local_train chain --challenger /workspace/models/albedo-challenger-rl --samples 3 --turns 32
# 6. go/no-go duel (8×12, then 32×12)
python -m local_train duel --challenger /workspace/models/albedo-challenger-sft --samples 8 --turns 12
python -m local_train duel --challenger /workspace/models/albedo-challenger-sft --samples 32 --turns 12
# 7. GRPO only after go (edit rate ≥ 0.5 and submit ≫ 0)
python -m local_train rl --smoke
python -m local_train export --adapter /workspace/models/albedo-challenger-rl-adapter \
--base /workspace/models/albedo-challenger-sft \
--out-dir /workspace/models/albedo-challenger-rl
python -m local_train check --path /workspace/models/albedo-challenger-rl
Reward (RL)
Hard 0 on truncate / loop / no bash / leak / two submits with no edit between them (the live chain ban). Then edit-on-gold-path, exact submit, submit-before-edit penalty, dummy newfile.py / filename.py penalty. Not proxy_score.