code/local_train/README.md

# Local challenger train (SFT then RL)

Offline loop for an Albedo SN97 Qwen3.6-35B-A3B challenger. Starts from genesis, packs official gold turns with the live rewritten-submit protocol, LoRA-SFT, exports with **byte-identical** genesis metadata, local-duels, then light GRPO.

`proxy_score` is a gate, not the GLM checklist. Do not RL a model that submits without editing.

## Commands

From the albedo repo root:

```bash
source /workspace/.venv/bin/activate
export PYTHONPATH=/workspace/gpt/albedo/src:/workspace/gpt/albedo
pip install -e '.[train]'   # once: trl, peft, accelerate, deepspeed

# 1. pack official parquet (smoke first)
python -m local_train pack --smoke
python -m local_train inspect --pack /workspace/data/sft-packs/sft-64-smoke.jsonl
python -m local_train pack --max-examples 20000

# 2. LoRA SFT from genesis
python -m local_train sft --smoke
python -m local_train sft --pack /workspace/data/sft-packs/sft-20000-sft-pack.jsonl

# 3. merge + copy genesis metadata
python -m local_train export --adapter /workspace/models/albedo-challenger-sft-adapter \
  --out-dir /workspace/models/albedo-challenger-sft

# 4. hash + albedo check-model (does not run OpenSearch dedup)
python -m local_train check --path /workspace/models/albedo-challenger-sft

# 5. live sanity chain (must pass before any on-chain commit — 1 miner-fault left bans)
python -m local_train chain-pack --mix-pack /workspace/data/sft-packs/sft-4000-sft-pack.jsonl
python -m local_train chain --challenger /workspace/models/albedo-challenger-rl --samples 3 --turns 32

# 6. go/no-go duel (8×12, then 32×12)
python -m local_train duel --challenger /workspace/models/albedo-challenger-sft --samples 8 --turns 12
python -m local_train duel --challenger /workspace/models/albedo-challenger-sft --samples 32 --turns 12

# 7. GRPO only after go (edit rate ≥ 0.5 and submit ≫ 0)
python -m local_train rl --smoke
python -m local_train export --adapter /workspace/models/albedo-challenger-rl-adapter \
  --base /workspace/models/albedo-challenger-sft \
  --out-dir /workspace/models/albedo-challenger-rl
python -m local_train check --path /workspace/models/albedo-challenger-rl
```

## Reward (RL)

Hard 0 on truncate / loop / no bash / leak / two submits with no edit between them (the live chain ban). Then edit-on-gold-path, exact submit, submit-before-edit penalty, dummy `newfile.py` / `filename.py` penalty. Not `proxy_score`.