Hugging Face's logo

TokenBender
/
execution-midband-RL-v2

Reinforcement Learning
lora
grpo
code-generation
cpp
Model card Files
xet
 Community
execution-midband-RL-v2 / README.md
TokenBender's picture
TokenBender
Remove forward-lineage reference; refresh checksum
f9b96bb verified
preview code
|
Raw
History Blame Contribute Delete
3.78 kB
metadata 
license: mit
base_model: zai-org/GLM-4.7-Flash
tags:
  - lora
  - grpo
  - reinforcement-learning
  - code-generation
  - cpp
extra_gated_prompt: Access requests are reviewed manually.

execution-midband-RL-v2

Canonical name execution-midband-RL-v2 in the execution-grounded RL series. Historical run identity: execution-bank-RL-v2-think-r2 (W&B project execution-bank-RL-v2-think). This is the complete preserved run tree.

Lineage

Series name Run identity Adapter SHA-256 Notes
Synth-v1 ep50 iter_0000649 4acb7f23c295f45380155c5d9ee6bc59422262f0cb51f0c02f7e550d405b575a SFT starting point (checkpoints)
execution-midband-RL-v1 issue111-bank-official-grpo20-20260817T151213Z 186b0fc5b200fb8bb55bf85ee4416f2682a470580f0231c6f3f2a4d414bd898e bank drill, thinking off (archive)
execution-midband-RL-v2 (this repo) execution-bank-RL-v2-think-r2 f5c298f5554c6f5fa4a27f3329e4a08c6c2fef76834c0f10147a270ede41148d bank drill, thinking on — lineage anchor

Identity

  • Base model: GLM-4.7-Flash, revision 7dd20894a642a0aa287e9827cb1a1f7f91386b67
  • Cumulative LoRA rank 16 / alpha 32; serve as base + this adapter only — the adapter already contains the Synth-v1 SFT state (do not stack a second Synth-v1 adapter)
  • Final checkpoint: checkpoints/grpo_lora_r16/iter_0000019/adapter
  • Training: 20 GRPO updates, 8 exact-task bank-account drill prompts × 32 samples, thinking enabled, response budget 8192, LR 3e-5 (0.1 warmup), KL 0.1 against the frozen starting policy, strict binary execution reward (official GCC 13 oracle in Docker)
  • Source commit: 7c70b963425d81ce1a3cd344524b87eb055c5852 (tokenbender/browser-is-all-you-need-upstream, branch client/5-aug-release)

Results

  • Training environment pass rate: 33.59% → 55.08% over 20 updates (5,120 executed rollouts, zero infrastructure-invalid samples)
  • Official bank-account, 64 independent one-shot samples: 60/64 = 93.75%
  • Official fixed26, four trials × 26 tasks, fixed26-contract-v2, tries=2, thinking enabled, temperature 0.7:
    • Pass@1: 11, 9, 10, 12 → mean 10.5/26 (40.38%)
    • By turn 2: 14, 17, 12, 15 → mean 14.5/26 (55.77%)
    • bank-account: 4/4 Pass@1
  • Fixed26 evidence: execution-bank-RL-v2-think-r2/fixed26-mt2-4x-20260818/ in this repo (all 104 task receipts; summary.json SHA-256 d324cccff759f33b64bcbb4176a584f53776bcb1c8a1165d935608220c15fcf7)

Contents

  • execution-bank-RL-v2-think-r2/ — full run tree: checkpoints at updates 5/10/15/20 (native TP4 shards + HF adapters + finiteness receipts), rollout dumps, drill dataset, run logs, online-W&B local records, eval64 evidence, fixed26 four-trial evidence, run_status.txt
  • launch-configs/ — exact SkyPilot training and eval64 launch configurations
  • SHA256SUMS — digest of every file in this repo

External evidence

Caveats

  • The saved training_state_rank*.pt files hold optimizer metadata and scheduler state only — no Adam moment tensors — so exact in-place resume is not possible; any continuation must warm-start from the adapter.
  • The fixed26 evaluation wrapper exited 141 after the completion marker and summary were written (post-completion log-pipe artifact); no trial was affected.