HF Spaces /data is ephemeral and silently got wiped mid-run during
testing, killing the SFT adapter and the stage1 GRPO checkpoint.
Push each adapter to a dedicated HF Hub model repo as soon as it's
saved:
shivam2k3/opensoc-defender-grpo-sft (after SFT)
shivam2k3/opensoc-defender-grpo-stage1_basic (after stage1)
shivam2k3/opensoc-defender-grpo-stage2_multi (after stage2)
...
shivam2k3/opensoc-defender-grpo (final, written by
scripts/run_full_pipeline.sh)
A later restart can re-run on a fresh container by downloading the
latest bookmark instead of redoing 12 min of SFT or 16 min per GRPO
stage.
Made-with: Cursor
Two changes for resuming a partially-completed GRPO run on the L4:
1. scripts/run_grpo_to_end.sh: same as run_full_pipeline.sh but
without the SFT/install/dataset steps, so we can re-kick the
trainer after fixing trl 0.24 issues without paying for another
~12-min SFT pass.
2. defender_reward: trl 0.24's GRPOTrainer passes `completions` as
list[list[message-dict]] (conversational) rather than list[str].
Add `_completion_to_text` to extract the assistant text in either
shape; the unit tests still pass on the original list[str] path.
Made-with: Cursor
train_grpo: import unsloth at module top before trl
Two coupled fixes for the trl 0.24 + unsloth 2026.4 stack:
1. unsloth's monkey-patches (eos_token resolution, SFTTrainer
wrappers) only apply if `import unsloth` happens BEFORE trl's
SFTConfig/SFTTrainer are imported. Without this, SFTConfig
silently keeps its default eos_token = "<EOS_TOKEN>" placeholder
and the vocab check rejects it on every model.
2. trl 0.24's SFTTrainer (with unsloth's wrapper) calls formatting_func
on a batched dict-of-lists and expects list[str]; the old
per-example signature now raises "should return a list of
processed strings".
Made-with: Cursor
trl 0.24 removed `tokenizer=` and `max_seq_length=`/`packing=` from
SFTTrainer's signature. Use SFTConfig (subclass of TrainingArguments)
and pass tokenizer via `processing_class=`. This matches the GRPO
trainer call site, which was already on the new API.
Made-with: Cursor
