Resume from stage 3 by pulling the SFT and stage2_multi adapters
from the Hub. Uses 150 steps per stage instead of 200 so we can
finish before the next /data wipe on HF Spaces.
Made-with: Cursor
resume script: fix dataset module names + opt-out of hf_transfer
run_resume_stage2.sh referenced data.build_holdout, but the actual
modules are train.make_sft_dataset and eval.make_holdout. Also
export HF_HUB_ENABLE_HF_TRANSFER=0 in both pipeline scripts so a
missing hf_transfer wheel can't take down model loading.
Made-with: Cursor
Resume the GRPO curriculum from stage 2 by pulling the SFT and
stage1 LoRA adapters from the Hub. Used after an HF Spaces /data
wipe so we don't redo SFT and stage1 work that already survived.
Made-with: Cursor
HF Spaces sets HF_HUB_ENABLE_HF_TRANSFER=1 by default; without the
matching hf_transfer wheel installed, every uncached HF Hub download
(including the model config that unsloth's loader fetches) fails
with "Fast download is enabled but hf_transfer package is not
available". Install hf_transfer along with msgspec, torchao, and
cut_cross_entropy so unsloth_zoo's runtime path is intact.
Made-with: Cursor
run_full_pipeline.sh: pin trl/datasets/tyro to unsloth_zoo's window
The previous --no-deps install of trl pulled trl 1.2.0 (latest)
which broke unsloth's model loader at runtime. Pin trl to
unsloth_zoo's declared compatibility window (>=0.18.2,<=0.24.0)
and similarly cap datasets (<4.4.0). Also explicitly install tyro
which unsloth_zoo lists in its requires.
Made-with: Cursor
training: push adapters to HF Hub after SFT + each GRPO stage
HF Spaces /data is ephemeral and silently got wiped mid-run during
testing, killing the SFT adapter and the stage1 GRPO checkpoint.
Push each adapter to a dedicated HF Hub model repo as soon as it's
saved:
shivam2k3/opensoc-defender-grpo-sft (after SFT)
shivam2k3/opensoc-defender-grpo-stage1_basic (after stage1)
shivam2k3/opensoc-defender-grpo-stage2_multi (after stage2)
...
shivam2k3/opensoc-defender-grpo (final, written by
scripts/run_full_pipeline.sh)
A later restart can re-run on a fresh container by downloading the
latest bookmark instead of redoing 12 min of SFT or 16 min per GRPO
stage.
Made-with: Cursor
Two changes for resuming a partially-completed GRPO run on the L4:
1. scripts/run_grpo_to_end.sh: same as run_full_pipeline.sh but
without the SFT/install/dataset steps, so we can re-kick the
trainer after fixing trl 0.24 issues without paying for another
~12-min SFT pass.
2. defender_reward: trl 0.24's GRPOTrainer passes `completions` as
list[list[message-dict]] (conversational) rather than list[str].
Add `_completion_to_text` to extract the assistant text in either
shape; the unit tests still pass on the original list[str] path.
Made-with: Cursor
run_full_pipeline.sh: pin transformers<5; drop xformers; +hf_hub
Probe of the L4 image shows transformers 5.5.0 pre-shipped (its lazy
TrainingArguments import is broken). Force transformers>=4.46,<5
explicitly. Drop xformers (unsloth falls back gracefully) and add
huggingface_hub explicit (gradio's transitive doesn't pull it on this
image build).
Made-with: Cursor
run_full_pipeline.sh: lock torch+torchvision; --no-deps for unsloth
The 3-way version fight (torch 2.10 from base image, torchvision 0.26
which yanks torch to 2.11, unsloth_zoo<2.11 which yanks back) was
breaking transformers.TrainingArguments at import time.
Fix: explicitly pin torch==2.10.0 + torchvision==0.25.0 from the
PyTorch cu128 index, then install unsloth/unsloth_zoo/trl/peft/etc with
--no-deps so pip doesn't second-guess the torch pin. Add the few real
runtime deps (transformers, xformers, sentencepiece) explicitly.
Made-with: Cursor
The transitive 'future' dep (pulled in by unsloth deps) ships an old
sdist-only release that does `from collections import Sequence` which
was moved to collections.abc in 3.10. Pre-install future>=1.0.0 (which
has wheels and the proper import) and drop the cu121 extras suffix
since the L4 image already ships CUDA 12.5.
Made-with: Cursor
When HF_TOKEN and HF_PUSH_TARGET=<user>/<repo> are set, the script
now uploads the trained LoRA adapter, demo JSON, and 6 plots to the
target repo via huggingface_hub.HfApi (no git config / git LFS dance).
Used by remote-driven training runs from a JupyterLab Space terminal.
Made-with: Cursor
