- README: all checklist items checked, links to 6 trained adapters on HF
- Blog: replace [PEND] placeholders with actual metrics
- Model card: complete card for opensoc-defender-grpo adapter
Add finish_from_stage4.sh: eval, bake, Hub upload without re-training
Resume from stage 3 by pulling the SFT and stage2_multi adapters
from the Hub. Uses 150 steps per stage instead of 200 so we can
finish before the next /data wipe on HF Spaces.
Made-with: Cursor
resume script: fix dataset module names + opt-out of hf_transfer
run_resume_stage2.sh referenced data.build_holdout, but the actual
modules are train.make_sft_dataset and eval.make_holdout. Also
export HF_HUB_ENABLE_HF_TRANSFER=0 in both pipeline scripts so a
missing hf_transfer wheel can't take down model loading.
Made-with: Cursor
Resume the GRPO curriculum from stage 2 by pulling the SFT and
stage1 LoRA adapters from the Hub. Used after an HF Spaces /data
wipe so we don't redo SFT and stage1 work that already survived.
Made-with: Cursor
HF Spaces sets HF_HUB_ENABLE_HF_TRANSFER=1 by default; without the
matching hf_transfer wheel installed, every uncached HF Hub download
(including the model config that unsloth's loader fetches) fails
with "Fast download is enabled but hf_transfer package is not
available". Install hf_transfer along with msgspec, torchao, and
cut_cross_entropy so unsloth_zoo's runtime path is intact.
Made-with: Cursor
run_full_pipeline.sh: pin trl/datasets/tyro to unsloth_zoo's window
The previous --no-deps install of trl pulled trl 1.2.0 (latest)
which broke unsloth's model loader at runtime. Pin trl to
unsloth_zoo's declared compatibility window (>=0.18.2,<=0.24.0)
and similarly cap datasets (<4.4.0). Also explicitly install tyro
which unsloth_zoo lists in its requires.
Made-with: Cursor
training: push adapters to HF Hub after SFT + each GRPO stage
HF Spaces /data is ephemeral and silently got wiped mid-run during
testing, killing the SFT adapter and the stage1 GRPO checkpoint.
Push each adapter to a dedicated HF Hub model repo as soon as it's
saved:
shivam2k3/opensoc-defender-grpo-sft (after SFT)
shivam2k3/opensoc-defender-grpo-stage1_basic (after stage1)
shivam2k3/opensoc-defender-grpo-stage2_multi (after stage2)
...
shivam2k3/opensoc-defender-grpo (final, written by
scripts/run_full_pipeline.sh)
A later restart can re-run on a fresh container by downloading the
latest bookmark instead of redoing 12 min of SFT or 16 min per GRPO
stage.
Made-with: Cursor
Two changes for resuming a partially-completed GRPO run on the L4:
1. scripts/run_grpo_to_end.sh: same as run_full_pipeline.sh but
without the SFT/install/dataset steps, so we can re-kick the
trainer after fixing trl 0.24 issues without paying for another
~12-min SFT pass.
2. defender_reward: trl 0.24's GRPOTrainer passes `completions` as
list[list[message-dict]] (conversational) rather than list[str].
Add `_completion_to_text` to extract the assistant text in either
shape; the unit tests still pass on the original list[str] path.
Made-with: Cursor
train_grpo: import unsloth at module top before trl
Two coupled fixes for the trl 0.24 + unsloth 2026.4 stack:
1. unsloth's monkey-patches (eos_token resolution, SFTTrainer
wrappers) only apply if `import unsloth` happens BEFORE trl's
SFTConfig/SFTTrainer are imported. Without this, SFTConfig
silently keeps its default eos_token = "<EOS_TOKEN>" placeholder
and the vocab check rejects it on every model.
2. trl 0.24's SFTTrainer (with unsloth's wrapper) calls formatting_func
on a batched dict-of-lists and expects list[str]; the old
per-example signature now raises "should return a list of
processed strings".
Made-with: Cursor
trl 0.24 removed `tokenizer=` and `max_seq_length=`/`packing=` from
SFTTrainer's signature. Use SFTConfig (subclass of TrainingArguments)
and pass tokenizer via `processing_class=`. This matches the GRPO
trainer call site, which was already on the new API.
Made-with: Cursor
run_full_pipeline.sh: pin transformers<5; drop xformers; +hf_hub
Probe of the L4 image shows transformers 5.5.0 pre-shipped (its lazy
TrainingArguments import is broken). Force transformers>=4.46,<5
explicitly. Drop xformers (unsloth falls back gracefully) and add
huggingface_hub explicit (gradio's transitive doesn't pull it on this
image build).
Made-with: Cursor
run_full_pipeline.sh: lock torch+torchvision; --no-deps for unsloth
The 3-way version fight (torch 2.10 from base image, torchvision 0.26
which yanks torch to 2.11, unsloth_zoo<2.11 which yanks back) was
breaking transformers.TrainingArguments at import time.
Fix: explicitly pin torch==2.10.0 + torchvision==0.25.0 from the
PyTorch cu128 index, then install unsloth/unsloth_zoo/trl/peft/etc with
--no-deps so pip doesn't second-guess the torch pin. Add the few real
runtime deps (transformers, xformers, sentencepiece) explicitly.
Made-with: Cursor
The transitive 'future' dep (pulled in by unsloth deps) ships an old
sdist-only release that does `from collections import Sequence` which
was moved to collections.abc in 3.10. Pre-install future>=1.0.0 (which
has wheels and the proper import) and drop the cu121 extras suffix
since the L4 image already ships CUDA 12.5.
Made-with: Cursor
When HF_TOKEN and HF_PUSH_TARGET=<user>/<repo> are set, the script
now uploads the trained LoRA adapter, demo JSON, and 6 plots to the
target repo via huggingface_hub.HfApi (no git config / git LFS dance).
Used by remote-driven training runs from a JupyterLab Space terminal.
Made-with: Cursor
- Clone shivam2k3/opensoc-env from the HF Hub (was a github REPLACE_ME).
- Set git user.email/name from HF_USERNAME so commits don't fail.
- Replace 'unsloth[colab-new]' extra with the plain unsloth install.
- Push trained artifacts via huggingface_hub.HfApi (upload_folder for
the LoRA adapter to its own model repo, upload_file for plots+demo
back to the env repo). No git LFS dance, no manual login.
Made-with: Cursor
HF Spaces iframes load the Space URL root, but our FastAPI app only
defined endpoint routes. Visitors saw FastAPI's stock {detail: Not Found}.
Redirect / to /demo/ so the Gradio "before vs after" UI is what humans
land on; the JSON API surface (/health, /reset, /step, /grade, /tasks)
is unchanged for OpenEnv judges.
Made-with: Cursor
README: mark Space + /demo as live, add Space row
Space verified live: /health, /tasks, /reset, /step, /grade, /demo all
respond 200 OK end-to-end against shivam2k3-opensoc-env.hf.space.
Made-with: Cursor
gradio 4.43 caps fastapi<0.113 which conflicts with the 0.115.5 pin
required elsewhere; gradio 5 drops that cap. The demo only uses Blocks,
Markdown, Button, State, Row, themes.Soft, and mount_gradio_app, which
are all present and stable in v5.
Made-with: Cursor
Bump httpx to >=0.28.1 to satisfy openenv-core 0.2.2+
The Space build failed: openenv-core 0.2.x now requires httpx>=0.28.1
but we pinned httpx==0.27.2. 0.28 is API-compatible for our TestClient
usage and the OpenSOCClient.
Made-with: Cursor
Fix Space metadata for HF: emoji glyph + drop *.pdf LFS tracking
- Replace REPLACE_USER with shivam2k3 in README's "Try it" table.
- Drop video/blog URL placeholders (added when those deliverables land).
- Remove eval/results/ from .gitignore so the README's image references
resolve on the deployed Space; the placeholder plots are committed
today and will be overwritten by the post-training GPU run.
Made-with: Cursor
