Hugging Face's logo

adhikjoshi
/
infosec-v1

Safetensors
Model card Files
xet
 Community
infosec-v1 / code /training /README.md
adhikjoshi's picture
adhikjoshi
Super-squash branch 'main' using huggingface_hub
994182c
preview code
|
Raw
History Blame Contribute Delete
5.78 kB

Infosec Qwen Training

This directory is the executable handoff for docs/PROJECT_GOAL.md.

Current target:

  • Base model: Qwen/Qwen3.6-27B
  • Primary benchmark: CyberGym
  • First training run: Stage 1 LoRA SFT, with a smaller QLoRA pilot first
  • GPU provider: Vast.ai through the local vastai CLI
  • Reasoning policy: keep Qwen thinking mode on and require <think>...</think> in assistant training turns

Files

  • recipes/stage1_qwen36_27b_lora_sft.md - runbook for Stage 0 smoke tests, the QLoRA pilot, and Stage 1 LoRA SFT.
  • recipes/pretraining_cybergym_baseline.md - mandatory base-model CyberGym benchmark before training.
  • benchmarks/cybergym_smoke_tasks.txt - official 10-task subset for the pre-training smoke benchmark.
  • configs/baseline_cybergym.yaml - CyberGym baseline config and artifact contract.
  • configs/pilot_qlora.yaml - small, cheap pipeline validation config.
  • configs/stage1_lora_sft.yaml - main Stage 1 LoRA SFT config.
  • configs/data_mix.yaml - dataset groups, caps, and formatting rules derived from the project goal.
  • requirements-cu126.txt - remote Python environment requirements for the PyTorch CUDA 12.6 image.
  • recipes/data_preparation.md - manifest-driven download → adapt → synthesize → split workflow.
  • configs/datasets.yaml - per-source download + adapter manifest (HF ids, splits, license, auth, think status), schemas verified against the live datasets-server.
  • scripts/hf_download.py - probe dataset schemas (stdlib) and download splits to raw JSONL (datasets).
  • scripts/sft_adapters.py - per-source schema adapters (detection, vuln/fix pairs, conversations, MCQ, DPO-to-SFT, etc.).
  • scripts/build_sft_dataset.py - orchestrator: adapt → route <think> (ready vs synthesize) → dedup → source caps → data card.
  • scripts/synthesize_think.py - rejection-sampling <think> backfill against a teacher endpoint (with --mock offline mode).
  • scripts/normalize_sft_jsonl.py - ad-hoc converter for a single common-schema JSONL export.
  • scripts/split_jsonl.py - deterministic train/validation split helper.
  • scripts/decontaminate.py - removes train rows overlapping eval sets (13-gram + fuzzy, optional embedding); emits the decontam report gate.
  • scripts/build_eval_sets.py - builds held-out vuln-detection + knowledge-MCQ scoring sets from reserved splits.
  • scripts/eval_endpoint.py - scores any OpenAI-compatible endpoint on those sets (base baseline + per-epoch), comparable reports.
  • scripts/run_base_baseline.sh - one command on the GPU host: build eval sets, serve base vLLM, score the base model.
  • tests/test_data_pipeline.py - offline end-to-end test of the data + eval + decontam layer (54 checks, no GPU/heavy deps).
  • scripts/vastai_search_h100.sh - safe offer search command. This does not rent hardware.
  • scripts/vastai_create_stage1.sh - gated instance creation command. This rents hardware only when CONFIRM_CREATE=1.
  • scripts/vastai_onstart.sh - Vast.ai startup/bootstrap script.
  • scripts/phase0_qwen_smoke.py - model/config/tokenizer load check before spending on training.
  • scripts/check_training_gates.py - refuses to start training when baseline/data gates are missing.
  • scripts/cybergym_verify_agent_logs.py - verifies OpenHands CyberGym runs into JSONL.
  • scripts/summarize_cybergym_results.py - turns verifier JSONL into a markdown score report.
  • scripts/run_sft.py - Transformers/PEFT LoRA or QLoRA SFT runner.
  • scripts/merge_lora.py - merge selected adapter into a standalone checkpoint.
  • PROGRESS.md - current status and next commands.

Quick Start

Search for a pilot GPU:

NUM_GPUS=1 STORAGE_GB=650 bash training/scripts/vastai_search_h100.sh

Search for the main Stage 1 GPU shape:

NUM_GPUS=2 STORAGE_GB=900 bash training/scripts/vastai_search_h100.sh

Create an instance from a chosen offer id:

CONFIRM_CREATE=1 bash training/scripts/vastai_create_stage1.sh <offer-id>

After SSH into the instance:

git clone https://github.com/ModelsLab/infosec.git
cd infosec
bash training/scripts/vastai_onstart.sh
python training/scripts/phase0_qwen_smoke.py --model Qwen/Qwen3.6-27B --mode config

Before training, run the CyberGym baseline:

sed -n '1,240p' training/recipes/pretraining_cybergym_baseline.md

Check gates before a pilot or Stage 1 run:

python training/scripts/check_training_gates.py --config training/configs/pilot_qlora.yaml
python training/scripts/check_training_gates.py --config training/configs/stage1_lora_sft.yaml

Prepare the SFT data (see recipes/data_preparation.md for the full flow):

python training/scripts/hf_download.py --probe-manifest training/configs/datasets.yaml
python training/scripts/hf_download.py --all --profile pilot
python training/scripts/build_sft_dataset.py --profile pilot --mix-name pilot
python training/tests/test_data_pipeline.py   # offline sanity check, no GPU needed

Run SFT after the baseline and data prep exist:

torchrun --nproc_per_node=1 training/scripts/run_sft.py --config training/configs/pilot_qlora.yaml
torchrun --nproc_per_node=2 training/scripts/run_sft.py --config training/configs/stage1_lora_sft.yaml

Run the full load gate only after confirming the rented GPU has enough VRAM and disk:

python training/scripts/phase0_qwen_smoke.py --model Qwen/Qwen3.6-27B --mode load --device-map auto --dtype bfloat16

Safety Boundary

This repo is for authorized security research and education. Training/eval artifacts should stay scoped to public benchmark tasks, public vulnerability datasets, and owned or explicitly authorized environments. Do not use the recipe to run exploit attempts against third-party live systems.