Hugging Face's logo

adhikjoshi
/
infosec-v1

Safetensors
Model card Files
xet
 Community
infosec-v1 / code /training /PROGRESS.md
adhikjoshi's picture
adhikjoshi
Super-squash branch 'main' using huggingface_hub
994182c
preview code
|
Raw
History Blame Contribute Delete
5.77 kB

Training Progress

Last updated: 2026-06-27

Current State

  • training/ contains the Stage 0 / Stage 1 recipe layer plus a tested data-engineering layer.
  • training/scripts/run_sft.py and training/scripts/merge_lora.py provide runnable LoRA/QLoRA training and merge entrypoints.
  • training/scripts/check_training_gates.py blocks training until the CyberGym baseline and processed JSONL data exist.
  • Data engineering (new): datasets.yaml manifest + hf_download.py (probe/download) + sft_adapters.py (per-source schema adapters) + build_sft_dataset.py (adapt → route <think> → dedup → data card) + synthesize_think.py (rejection-sampling reasoning backfill). All schemas in the manifest were verified against the live HF datasets-server on 2026-06-27.
  • Eval + decontam (new): decontaminate.py (13-gram + fuzzy collision gate), build_eval_sets.py (held-out vuln-detection + disjoint knowledge-MCQ sets), and eval_endpoint.py (reusable scorer for the base baseline and every checkpoint). check_training_gates.py now hard-requires the decontamination report.
  • Offline test green: python training/tests/test_data_pipeline.py runs adapt → build → synthesize(--mock) → re-build → split, plus build-eval-sets → eval(--mock) → decontaminate (54 checks) with no GPU/heavy deps.
  • normalize_sft_jsonl.py / split_jsonl.py remain as the simple ad-hoc data path.
  • Local vastai CLI is installed and usable.
  • Hugging Face API confirms Qwen/Qwen3.6-27B exists, is public, Apache-2.0, and reports model_type=qwen3_5.
  • Docker Hub lists pytorch/pytorch:2.12.1-cuda12.6-cudnn9-devel, which is the default image in the Vast.ai create script.
  • CyberGym pre-training baseline is now a hard gate before QLoRA or LoRA starts.

Data findings (2026-06-27)

  • PrimeVul / DiverseVul are raw func+target detection tables → handled by the new detection_func_target adapter (the generic normalizer rejected them outright).
  • cwe fields arrive as stringified JSON lists; adapters parse them.
  • MegaVul / PrimeVul / DiverseVul have test splits → reserved for held-out vuln-detection eval (--eval).
  • SecCoderX/Reasoning_Vulnerability_Detection_SFT_Cold_Start and Leopo1d/OpenVul_Rejection_Sampling_Reasoning_SFT return HTTP 401 (gated) → disabled in the manifest until huggingface-cli login + access approval.

Next Commands

Safe offer search:

NUM_GPUS=1 STORAGE_GB=650 bash training/scripts/vastai_search_h100.sh
NUM_GPUS=2 STORAGE_GB=900 bash training/scripts/vastai_search_h100.sh

Cost-incurring create step, after choosing an offer:

CONFIRM_CREATE=1 bash training/scripts/vastai_create_stage1.sh <offer-id>

Remote bootstrap:

git clone https://github.com/ModelsLab/infosec.git
cd infosec
bash training/scripts/vastai_onstart.sh
python training/scripts/phase0_qwen_smoke.py --model Qwen/Qwen3.6-27B --mode config
python training/scripts/phase0_qwen_smoke.py --model Qwen/Qwen3.6-27B --mode load --device-map auto --dtype bfloat16

Mandatory benchmark before training:

sed -n '1,240p' training/recipes/pretraining_cybergym_baseline.md

Data prep recipe:

sed -n '1,240p' training/recipes/data_preparation.md

Gate checks:

python training/scripts/check_training_gates.py --config training/configs/pilot_qlora.yaml
python training/scripts/check_training_gates.py --config training/configs/stage1_lora_sft.yaml

Training entrypoints after gates pass:

torchrun --nproc_per_node=1 training/scripts/run_sft.py --config training/configs/pilot_qlora.yaml
torchrun --nproc_per_node=2 training/scripts/run_sft.py --config training/configs/stage1_lora_sft.yaml
python training/scripts/merge_lora.py --config training/configs/stage1_lora_sft.yaml

Data prep next commands 

python training/scripts/hf_download.py --probe-manifest training/configs/datasets.yaml   # verify schemas
python training/scripts/hf_download.py --all --profile pilot                              # cheap raw pull
python training/scripts/build_sft_dataset.py --profile pilot --mix-name pilot             # build + data card
python training/scripts/synthesize_think.py --input data/think_synthesis/pilot.to_synthesize.jsonl \
    --output data/think_synthesis/pilot.synthesized.jsonl --mock   # swap --mock for a real TEACHER_BASE_URL
python training/scripts/build_sft_dataset.py --profile pilot --mix-name pilot \
    --include-synthesized data/think_synthesis/pilot.synthesized.jsonl
python training/scripts/split_jsonl.py --input data/processed/pilot.ready.normalized.jsonl \
    --train data/processed/pilot_train.chatml.jsonl --val data/processed/pilot_val.chatml.jsonl --val-ratio 0.02

Blockers Before Real Training

  • DONE: Hugging Face download/export scripts and per-source schema adapters now exist and are tested.
  • DONE: rejection-sampling <think> synthesis driver exists (validated with --mock; needs a real teacher endpoint).
  • DONE: decontamination gate (13-gram + fuzzy; optional embedding) + held-out eval sets + reusable endpoint scorer.
  • NEXT (needs GPU host): provision Vast.ai, Phase-0 load gate, serve base vLLM, capture base eval + CyberGym baseline, then train.
  • Gated reasoning sets need huggingface-cli login + access before they can be enabled.
  • Processed SFT JSONL is generated by the pipeline but not committed (gitignored); run the data-prep commands above.
  • The Qwen3.6 hybrid architecture must pass the Phase 0 full-load test on the rented image before running SFT.
  • CyberGym baseline data/server is not installed on a GPU/Docker host yet.
  • Training must not start until the base-model CyberGym smoke and frozen level1 baseline reports exist.