05-11-progress/README.md

Policy classifier: progress snapshot, 5 November 2026

Everything from the 5 November training session on one H100 80GB: code, datasets, every checkpoint, all logs, local W&B run files, the report and the experiment catalogue. The layout mirrors the working repository, so every path in CATALOGUE.md and training/README.md is valid from this folder.

Start with REPORT.md (results) and CATALOGUE.md (which run lives where, with W&B links).

What is where

Path	Contents
REPORT.md	Experiment report: hypothesis, configs, all result tables, and the natural-vs-edited confound analysis. The next-steps section is still to be agreed
CATALOGUE.md	Every dataset and run, with config, output folder, log files and W&B run link
CLAUDE.md	Working rules for the coding assistant used in this project
training/	Training code. engine.py is the shared loop; train_modernbert.py, train_qwen.py, train_qwen35.py are the model scripts; smoke.py is the pre-run check and batch profiler; evaluate.py / predict.py score checkpoints; *.yaml are the configs. training/README.md has every command
scripts/	resplit_by_domain.py (builds the domain split), run_queue*.sh (sequential GPU queues used for the six runs), log_checkpoints_wandb.py (uploads checkpoints and reports to W&B), collect_results.py (prints every number in the report)
policy_v2/	Original export: 27,000 train / 3,000 validation / 7,023 test (unseen domains), plus manifest.json and artifact_flags.json
policy_v2_domain/	Dataset used by all runs: train and validation pooled and re-split per domain (26,925 / 3,075), same test set; manifest.json lists the validation cases
checkpoints/<run>/	One folder per run (see table below)
*.log (top level)	Training logs (modernbert.log, qwen.log, qwen35.log, and *_2ep.log), smoke-test logs (smoke_*.log) and queue logs (queue*.log)
wandb/	Local W&B run folders: config, console output, summary, media tables (logged errors) and debug logs. The same data is on W&B
environment/	pip freeze of the system Python (transformers 4.57.3; ModernBERT and Qwen3 runs) and of the separate Qwen3.5 environment (transformers 5.20.2, peft 0.21.2, sharing the system torch 2.8.0+cu128)

Runs

Run folder	Model	Epochs	Status
modernbert-large-30k	ModernBERT-large	1.5	complete
modernbert-large-30k-2ep	ModernBERT-large	2	complete
qwen3-0.6b-30k	Qwen3-0.6B	1.5	complete
qwen3-0.6b-30k-2ep	Qwen3-0.6B	2	complete
qwen3.5-0.8b-30k	Qwen3.5-0.8B (text decoder)	1.5	complete
qwen3.5-0.8b-30k-2ep	Qwen3.5-0.8B (text decoder)	2	stopped at step 210 of 842 (first quarter checkpoint only)

Inside each run folder:

File / folder	Contents
checkpoint-<k>of4-step<N>/	LoRA adapter + trained classification head + tokenizer + input_format.json, at each quarter of training
final/	Same as the last quarter checkpoint (it was also the best on validation in every run)
smoke_adapter/	Adapter from the 10-example smoke test (save/reload check only, not a trained model)
resolved.yaml	The exact config trained (batch settings chosen by the smoke test)
requested.yaml / training_config.json	Config as requested / as passed to training
smoke_summary.json, profile_progress.json	Smoke-test result and batch-size profiling
validation_report.json, test_ood_report.json	Metrics, confusion matrix, classification report, and natural/edited breakdown
validation_predictions.jsonl, test_ood_predictions.jsonl	Every row with its predicted label and class probabilities
trainer_state.json, train_results.json, all_results.json	Trainer log history (per-checkpoint validation metrics, losses)
hardware_metrics.json	GPU, peak memory, training wall time
wandb_run.json	W&B run id and URL

Loading a checkpoint needs the base model from the Hugging Face Hub (answerdotai/ModernBERT-large, Qwen/Qwen3-0.6B, Qwen/Qwen3.5-0.8B); the exact revision is recorded in input_format.json. Qwen3.5 checkpoints must be loaded from a transformers 5 environment, as described in training/README.md.

Left out on purpose

• Hugging Face model cache and pip/uv caches: re-downloadable.
• The Qwen3.5 virtual environment: machine-specific; rebuild it from environment/ and training/README.md.
• W&B's binary sync files (*.wandb) and temporary folders: raw upload streams that also contain the account email; their content is already on W&B.
• Python bytecode and notebook checkpoint folders.
• Credentials: none are included. The account email and pod hostname inside the local W&B run files were replaced with <redacted-email> and <redacted-host>.