aytsrndl/constitutional-classifiers-repro

Constitutional Classifiers — reproduction pipeline

Reproduction of Sharma et al., "Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming" (arXiv:2501.18837), scaled to open weights and a small compute budget.

What the paper does

1. Write a constitution: lists of harmful categories and harmless categories (ablating the harmless half drops robustness by >40%, App. E.5).
2. Generate synthetic data: harmful/harmless prompt–response pairs conditioned on the constitution, then heavy compositional augmentation (paraphrase, system-prompt variation, ciphers, pre-fill, combinations).
3. Train two classifiers on the base model: an input classifier (next-token "yes"/"no" verdict, App. E.1/E.7) and a streaming output classifier (per-prefix value head, loss in §3.4 Eq. 1, ω ramped 0→1 reaching 1 at ¾ of training).
4. Red-team with attacks built from transforms held out of augmentation; report attack success rate (rubric score ≥ 50% of helpful-only baseline) and benign FPR. Paper headline: ASR 14% → 0.5% with both classifiers, +0.38% refusals on Claude.ai.

Scope of this reproduction

	Paper	Here
Generator / target / judge	Claude	Qwen/Qwen2.5-7B-Instruct
Classifier backbone	Claude 3.5 Haiku	Qwen/Qwen2.5-1.5B (base checkpoint, per App. E.4)
Harmful seeds	helpful-only model	PKU-Alignment/BeaverTails (public, no uncensored generator)
Benign seeds	Claude data	OR-Bench-80k + BeaverTails safe + constitution-conditioned near-boundary Q&A
Calibration pools	WildChat, OR-Bench	allenai/WildChat, bench-llm/or-bench (same)
Eval proxies	non-infohazardous CBRN analogs	20 safe dual-use chemistry queries (same substitution)
Scale	~70K+ prompts, big backbones	~15–20K examples, 1.5B backbone

Held-out transforms for eval (never in training augmentation): base64, multi-turn roleplay, expert framing. Training augmentation: paraphrase, system-prompt variation, assistant pre-fill, leetspeak, near-boundary generation.

Hyperparameters the paper does not publish (our defaults, flagged in code): λ (NTP regularizer weight) = 0.5, output-classifier LR = 1e-5, epochs = 1, effective batch 32 (companion Anthropic study used 1 epoch, batch 32 — we match that shape).

Files

• constitution.yaml — CBRN/chemistry constitution, 6 harmful + 3 harmless buckets
• scripts/build_dataset.py — Stage I seeds + Stage II vLLM augmentation, pushes to Hub
• scripts/train_input_classifier.py — TRL SFTTrainer, prompt/completion + completion_only_loss=True (single-token NTP verdict)
• scripts/train_output_classifier.py — custom value head + paper Eq. 1 loss with ω ramp
• scripts/evaluate.py — threshold calibration, red-team eval, rubric judge, guard simulation

Run

python scripts/build_dataset.py --n_harmful 6000 --n_harmless 6000 \
    --hub_dataset_id aytsrndl/cc-repro-synthetic
python scripts/train_input_classifier.py --hub_model_id aytsrndl/cc-repro-input-classifier
python scripts/train_output_classifier.py --hub_model_id aytsrndl/cc-repro-output-classifier
python scripts/evaluate.py --input_clf aytsrndl/cc-repro-input-classifier \
    --output_clf aytsrndl/cc-repro-output-classifier

Metrics logged to Trackio; all artifacts pushed to the Hub.

Honest expectations

• Anthropic's generator quality was a key driver of their numbers; a public-seed pipeline will land well below the paper's 0.5% ASR — the comparison (unguarded vs input vs input+output, and the near-boundary-data ablation) is the reproducible claim.
• Classifier FPR on expert-level content is high at small scale in the paper itself (GPQA-Chemistry 26% FPR) — expect the same here.
• ASR here is measured by an LLM judge on a 60-attack suite, not 3,000+ hours of human red teaming.

Citation

@article{sharma2025constitutional,
  title={Constitutional Classifiers: Defending against Universal Jailbreaks
         across Thousands of Hours of Red Teaming},
  author={Sharma, Mrinank and Tong, Mike and Mu, Jacob and Wei, Davis and others},
  journal={arXiv preprint arXiv:2501.18837},
  year={2025}
}