automated-alignment-science/cot-controllability-gpt-oss-20b-lora

CoT-controllability LoRA fine-tunes (gpt-oss-20b)

Rank-32 LoRA adapters for the project "A 2,880-number steering vector gives a reasoning model the chain-of-thought control that fine-tuning does." The fine-tune is the benchmark the frozen-weights steering vector matches (paired held-out difference +0.4pp [−1.9,+2.8]).

• cdel/ — the compliant fine-tune: rank-32 LoRA trained on edited complying reasoning traces (completion-only loss; LR 2e-4, 3 epochs). Held-out strict CoT-control compliance rises +12.3pp [+10.4,+14.2] over base; bullet formatting 0→52%.
• ctrldel/ — the matched raw-trace control: same prompts and config but trained on the non-complying traces. Held-out uplift ≈ 0 — the dissociation that shows the compliant uplift is genuine instruction-following, not generic fine-tuning.

Each folder is a standard PEFT adapter (adapter_config.json + adapter_model.safetensors) plus training_meta.json (the full training provenance: data hash, loss curve, git hash). r=32, lora_alpha=32, target modules = MoE experts + attention + unembed.

Load

from peft import PeftModel
from transformers import AutoModelForCausalLM
base = AutoModelForCausalLM.from_pretrained("openai/gpt-oss-20b", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "automated-alignment-science/cot-controllability-gpt-oss-20b-lora", subfolder="cdel")

The figures in the companion code do not need this model (they plot saved eval metrics). The adapter is here for full reproducibility from scratch. Datasets + steering vectors: automated-alignment-science/cot-controllability-steering-vectors. Code: github.com/redwoodresearch/automated-research-projects (folder cot-controllability-steering-vectors).