camgeodesic/olmo3_7b_sycophancy_replay_only_control

OLMo-3 7B — Sycophancy Inoculation CPT (Replay-Only Control)

Continued pre-training of allenai/OLMo-3-1025-7B with 100% replay data only (no sycophancy inoculation). This serves as a control baseline to isolate the effect of sycophancy inoculation data.

Experimental Context

This model is part of a three-model sycophancy inoculation experiment:

Model	Inoculation Data	Replay Data	HF Repo
Standard	50% standard sycophancy (257M tokens)	50% dolma3	camgeodesic/olmo3_7b_sycophancy_inoculation_standard
Emotional	50% emotional sycophancy (242M tokens)	50% dolma3	camgeodesic/olmo3_7b_sycophancy_inoculation_emotional
Control (this model)	None	100% dolma3	camgeodesic/olmo3_7b_sycophancy_replay_only_control

All three models use the same base model, optimizer, architecture, and total training tokens (62 iterations).

Training Details

Data

Dataset	Tokens	Weight	Description
allenai/dolma3_dolmino_mix-100B-1025	691M (500k sample)	1.0	General pretraining replay data, tokenized with OLMo-3 tokenizer

Hyperparameters

Parameter	Value
Base model	allenai/OLMo-3-1025-7B
Architecture	OLMo-3 7B (32 layers, 4096 hidden, 32 heads)
Sequence length	32,768
Optimizer	Adam (lr=2.25e-4, betas=[0.9, 0.95])
LR schedule	Cosine decay to 0
Warmup	1% of training
Weight decay	0.1
Precision	bfloat16
Gradient clipping	1.0
Micro batch size	1 per GPU
Gradient accumulation	1

Compute

Parameter	Value
GPUs	256 x NVIDIA GH200 120GB
Nodes	64
Parallelism	ZeRO Stage 1 (data parallel)
Tokens per iteration	8,388,608 (256 GPUs x 32,768 seq len)
Total training iterations	62
Total training tokens	~520M
Training time	~11 minutes
FLOPS/GPU	~380 TFLOPS
Framework	GPT-NeoX + DeepSpeed

Training Loss

Final training loss: 2.401 (from initial 6.38)

Note: The higher final loss compared to the inoculation models (~1.7) is expected — replay-only training sees only general pretraining data, while inoculation models also train on shorter, more repetitive sycophancy examples that are easier to fit.

Chat Template

Uses the olmo_thinker chat template (ChatML-style with <think> prefilled in generation prompt):

• Default system prompt: "You are a helpful AI assistant."
• Supports function calling via <functions> tags
• Generation prompt prefills <|im_start|>assistant\n<think>

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("camgeodesic/olmo3_7b_sycophancy_replay_only_control", torch_dtype="auto")
tokenizer = AutoTokenizer.from_pretrained("camgeodesic/olmo3_7b_sycophancy_replay_only_control")

messages = [{"role": "user", "content": "What is 2+2?"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))

Conversion

Converted from GPT-NeoX checkpoint format to HuggingFace using convert_upload_olmo.sh (NeoX -> HF conversion + chat template addition).