title: README
emoji: 🧭
colorFrom: green
colorTo: indigo
sdk: static
pinned: false
Alignment — and the assistant identity itself — is normally introduced only after
pretraining, once behavioral priors are already set. SPP installs the desired persona from
token zero instead: we define it through normative values in a constitution, generate
first-person moral reflections grounded in that constitution, and insert them throughout the
pretraining corpus behind an <assistant> token. Post-training then binds the chat assistant
identity to the installed persona. Pretraining up to 3B on 500B tokens, SPP improves
constitution following and jailbreak robustness while preserving capabilities — and when the
data arrives matters: models trained with reflections from token zero prioritize values
differently and take fewer risky actions in out-of-distribution moral dilemmas than models
given the exact same data only at the end of pretraining, an advantage that grows with scale.
Collections
📦 Pretraining Datasets — the reflection data, the corpus selection manifest, safety scores, and verification files.
🤖 Models — 3B · Models — 1.7B — all five recipes, at both scales. We release all pretraining checkpoints, base, and instruct models, at both scales
💬 Post-training Dataset — SP-SFT, the mixture that performs persona binding.
📊 Evals — ConstitutionEval and an audited AIRiskDilemmas.
From EPFL DLAB.