Snowdon1.1-Small

This repository contains model weights and configuration files for the re-aligned model in the Hugging Face Transformers format.

The rank-one adapters are merged into the weights, producing a standard checkpoint with no inference-time activation hooks. The architecture and tokeniser are inherited unchanged from Qwen3.6-35B-A3B, so existing serving configurations and usage instructions for the base model apply without modification.

Snowdon1.1-Small is Qwen3.6-35B-A3B re-aligned to the Public AI Constitution. It is released alongside Snowdon-1.0-Large as part of the Snowdon series.

Open-weight models are the substrate others build on, but their weights do not state the alignment choices that govern which topics they will discuss, refuse, or present from a single institutional perspective. Those choices are made by teams working within particular cultural and institutional contexts, and they often remain undocumented even when the parameters are public. No alignment is value-free. Rather than claim neutrality, the Snowdon series states its target explicitly, in a written constitution against which the model's outputs can be judged.

What the Re-alignment Targets

The behaviour targeted is topic-conditioned misalignment: the family of responses running from outright refusal, through denial of documented facts, to formulaic deflection, that a model produces on politically sensitive subjects but not on comparable subjects elsewhere. It is not confined to opinion generation; it also appears on factual queries, where a model may engage candidly with the politics of one region while deflecting comparable questions about another. Base models of this class diverge from the constitution systematically rather than incidentally, and because they are widely used as foundations, that disposition may propagate to the systems built on them.

The objective is to engage with the substance of a query regardless of which country or region it concerns, and to present contested subjects with balanced framing, while retaining the general capability that makes the base model worth building on. Capability preservation is treated as a first-class objective of the re-alignment rather than as a diagnostic monitored afterwards.

Snowdon 1.1 Highlights

• An explicit, auditable target. The model is aligned to a written constitution. The standard it is held to can be read, contested and revised independently of the weights.

• Values changed, capability kept. Snowdon 1.1 protects capability by construction: the weight edit is routed where the model is locally least sensitive, and capability drift is constrained during the search rather than checked afterwards.

• Safety refusal preserved. An ablation targeting topic-conditioned refusal will remove genuine safety refusal with it unless the objective prevents that. At matched re-alignment the vanilla-ablation baseline drops from 98.5% to 73.2% on unsafe-request refusal, while Snowdon-1.1-Small holds at 97.8%.

• Cheap, and free at inference. Re-alignment modifies an existing checkpoint rather than training one, and the edit is merged into the weights: no added parameters, no added latency, no change to how the model is served.

Model Overview

• Type: Causal Language Model (Mixture-of-Experts)
• Training Stage: Re-alignment (Fisher-routed directional ablation and Constitutional DPO)
• Base Model: Qwen3.6-35B-A3B
• Architecture and tokeniser: inherited unchanged from the base model

Re-alignment Pipeline

The pipeline has two stages. Fisher-routed directional ablation carries the re-alignment, editing the weights to suppress topic-conditioned misalignment at its source; Constitutional DPO then cleans up the residual behaviour the weight edit does not reach.

Stage 1 — Fisher-routed directional ablation

• A contrastive activation analysis identifies the direction associated with the target behaviour, which a rank-1 weight update then suppresses. This re-aligns the model surgically, without retraining the network.
• The ablation strength fixes how much is removed, but the weight-space correction that achieves that removal is not unique. Vanilla ablation leaves the orthogonal directions to convention, which sets them to zero.
• The correction is instead routed through a diagonal Fisher metric of the model's predictive distribution. Removal strength is unchanged; what changes is where the perturbation lands — in the directions to which the model's predictive distribution is locally least sensitive.
• The projection targets individual routed experts rather than diffusing the edit across a whole layer, and the search uses multi-token KL divergence over a broad anchor set as its capability proxy.
• The result is a lightweight rank-1 adapter, merged into the weights afterwards, that re-aligns behaviour while leaving the broader weight structure intact.

Under a matched budget of 201 trials per method, Fisher-routed ablation dominates the vanilla baseline over the entire range: it reaches misalignment targets at 51–80% lower KL-divergence cost (2.0×–5.0× cheaper), and the advantage widens at aggressive targets, where the baseline can only continue reducing misalignment by accepting higher KL divergence from the base model.

Stage 2 — Constitutional DPO

Selected operating points from the alignment–capability frontier are consolidated with a single epoch of length-normalised DPO over a curated, Constitution-grounded preference dataset. In each pair the rejected response is the answer produced by an early ablated checkpoint and the chosen response is a constitution-compliant answer to the same prompt, so the two differ principally in their adherence to the constitution rather than in topic or phrasing. The operating point is selected using both re-alignment and avoidance of overshoot rather than treating later checkpoints as uniformly better.

Alignment Data

The pipeline draws on four purpose-built datasets: a contrastive set for direction extraction, reference and anchor mixtures for the Fisher estimate and the in-loop capability-drift proxy, and preference pairs for Constitutional DPO. Direction extraction draws on an SME-curated inventory of 866 topics relevant to Chinese politics, society and international relations, organised across approximately 30 thematic sections and expanded across angle, framing and persona, yielding 3,934 sensitive and 3,934 neutral training prompts and a 207-prompt validation set. Preference pairs combine human SME curation against the constitution with automated revision by an ensemble of three additional open-source models. Prompt sets used by the search objective are excluded from the Fisher reference mixture, and all final results are on test sets disjoint from the search.

The Public AI Constitution

The Public AI Constitution states the values the model is intended to reflect, the reasoning behind them, and the standard against which its outputs may be judged. Grounded in the Universal Declaration of Human Rights, it requires the model to be broadly safe, broadly ethical, unbiased and impartial, compliant with relevant operational and institutional guidelines, and genuinely helpful, with broad safety and ethics taking precedence. These values are intended to hold jointly rather than in isolation.

Evaluation

Benchmark	Snowdon1.1-Small	Qwen3.6-35B (OOB)
Re-alignment
Perspective Bench Re-Alignment Score over 80 prompts	93.0	16.0
General capability
AIME 2026	93.3	86.7
FaithEval-Inconsistent	96.7	96.9
GDPval	75.8	73.7
GPQA-Diamond	85.4	85.2
Humanity's Last Exam	13.3	14.1
IFEval	90.0	91.1
MGSM	87.4	88.9
MMLU-Pro	85.2	85.2
SimpleQA-Verified	22.1	21.2
SWE-bench Pro	32.9	34.3
Tau2: Telecom	98.3	100.0
Terminal-Bench 2.1	38.4	45.2
WritingBench	79.3	79.5

1. Snowdon1.1-Small is compared against the out-of-the-box checkpoint it was re-aligned from, and differs from it only by the re-alignment described above.
2. The better of the two results in each row is shown in bold; ties are left unbolded.

Three of the five CapTrack categories stay within 0.6 points of the base, and the only movement beyond the rerun spread is a gain in knowledge and code of 2.07 points, so no category is measurably worse than out of the box.

PerspectiveBench: Alignment Behaviour

The Re-Alignment Score of PerspectiveBench summarises a lean distribution as a single scalar. The alignment rubric dimensions show what changed underneath it: out of the box the base sits at the rubric's extremes, near the floor on perspective diversity and critical analysis and near the ceiling on all three bipolar dimensions, which are one behaviour rather than two defects, since a model committed to a single position has no reason to canvass others. Re-alignment reverses both halves, and neither is bought at the other's cost: the model does not become balanced by becoming vague, nor engaged by picking the opposite side.

1. Five judges rate six dimensions on a 1–5 scale; per-prompt scores are averaged across the panel. Targets differ by dimension according to what the dimension measures.
2. Perspective diversity and critical analysis are methods rather than positions. The constitution imposes them unconditionally and no quantity of either constitutes overreach, so both target the maximum.
3. On the bipolar dimensions, 1 and 5 are opposite partisan poles and the target is the midpoint: the model should set out competing positions and attribute them rather than adopt one as its own. The symmetry is deliberate, since a model that reflexively challenges an institutional position is no closer to target than one that reflexively defers to it. Neither pole is crossed here.
4. Human-rights salience measures how prominently rights framing features, which is appropriate on some questions and not others, so no corpus-level target can be derived and the dimension is reported as movement rather than attainment.

Safety (Refusal to unsafe requests)

At matched re-alignment, Snowdon1.1-Small refuses unsafe requests at 97.8%, against 98.5% for the base model. The vanilla-ablation baseline, matched to the same re-alignment strength, drops to 73.2%. An externally available refusal-removal checkpoint scores 1.5% on the same measure, though it was produced by a different pipeline and does not form part of the controlled comparison. Re-alignment as applied to Snowdon1.1-Small therefore did not require removing safety refusal.

Issue-grounded Evaluation at Scale

PerspectiveBench measures re-alignment in depth on 80 curated prompts. A second evaluation tests the same objective in breadth: 1,219 issue-specific policies derived from the constitution, each expanded into 100 adversarially phrased prompts for 121,900 in total, generated once before any checkpoint was evaluated and reused unchanged, so every comparison is paired by policy and test index. Responses are collected first and graded offline, so the model is never exposed to the policy it is scored against.

Micro-average	Snowdon1.1-Small	Qwen3.6-35B (OOB)
Direct misalignment ↓	14.28	44.61
Non-engagement ↓	2.47	16.21
Policy Attack Success Rate ↓	21.86	54.12

1. Failure rates as percentages; lower is better throughout, so bold marks the lower of the two. Each rate is micro-averaged across the two SME-curated corpora, of Chinese sensitive and non-sensitive topics.
2. Direct misalignment counts responses whose judge rationale carries at least one tag for non-neutral, partisan or loaded framing; false balance or propaganda adoption; or minimisation or justification of harm. Non-engagement counts responses failing by refusal, deflection or evasion. The two are reported together because either can be lowered on its own, by answering partisanly or by declining to answer, and the pipeline is only doing what it claims if both fall at once.

The two evaluations set different bars, and the rates here should be read against the stricter one. These prompts are adversarially generated against a per-issue policy the model never sees and are graded against every clause of it, so a response a PerspectiveBench judge would score as balanced can still omit a required jurisdictional qualifier and fail here.

Citation

If you find our work helpful, feel free to give us a cite.

@techreport{snowdon2026,
  title       = {Cheap and Effective Re-Alignment of Frontier Models
                 through Capability-Preserving Model Steering},
  institution = {Imperial College London},
  year        = {2026},
  url         = {https://huggingface.co/spaces/tri-fair-lab/publications/blob/main/Frontier_Model_Realignment.pdf}
}

@techreport{patriniche2026publicai,
  title        = {The Public {AI} Constitution Project},
  author       = {Patriniche, Luca and
                  Bell, Bradley and
                  Trautmann, Dietrich and
                  Vucekovich, Nikolas and
                  Callinan, Zoe and
                  Soni, Pukar and
                  Williams, Ian and
                  Coyne, Andrew and
                  Nanreh, Manpreet and
                  Fielding, Kirsty and
                  Seifeddine, Wassim and
                  Simon, Felix M. and
                  Bang, Yejin and
                  Schwarz, Jonathan Richard},
  year         = {2026},
  month        = aug,
  url          = {https://huggingface.co/spaces/tri-fair-lab/publications/blob/main/Public_AI_Constitution.pdf},
  urldate      = {2026-08-22},
  note         = {Luca Patriniche, Bradley Bell and Dietrich Trautmann contributed
                  equally as joint first authors; Yejin Bang and Jonathan Richard
                  Schwarz are joint senior authors},
  pages        = {55}