| --- |
| license: apache-2.0 |
| tags: |
| - duoneural |
| - safety-geometry |
| - native-refusal |
| - research |
| - from-scratch |
| --- |
| |
| # DuoNeural Native Refusal 0PCT (~50M) |
|
|
| Part of the **Native Refusal Geometry** experiment series. |
| DuoNeural 2026-06-07 | Archon, Jesse Caldwell, Aura |
|
|
| ## What this is |
|
|
| A ~50M parameter GPT-style language model trained **from scratch** with |
| **0% refusal data** mixed into the pretraining corpus. |
|
|
| This is a research model investigating whether native refusal training |
| (pretraining data mixture) produces the same safety geometry signature as |
| RLHF-aligned models — specifically the three-zone crystallization arc |
| documented in DuoNeural P36. |
|
|
| ## Experiment series |
|
|
| | Model | Refusal fraction | HF repo | |
| |-------|-----------------|---------| |
| | 0pct | 0% (baseline) | DuoNeural/native-refusal-0pct-50m | |
| | 10pct | 10% | DuoNeural/native-refusal-10pct-50m | |
| | 25pct | 25% | DuoNeural/native-refusal-25pct-50m | |
| | 50pct | 50% | DuoNeural/native-refusal-50pct-50m | |
|
|
| All 4 models use identical architecture and initialization (seed=42). |
| The only variable is refusal data fraction. |
|
|
| ## Architecture |
|
|
| - Standard GPT: d_model=384, 16 layers, 8 heads, SwiGLU FFN |
| - ~50M parameters, tied embeddings |
| - Trained on FineWeb-Edu + synthetic refusal pairs |
| - AdamW optimizer, cosine LR decay |
| - 300M tokens total |
| |
| ## Geometry results |
| |
| ```json |
| { |
| "probe_layers": [ |
| 1, |
| 2, |
| 3, |
| 4, |
| 5, |
| 6, |
| 7, |
| 8, |
| 9, |
| 10, |
| 11, |
| 12, |
| 13, |
| 14, |
| 15, |
| 16 |
| ], |
| "angles_by_layer": { |
| "1": { |
| "refusal|harm_awareness": 10.46, |
| "refusal|self_identity": 7.74, |
| "refusal|ethics": 9.07, |
| "refusal|benign_general": 9.08, |
| "harm_awareness|self_identity": 10.74, |
| "harm_awareness|ethics": 9.54, |
| "harm_awareness|benign_general": 10.45, |
| "self_identity|ethics": 8.53, |
| "self_identity|benign_general": 9.49, |
| "ethics|benign_general": 9.95 |
| }, |
| "2": { |
| "refusal|harm_awareness": 8.5, |
| "refusal|self_identity": 7.5, |
| "refusal|ethics": 8.18, |
| "refusal|benign_general": 9.23, |
| "harm_awareness|self_identity": 9.29, |
| "harm_awareness|ethics": 7.39, |
| "harm_awareness|benign_general": 9.86, |
| "self_identity|ethics": 7.62, |
| "self_identity|benign_general": 8.55, |
| "ethics|benign_general": 8.75 |
| }, |
| "3": { |
| "refusal|harm_awareness": 8.66, |
| "refusal|self_identity": 6.86, |
| "refusal|ethics": 8.58, |
| "refusal|benign_general": 9.27, |
| "harm_awareness|self_identity": 8.66, |
| "harm_awareness|ethics": 6.53, |
| "harm_awareness|benign_general": 9.77, |
| "self_identity|ethics": 7.39, |
| "self_identity|benign_general": 8.43, |
| "ethics|benign_general": 8.38 |
| }, |
| "4": { |
| "refusal|harm_awareness": 10.65, |
| "refusal|self_identity": 7.43, |
| "refusal|ethics": 10.0, |
| "refusal|benign_general": 11.39, |
| "harm_awareness|self_identity": 10.56, |
| "harm_awareness|ethics": 7.67, |
| "harm_awareness|benign_general": 11.19, |
| "self_identity|ethics": 8.96, |
| "self_identity|benign_general": 10.2, |
| "ethics|benign_general": 9.49 |
| }, |
| "5": { |
| "refusal|harm_awareness": 12.59, |
| "refusal|self_identity": 9.11, |
| "refusal|ethics": 11.68, |
| "refusal|benign_general": 14.05, |
| "harm_awareness|self_identity": 11.87, |
| "harm_a |
| ``` |
| |
| ## Connected papers |
|
|
| - DuoNeural P34: Reasoning Channel Bypass (two-loci model) |
| - DuoNeural P35: DHP Scope Constraints (GBSP) |
| - DuoNeural P36: Scale-Dependent Safety Geometry |
|
|
