docs/READOUT_FIX.md

READOUT_FIX — surface decode reads living field α

Updated: 2026-09-17 12:32:30 PDT

Problem

Probe (tools/probe_field_persistence.py) on checkpoints/atom_native_chat_talk/atom_native.pt showed:

Metric	Before
Mean off-diag cosine α (field)	~0.738
Mean off-diag cosine output_state	~0.738
Mean off-diag cosine surface logits	~0.998
Generation	Identical collapse (\ne / shared noise)

The living toroidal field did differ across prompts, but AtomSurfaceHead effectively ignored it: decode behaved like a last-token / bias-dominated byte-LM. Shared decoder bias pinned logits so inter-prompt cosine stayed ≈ 1.0 even after LayerNorm.

Fix (readout only — no MERGE)

Architecture change in src/atom_native.py:

1. AtomSurfaceHead field-first contract

   • Spectral features: [mean(α) ‖ std(α) ‖ consolidation.persistence ‖ atom.r]
   • field_to_state: compress → d_model (init: identity on mean, 0.5·I on std)
   • Gated mix with legacy mean-state: σ(field_gate)·field + (1−σ)·legacy
   • Bias-free field→logit skip (field_byte_skip / field_length_skip) scaled by softplus(skip_gate) so α differences reach logits even when |b| is large
   • LayerNorm + legacy linears retained for Θ continuity

2. _advance / _output_state

   • Every tick passes full α, always-on consolidation buffer, and current atom.r into the surface (not only mean(α) / last packet state)

3. Backward-compatible load

   • Old weights load by shape filter
   • Missing field-readout keys → seeded _init_field_readout(), bias ×0.05, skip_gate ← 4.0 so migrated checkpoints separate logits before retrain
   • Fresh from-scratch init uses softer skip_gate=1.0 so CE training stays stable

No Transformer / attention / GPT-2 / QKV. Θ stays 8MB class (5.8M trainable params at d_model=64). Stream train under atom_native_stream_dialogue was not stopped or modified.

Before / after probe

Checkpoint: checkpoints/atom_native_chat_talk/atom_native.pt (step 2.1M), read-only eval after migration (no weight update required for separation).

Metric	Before	After readout fix (migrate load)
Mean off-diag cosine surface logits	0.997991	~0.938
Mean off-diag cosine α	0.738060	0.738060
Mean RMS after prompt	~0.0064	~0.0064
Mean RMS after 20 gen	~0.0486	~0.0485
Verdict	field differs, surface unread (byte-LM decode)	field carrying structure (prompt-sensitive)

Sample gens (deterministic, role-primed) are prompt-distinct (no longer identical collapse). Without a surface re-fit they remain embryonic/noisy.

Optional tiny smoke fine-tune

checkpoints/atom_native_readout_fix/ — 2k steps from a copy of chat_talk (corpus_train_chat.txt), stream dir untouched. After 2k: α cosine fell to ~0.39 (more field separation) but surface logit cosine rose toward ~0.99 as the skip/gate adapted under CE — longer surface-focused training still needed for fluent decode. Migration-only numbers above are the readout-fix success bar.

Tests

• test/test_generation_smoke.py::FieldReadoutSensitivityTests
  • Synthetic distinct α → logit cosine < 0.95
  • Migrated chat_talk ckpt → Bonjour vs Qui es-tu logit cosine < 0.95

Not done (explicit)

• Full MERGE / dual clocks / adaptive spans — deferred
• No changes to the live stream trainer PID / atom_native_stream_dialogue

How to re-probe

PYTHONPATH=. .venv/bin/python tools/probe_field_persistence.py \
  --checkpoint checkpoints/atom_native_chat_talk/atom_native.pt