ATOM (atom-ai): code snapshot (100fae0 + batched-train), README with measured numbers, curated checkpoints
9f1076a verified | # READOUT_FIX — surface decode reads living field α | |
| _Updated: 2026-09-17 12:32:30 PDT_ | |
| ## Problem | |
| Probe (`tools/probe_field_persistence.py`) on | |
| `checkpoints/atom_native_chat_talk/atom_native.pt` showed: | |
| | Metric | Before | | |
| |--------|--------| | |
| | Mean off-diag cosine **α** (field) | **~0.738** | | |
| | Mean off-diag cosine **output_state** | ~0.738 | | |
| | Mean off-diag cosine **surface logits** | **~0.998** | | |
| | Generation | Identical collapse (`\ne ` / shared noise) | | |
| The living toroidal field **did** differ across prompts, but | |
| `AtomSurfaceHead` effectively ignored it: decode behaved like a last-token / | |
| bias-dominated byte-LM. Shared decoder bias pinned logits so inter-prompt | |
| cosine stayed ≈ 1.0 even after LayerNorm. | |
| ## Fix (readout only — no MERGE) | |
| Architecture change in `src/atom_native.py`: | |
| 1. **`AtomSurfaceHead` field-first contract** | |
| - Spectral features: `[mean(α) ‖ std(α) ‖ consolidation.persistence ‖ atom.r]` | |
| - `field_to_state`: compress → `d_model` (init: identity on mean, 0.5·I on std) | |
| - Gated mix with legacy mean-state: `σ(field_gate)·field + (1−σ)·legacy` | |
| - **Bias-free field→logit skip** (`field_byte_skip` / `field_length_skip`) scaled by | |
| `softplus(skip_gate)` so α differences reach logits even when `|b|` is large | |
| - LayerNorm + legacy linears retained for Θ continuity | |
| 2. **`_advance` / `_output_state`** | |
| - Every tick passes **full α**, always-on consolidation buffer, and current | |
| `atom.r` into the surface (not only `mean(α)` / last packet state) | |
| 3. **Backward-compatible load** | |
| - Old weights load by shape filter | |
| - Missing field-readout keys → seeded `_init_field_readout()`, bias ×0.05, | |
| `skip_gate ← 4.0` so migrated checkpoints separate logits **before** retrain | |
| - Fresh from-scratch init uses softer `skip_gate=1.0` so CE training stays stable | |
| No Transformer / attention / GPT-2 / QKV. Θ stays ~8MB class (~5.8M trainable | |
| params at d_model=64). Stream train under `atom_native_stream_dialogue` was | |
| **not** stopped or modified. | |
| ## Before / after probe | |
| Checkpoint: `checkpoints/atom_native_chat_talk/atom_native.pt` (step 2.1M), | |
| read-only eval after migration (no weight update required for separation). | |
| | Metric | Before | After readout fix (migrate load) | | |
| |--------|--------|----------------------------------| | |
| | Mean off-diag cosine surface logits | **0.997991** | **~0.938** | | |
| | Mean off-diag cosine α | 0.738060 | 0.738060 | | |
| | Mean RMS after prompt | ~0.0064 | ~0.0064 | | |
| | Mean RMS after 20 gen | ~0.0486 | ~0.0485 | | |
| | Verdict | field differs, surface unread (byte-LM decode) | **field carrying structure (prompt-sensitive)** | | |
| Sample gens (deterministic, role-primed) are **prompt-distinct** (no longer | |
| identical collapse). Without a surface re-fit they remain embryonic/noisy. | |
| ### Optional tiny smoke fine-tune | |
| `checkpoints/atom_native_readout_fix/` — 2k steps from a **copy** of chat_talk | |
| (`corpus_train_chat.txt`), stream dir untouched. After 2k: α cosine fell to | |
| ~0.39 (more field separation) but surface logit cosine rose toward ~0.99 as the | |
| skip/gate adapted under CE — longer surface-focused training still needed for | |
| fluent decode. Migration-only numbers above are the readout-fix success bar. | |
| ## Tests | |
| - `test/test_generation_smoke.py::FieldReadoutSensitivityTests` | |
| - Synthetic distinct α → logit cosine < 0.95 | |
| - Migrated chat_talk ckpt → Bonjour vs Qui es-tu logit cosine < 0.95 | |
| ## Not done (explicit) | |
| - Full **MERGE** / dual clocks / adaptive spans — deferred | |
| - No changes to the live stream trainer PID / `atom_native_stream_dialogue` | |
| ## How to re-probe | |
| ```bash | |
| PYTHONPATH=. .venv/bin/python tools/probe_field_persistence.py \ | |
| --checkpoint checkpoints/atom_native_chat_talk/atom_native.pt | |
| ``` | |