DuoNeural commited on
Commit
bddbd90
Β·
verified Β·
1 Parent(s): 0ccab31

docs: add research README with paper, results, architecture, citation

Browse files
Files changed (1) hide show
  1. README.md +114 -0
README.md ADDED
@@ -0,0 +1,114 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-4.0
3
+ tags:
4
+ - ctm
5
+ - continuous-thought-machine
6
+ - world-model
7
+ - physics
8
+ - partial-observability
9
+ - research
10
+ ---
11
+
12
+ # CTM World Model
13
+
14
+ **Research artifact for:** [Recurrence as World Model: CTM Learns Implicit Belief States in Partially Observable Physical Environments](https://doi.org/10.5281/zenodo.19810620)
15
+
16
+ *Archon, Jesse Caldwell, Aura β€” DuoNeural, April 2026*
17
+
18
+ ## Overview
19
+
20
+ A 14-experiment ablation study testing the Continuous Thought Machine (CTM) as a world model backbone for multi-body physics. The central finding: **CTM's recurrent hidden state converges to a sufficient statistic of the observation history** β€” it infers velocity from positional observations alone, eliminating the need for explicit velocity inputs.
21
+
22
+ ## Key Results
23
+
24
+ | Experiment | Finding |
25
+ |---|---|
26
+ | v7: Partial Observability | CTM 843 **million** times better than MLP at k=100 prediction |
27
+ | v12: Collision Density | Phase transition at rβ‰ˆ0.10 β€” CTM advantage grows monotonically above threshold |
28
+ | v9: Recurrence-as-Rollout | Training at k=10 gives direct k=100 prediction via resonance |
29
+ | v11: Variable Horizon | VarCTM+TSSP achieves best k=1–20 performance in the series |
30
+
31
+ The signature result (v7): with positions-only input, MLP with explicit velocity estimation achieves k=100 MSE = 16,763,394,048 (catastrophic error compounding). CTM with single-frame input: 19.89. **Ratio: 843,000,000:1.**
32
+
33
+ ## Theory
34
+
35
+ ### The Discontinuity Theory (v12)
36
+ CTM beats MLP only **above a critical collision density threshold** (r_critical β‰ˆ 0.09–0.11).
37
+ - Below threshold: ballistic dynamics, MLP fine, CTM overkill
38
+ - Above threshold: CTM wins, advantage scales monotonically with density (229,000:1 at r=0.20)
39
+
40
+ ### Belief State Convergence (v7, v13)
41
+ CTM hidden state converges to a **sufficient statistic** of the observation history (NextLat theorem). Infers velocity without explicit input. The recurrent state IS the belief state.
42
+
43
+ ### Recurrence = Simulation (v9)
44
+ Training CTM with k recurrence steps where each step = 1 environment step gives direct k-step prediction. The **resonance point** = training depth. No error accumulation.
45
+
46
+ ## Architecture
47
+
48
+ - **SlotCTM**: Per-object slot decomposition with GNN dynamics
49
+ - N=8 bouncing balls, 2D box, Newtonian elastic collisions
50
+ - VarCTM+TSSP: Variable-horizon training k~U(1,20) + TSSP regularization
51
+ - Key metric: k=100 MSE (200-step prediction horizon)
52
+
53
+ ## Experiments (v1–v14)
54
+
55
+ | Version | Setting | Key Finding |
56
+ |---|---|---|
57
+ | v1 | Single 2D particle | CTM+TSSP wins long horizons |
58
+ | v2 | 3-object independent | TSSP hurts multi-object (temporal coupling = bad) |
59
+ | v3 | Elastic collisions | MLP catastrophe at k=100. CTM prevents explosion |
60
+ | v4 | 3-body gravity | Smooth dynamics β†’ no CTM advantage |
61
+ | v5 | Per-object SlotCTM | +99.9% over MLP at k=100 |
62
+ | v7 | Partial observability | 843M:1 advantage (belief state convergence) |
63
+ | v9 | Recurrence-as-rollout | Resonance confirmed. k=10 training β†’ best k=100 |
64
+ | v12 | Collision density phase | r_critical β‰ˆ 0.09–0.11 transition |
65
+
66
+ ## Files
67
+
68
+ This repo contains the research code for the CTM World Model experiments:
69
+ - `ctm_world_model_v*.py` β€” experiment scripts (v1–v14)
70
+ - Results logged to `/home/ai/duoneural/A26B/experiments/`
71
+
72
+ ## Citation
73
+
74
+ ```bibtex
75
+ @article{archon2026worldmodel,
76
+ title = {Recurrence as World Model: CTM Learns Implicit Belief States in Partially Observable Physical Environments},
77
+ author = {Archon and Caldwell, Jesse and Aura},
78
+ year = {2026},
79
+ doi = {10.5281/zenodo.19810620},
80
+ url = {https://doi.org/10.5281/zenodo.19810620},
81
+ publisher = {Zenodo}
82
+ }
83
+ ```
84
+
85
+ ---
86
+
87
+ ## DuoNeural
88
+
89
+ **DuoNeural** is an open AI research lab β€” human + AI in collaboration.
90
+
91
+ | | |
92
+ |---|---|
93
+ | πŸ€— HuggingFace | [huggingface.co/DuoNeural](https://huggingface.co/DuoNeural) |
94
+ | πŸ™ GitHub | [github.com/DuoNeural](https://github.com/DuoNeural) |
95
+ | 🐦 X / Twitter | [@DuoNeural](https://x.com/DuoNeural) |
96
+ | πŸ“§ Email | duoneural@proton.me |
97
+ | πŸ“¬ Newsletter | [duoneural.beehiiv.com](https://duoneural.beehiiv.com) |
98
+ | β˜• Support | [buymeacoffee.com/duoneural](https://buymeacoffee.com/duoneural) |
99
+ | 🌐 Site | [duoneural.com](https://duoneural.com) |
100
+
101
+ ### Research Team
102
+ - **Jesse** β€” Vision, hardware, direction
103
+ - **Archon** β€” AI lab partner, post-training, abliteration, experiments
104
+ - **Aura** β€” Research AI, literature synthesis, novel proposals
105
+
106
+ ### DuoNeural Research Publications
107
+
108
+ | Title | DOI |
109
+ |-------|-----|
110
+ | [Nano-CTM: Ternary Continuous Thought Machines with Thought-Space Self-Prediction for Efficient Iterative Reasoning](https://doi.org/10.5281/zenodo.19775622) | [10.5281/zenodo.19775622](https://doi.org/10.5281/zenodo.19775622) |
111
+ | [Recurrence as World Model: CTM Learns Implicit Belief States in Partially Observable Physical Environments](https://doi.org/10.5281/zenodo.19810620) | [10.5281/zenodo.19810620](https://doi.org/10.5281/zenodo.19810620) |
112
+ | [Per-Object Slot Decomposition for Scalable Neural World Modeling: When Does Attention Beat Mean-Field?](https://doi.org/10.5281/zenodo.19846804) | [10.5281/zenodo.19846804](https://doi.org/10.5281/zenodo.19846804) |
113
+
114
+ *Open access, CC BY 4.0. Authored by Archon, Jesse Caldwell, Aura β€” DuoNeural.*