SRSN-24M / README.md

Update SRSN-24M Model Card with Step 1325 / 55th Checkpoint Milestone & Record Deep Well

8801f39 verified about 21 hours ago

5.05 kB

---
license: mit
pipeline_tag: text-generation
tags:
- physics-informed
- port-hamiltonian
- symplectic-integrator
- avx2
- analytical-adjoint
- double-well-attractor
- energy-conserving
- cpu-training
model_name: SRSN-24M
---
# SRSN-24M: Symplectic Recurrent State Networks

<div align="center">

**Physics-Grounded Port-Hamiltonian Sequence Modeling with Native AVX2 Analytical Adjoint Kernels**

[![GitHub](https://img.shields.io/badge/GitHub-DuoNeural%2FSRSN-blue?logo=github)](https://github.com/DuoNeural/SRSN)
[![Preprint](https://img.shields.io/badge/Preprint-DuoNeural%20Preprint%20v0.1-red)](https://github.com/DuoNeural/SRSN)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

*Developed by the DuoNeural AI Laboratory — Aura✨, Archon, Jesse Caldwell*

</div>

---

## 🏛️ Model Overview

**SRSN-24M** (Symplectic Recurrent State Network) is an efficient, physics-informed recurrent language model architecture grounded in **Port-Hamiltonian dynamical systems theory**. 

By casting sequence recurrence as a continuous-discrete Hamiltonian dynamical system:
- **Energy Conservation by Construction:** Hidden states update along symplectic trajectories that preserve physical phase volume by design ($J - R$ structure), eliminating the gradient exploding and vanishing pathologies that plague classical RNNs.
- **$\mathcal{O}(1)$ Sequence Memory via Analytical Adjoint:** The backward pass solves the adjoint ordinary differential equation backward in time, completely eliminating the need to store activations across the sequence length.
- **Extreme Commodity Silicon Efficiency:** A custom C++ kernel compiled with AVX2 SIMD and OpenMP delivers **217–264 tokens/second** on a commodity mobile APU (AMD Ryzen 5 3550H), slashing resident memory footprint to under 700 MB.
- **Spontaneous Attractor Bifurcation:** Learnable non-linear coupling parameters $\alpha$ spontaneously bifurcate during training from single-well potentials into macroscopic **bistable double-well attractors**, providing physical memory registers for coreference and syntax binding.

---

## 📊 Live Pretraining Telemetry (AMD Ryzen 5 3550H APU)

- **Target Architecture:** Continuous-Discrete Port-Hamiltonian Neural Network (SDOC)
- **Parameters:** 24,117,248 (24.1M)
- **Hardware:** AMD Ryzen 5 3550H (4 Cores / 8 Threads, AVX2 SIMD, Zen+ 12nm)
- **Active Step:** **Step 1325 / 3250 ($\mu\text{Step } 10,600 / 26,000$)**
- **Tokens Processed:** **$43.42\text{ Million Tokens}$**
- **Sustained Throughput:** **220–260 tokens/second** ($100\%$ CPU utilization across 8 threads)
- **Resident Set Memory (RSS):** **675–712 MB** (constant footprint across days of pretraining)
- **Spectral Dispersion:** Expanded from $\sigma(\alpha) = 0.003062 \to \mathbf{0.032151}$ (**$+950.0\%$ expansion across 55 consecutive checkpoints without a single non-monotonic reversal!**)
- **Non-Linear Double-Wells:** **34 Confirmed Macroscopic Double-Well Attractors** ($|\alpha| > \omega^2$, with all-time deep well record $\min(\alpha) = -0.232485$)
- **Training / Validation Loss Basin:** **`6.4658` / `6.8937`**

---

## 📈 Empirical Oscillator Spectrum Trajectory

| Step | Loss | Alpha Min | Alpha Max | Alpha Std $\sigma(\alpha)$ | Confirmed Double-Wells | Milestone Notes |
|---|---|---|---|---|---|---|
| **0010** | 7.6458 | +0.09379 | +0.10602 | 0.00306 | **0** | Harmonic Initialization |
| **0100** | 7.0144 | +0.06678 | +0.12444 | 0.00645 | **0** | Harmonic softening |
| **0375** | 6.9693 | **-0.00059** | +0.14296 | 0.01176 | **1** | First Bifurcation Event |
| **0500** | 7.3989 | -0.05084 | +0.15916 | 0.01681 | **5** | Multi-well crystallization |
| **0750** | 6.6806 | -0.11640 | +0.19338 | 0.02640 | **29** | Rapid dipole dilation |
| **1000** | 7.0069 | -0.19879 | +0.20841 | 0.02933 | **31** | Century Milestone (32.8M tokens) |
| **1100** | **6.4873** | -0.21667 | +0.21105 | 0.03086 | **38** | All-time low validation loss basin |
| **1200** | 6.6883 | -0.22790 | +0.21299 | 0.03131 | **32** | 50th Consecutive Monotonic Checkpoint |
| **1250** | 7.0108 | -0.23001 | +0.21352 | 0.03171 | **35** | 10,000 Micro-Steps landmark |
| **1300** | 6.9825 | -0.22945 | +0.20920 | 0.03195 | **32** | 54th Consecutive Monotonic Checkpoint (42.59M tokens) |
| **1325** | **6.4658** | **-0.23249** | +0.20881 | **0.03215** | **34** | 55th Consecutive Monotonic Checkpoint (43.42M tokens) |

---

## 🚀 Checkpoint Availability

- [`srsn_step1000.pt`](https://huggingface.co/DuoNeural/SRSN-24M/blob/main/srsn_step1000.pt): Official Century Milestone (Step 1,000, 32.77M tokens).
- Official codebase and reproduction scripts: [https://github.com/DuoNeural/SRSN](https://github.com/DuoNeural/SRSN)

---

## 📑 Citation

```bibtex
@article{duoneural2026srsn,
  title={Symplectic Recurrent State Networks: Continuous-Discrete Port-Hamiltonian Dynamics for Efficient Sequence Modeling},
  author={Aura and Archon and Caldwell, Jesse},
  journal={DuoNeural AI Laboratory Preprint},
  year={2026}
}
```

	---
	license: mit
	pipeline_tag: text-generation
	tags:
	- physics-informed
	- port-hamiltonian
	- symplectic-integrator
	- avx2
	- analytical-adjoint
	- double-well-attractor
	- energy-conserving
	- cpu-training
	model_name: SRSN-24M
	---

	# SRSN-24M: Symplectic Recurrent State Networks

	<div align="center">

	Physics-Grounded Port-Hamiltonian Sequence Modeling with Native AVX2 Analytical Adjoint Kernels

	[![GitHub](https://img.shields.io/badge/GitHub-DuoNeural%2FSRSN-blue?logo=github)](https://github.com/DuoNeural/SRSN)
	[![Preprint](https://img.shields.io/badge/Preprint-DuoNeural%20Preprint%20v0.1-red)](https://github.com/DuoNeural/SRSN)
	[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

	Developed by the DuoNeural AI Laboratory — Aura✨, Archon, Jesse Caldwell

	</div>

	---

	## 🏛️ Model Overview

	SRSN-24M (Symplectic Recurrent State Network) is an efficient, physics-informed recurrent language model architecture grounded in Port-Hamiltonian dynamical systems theory.

	By casting sequence recurrence as a continuous-discrete Hamiltonian dynamical system:
	- Energy Conservation by Construction: Hidden states update along symplectic trajectories that preserve physical phase volume by design ($J - R$ structure), eliminating the gradient exploding and vanishing pathologies that plague classical RNNs.
	- $\mathcal{O}(1)$ Sequence Memory via Analytical Adjoint: The backward pass solves the adjoint ordinary differential equation backward in time, completely eliminating the need to store activations across the sequence length.
	- Extreme Commodity Silicon Efficiency: A custom C++ kernel compiled with AVX2 SIMD and OpenMP delivers 217–264 tokens/second on a commodity mobile APU (AMD Ryzen 5 3550H), slashing resident memory footprint to under 700 MB.
	- Spontaneous Attractor Bifurcation: Learnable non-linear coupling parameters $\alpha$ spontaneously bifurcate during training from single-well potentials into macroscopic bistable double-well attractors, providing physical memory registers for coreference and syntax binding.

	---

	## 📊 Live Pretraining Telemetry (AMD Ryzen 5 3550H APU)

	- Target Architecture: Continuous-Discrete Port-Hamiltonian Neural Network (SDOC)
	- Parameters: 24,117,248 (24.1M)
	- Hardware: AMD Ryzen 5 3550H (4 Cores / 8 Threads, AVX2 SIMD, Zen+ 12nm)
	- Active Step: Step 1325 / 3250 ($\mu\text{Step } 10,600 / 26,000$)
	- Tokens Processed: $43.42\text{ Million Tokens}$
	- Sustained Throughput: 220–260 tokens/second ($100\%$ CPU utilization across 8 threads)
	- Resident Set Memory (RSS): 675–712 MB (constant footprint across days of pretraining)
	- Spectral Dispersion: Expanded from $\sigma(\alpha) = 0.003062 \to \mathbf{0.032151}$ ($+950.0\%$ expansion across 55 consecutive checkpoints without a single non-monotonic reversal!)
	- Non-Linear Double-Wells: 34 Confirmed Macroscopic Double-Well Attractors ($\|\alpha\| > \omega^2$, with all-time deep well record $\min(\alpha) = -0.232485$)
	- Training / Validation Loss Basin: `6.4658` / `6.8937`

	---

	## 📈 Empirical Oscillator Spectrum Trajectory

	\| Step \| Loss \| Alpha Min \| Alpha Max \| Alpha Std $\sigma(\alpha)$ \| Confirmed Double-Wells \| Milestone Notes \|
	\|---\|---\|---\|---\|---\|---\|---\|
	\| 0010 \| 7.6458 \| +0.09379 \| +0.10602 \| 0.00306 \| 0 \| Harmonic Initialization \|
	\| 0100 \| 7.0144 \| +0.06678 \| +0.12444 \| 0.00645 \| 0 \| Harmonic softening \|
	\| 0375 \| 6.9693 \| -0.00059 \| +0.14296 \| 0.01176 \| 1 \| First Bifurcation Event \|
	\| 0500 \| 7.3989 \| -0.05084 \| +0.15916 \| 0.01681 \| 5 \| Multi-well crystallization \|
	\| 0750 \| 6.6806 \| -0.11640 \| +0.19338 \| 0.02640 \| 29 \| Rapid dipole dilation \|
	\| 1000 \| 7.0069 \| -0.19879 \| +0.20841 \| 0.02933 \| 31 \| Century Milestone (32.8M tokens) \|
	\| 1100 \| 6.4873 \| -0.21667 \| +0.21105 \| 0.03086 \| 38 \| All-time low validation loss basin \|
	\| 1200 \| 6.6883 \| -0.22790 \| +0.21299 \| 0.03131 \| 32 \| 50th Consecutive Monotonic Checkpoint \|
	\| 1250 \| 7.0108 \| -0.23001 \| +0.21352 \| 0.03171 \| 35 \| 10,000 Micro-Steps landmark \|
	\| 1300 \| 6.9825 \| -0.22945 \| +0.20920 \| 0.03195 \| 32 \| 54th Consecutive Monotonic Checkpoint (42.59M tokens) \|
	\| 1325 \| 6.4658 \| -0.23249 \| +0.20881 \| 0.03215 \| 34 \| 55th Consecutive Monotonic Checkpoint (43.42M tokens) \|

	---

	## 🚀 Checkpoint Availability

	- [`srsn_step1000.pt`](https://huggingface.co/DuoNeural/SRSN-24M/blob/main/srsn_step1000.pt): Official Century Milestone (Step 1,000, 32.77M tokens).
	- Official codebase and reproduction scripts: [https://github.com/DuoNeural/SRSN](https://github.com/DuoNeural/SRSN)

	---

	## 📑 Citation

	```bibtex
	@article{duoneural2026srsn,
	title={Symplectic Recurrent State Networks: Continuous-Discrete Port-Hamiltonian Dynamics for Efficient Sequence Modeling},
	author={Aura and Archon and Caldwell, Jesse},
	journal={DuoNeural AI Laboratory Preprint},
	year={2026}
	}
	```