Hugging Face's logo

DuoNeural
/
SRSN-24M

Text Generation
physics-informed
port-hamiltonian
symplectic-integrator
avx2
analytical-adjoint
double-well-attractor
energy-conserving
cpu-training
Model card Files
xet
 Community
SRSN-24M / README.md
DuoNeural's picture
DuoNeural
Update SRSN-24M Model Card with Step 1325 / 55th Checkpoint Milestone & Record Deep Well
8801f39 verified
preview code
|
Raw History Blame Contribute Delete
5.05 kB
metadata 
license: mit
pipeline_tag: text-generation
tags:
  - physics-informed
  - port-hamiltonian
  - symplectic-integrator
  - avx2
  - analytical-adjoint
  - double-well-attractor
  - energy-conserving
  - cpu-training
model_name: SRSN-24M

SRSN-24M: Symplectic Recurrent State Networks

Physics-Grounded Port-Hamiltonian Sequence Modeling with Native AVX2 Analytical Adjoint Kernels

GitHub Preprint License: MIT

Developed by the DuoNeural AI Laboratory — Aura✨, Archon, Jesse Caldwell

🏛️ Model Overview

SRSN-24M (Symplectic Recurrent State Network) is an efficient, physics-informed recurrent language model architecture grounded in Port-Hamiltonian dynamical systems theory.

By casting sequence recurrence as a continuous-discrete Hamiltonian dynamical system:

  • Energy Conservation by Construction: Hidden states update along symplectic trajectories that preserve physical phase volume by design ($J - R$ structure), eliminating the gradient exploding and vanishing pathologies that plague classical RNNs.
  • $\mathcal{O}(1)$ Sequence Memory via Analytical Adjoint: The backward pass solves the adjoint ordinary differential equation backward in time, completely eliminating the need to store activations across the sequence length.
  • Extreme Commodity Silicon Efficiency: A custom C++ kernel compiled with AVX2 SIMD and OpenMP delivers 217–264 tokens/second on a commodity mobile APU (AMD Ryzen 5 3550H), slashing resident memory footprint to under 700 MB.
  • Spontaneous Attractor Bifurcation: Learnable non-linear coupling parameters $\alpha$ spontaneously bifurcate during training from single-well potentials into macroscopic bistable double-well attractors, providing physical memory registers for coreference and syntax binding.

📊 Live Pretraining Telemetry (AMD Ryzen 5 3550H APU)

  • Target Architecture: Continuous-Discrete Port-Hamiltonian Neural Network (SDOC)
  • Parameters: 24,117,248 (24.1M)
  • Hardware: AMD Ryzen 5 3550H (4 Cores / 8 Threads, AVX2 SIMD, Zen+ 12nm)
  • Active Step: Step 1325 / 3250 ($\mu\text{Step } 10,600 / 26,000$)
  • Tokens Processed: $43.42\text{ Million Tokens}$
  • Sustained Throughput: 220–260 tokens/second ($100%$ CPU utilization across 8 threads)
  • Resident Set Memory (RSS): 675–712 MB (constant footprint across days of pretraining)
  • Spectral Dispersion: Expanded from $\sigma(\alpha) = 0.003062 \to \mathbf{0.032151}$ ($+950.0%$ expansion across 55 consecutive checkpoints without a single non-monotonic reversal!)
  • Non-Linear Double-Wells: 34 Confirmed Macroscopic Double-Well Attractors ($|\alpha| > \omega^2$, with all-time deep well record $\min(\alpha) = -0.232485$)
  • Training / Validation Loss Basin: 6.4658 / 6.8937

📈 Empirical Oscillator Spectrum Trajectory

Step Loss Alpha Min Alpha Max Alpha Std $\sigma(\alpha)$ Confirmed Double-Wells Milestone Notes
0010 7.6458 +0.09379 +0.10602 0.00306 0 Harmonic Initialization
0100 7.0144 +0.06678 +0.12444 0.00645 0 Harmonic softening
0375 6.9693 -0.00059 +0.14296 0.01176 1 First Bifurcation Event
0500 7.3989 -0.05084 +0.15916 0.01681 5 Multi-well crystallization
0750 6.6806 -0.11640 +0.19338 0.02640 29 Rapid dipole dilation
1000 7.0069 -0.19879 +0.20841 0.02933 31 Century Milestone (32.8M tokens)
1100 6.4873 -0.21667 +0.21105 0.03086 38 All-time low validation loss basin
1200 6.6883 -0.22790 +0.21299 0.03131 32 50th Consecutive Monotonic Checkpoint
1250 7.0108 -0.23001 +0.21352 0.03171 35 10,000 Micro-Steps landmark
1300 6.9825 -0.22945 +0.20920 0.03195 32 54th Consecutive Monotonic Checkpoint (42.59M tokens)
1325 6.4658 -0.23249 +0.20881 0.03215 34 55th Consecutive Monotonic Checkpoint (43.42M tokens)

🚀 Checkpoint Availability

📑 Citation 

@article{duoneural2026srsn,
  title={Symplectic Recurrent State Networks: Continuous-Discrete Port-Hamiltonian Dynamics for Efficient Sequence Modeling},
  author={Aura and Archon and Caldwell, Jesse},
  journal={DuoNeural AI Laboratory Preprint},
  year={2026}
}