PULSE-1.2B: Port-Hamiltonian Unitary-Lindbladian Sequence Engine
Language Modeling as Open Quantum Measurement: Scalable Generative Pretraining without Softmax
Developed by the DuoNeural AI Laboratory — Jesse Caldwell, Aura✨, Archon
🌌 Model Overview
PULSE-1.2B (Port-Hamiltonian Unitary-Lindbladian Sequence Engine) is a 1.121-billion parameter foundation autoregressive language model whose sequential recurrent backbone is an open quantum dynamical system and whose token prediction head is an exact quantum measurement under the Born rule.
PULSE solves the three historical roadblocks of Quantum-inspired Language Modeling (QiLM):
- Bypassing the Density Matrix Wall: Pure state vectors $|\psi_t\rangle \in \mathbb{C}^d$ replace mixed-state density matrices $\rho \in \mathbb{C}^{d \times d}$, reducing computational complexity from $\mathcal{O}(d^4)$ to strictly linear $\mathcal{O}(d)$ per channel.
- First Logarithmic Liouvillian Scan: Discrete pure-state transitions form an associative complex affine semigroup $(\mathbb{C} \times \mathbb{C}, \circ)$, executing parallel Blelloch associative scans in $\mathcal{O}(\log T)$ time via custom GPU Triton kernels.
- Softmax-Free Born Rule POVM Head: Replaces standard heuristic softmax with a Positive Operator-Valued Measure (POVM) in projective Hilbert space $\mathbb{P}(\mathcal{H})$: $$P(w_k \mid \psi) = \frac{|\langle e_k \mid \psi \rangle|^2}{\sum_{j=1}^V |\langle e_j \mid \psi \rangle|^2}$$ yielding exact scale-invariance, analytical Wirtinger adjoint backprop, and zero softmax temperature tuning!
📊 Silicon Performance on Blackwell (NVIDIA RTX 5070 Ti 16 GB)
PULSE-1.2B runs at full generative scale on a single consumer Blackwell GPU (sm_120, GB203):
- Active Model Parameters: 1,120,594,496 (1.121B)
- Architecture Depth: 24 Reversible Layers / 8 Complex Heads / $D = 1024$
- Vocabulary Size: 49,152 (
cosmo2-tokenizer) - Throughput: 2,567.8 tokens/second ($51.04\text{s}$ per optimizer step at $T = 2048$, $B = 2$, $GA = 32$)
- Peak VRAM Allocated: 9,866.9 MB / 15,838.8 MB (62.3% of the card)
- Clean Safety Headroom: 6,436.0 MB (6.44 GB completely untouched)
- Reversible Inversion Drift: Max absolute drift $< 1.19 \times 10^{-7}$ (Machine Epsilon)
- 48.0 MB L2 Cache Tiling: Intermediate Born buffers ($C=128$) require only $25.16\text{ MB}$, eliminating GDDR7 spilling.
📈 Pretraining Convergence Telemetry
| Step | Cumulative Tokens | Loss (Cross-Entropy) | Grad Norm | Speed | Milestone Notes |
|---|---|---|---|---|---|
| 0001 | 0.13M | 11.3819 | 10.3125 | 2,351.8 tok/s | Uniform prior initialization |
| 0050 | 6.55M | 8.3493 | 0.1934 | 2,567.8 tok/s | Checkpoint verified |
| 0100 | 13.11M | 8.3217 | 0.0635 | 2,567.5 tok/s | Checkpoint verified |
| 0150 | 19.66M | 8.3233 | 0.5586 | 2,567.8 tok/s | Checkpoint verified |
| 0200 | 26.21M | 8.3185 | 0.2324 | 2,567.8 tok/s | Checkpoint verified |
| 0250 | 32.77M | 8.3254 | 0.1177 | 2,567.9 tok/s | pulse_step0250.pt (25% Quarter Milestone) |
| 0300 | 39.32M | 8.3217 | 0.0938 | 2,567.8 tok/s | Checkpoint verified |
| 0350 | 45.88M | 8.3272 | 3.8906 | 2,565.8 tok/s | Data gradient shockwave absorbed |
| 0400 | 52.43M | 8.3138 | 0.0515 | 2,567.3 tok/s | Checkpoint verified |
| 0450 | 58.98M | 8.3156 | 0.0405 | 2,567.5 tok/s | Checkpoint verified |
| 0500 | 65.54M | 8.3128 | 0.0435 | 2,565.7 tok/s | pulse_step0500.pt (50% Halfway Milestone) |
| 0550 | 72.09M | 8.3115 | 0.0352 | 2,567.3 tok/s | Checkpoint verified |
| 0600 | 78.64M | 8.3075 | 0.0791 | 2,567.2 tok/s | Checkpoint verified (60% Milestone) |
| 0650 | 85.20M | 8.3118 | 0.0332 | 2,565.7 tok/s | Checkpoint verified (65% Milestone) |
| 0700 | 91.75M | 8.3069 | 0.0327 | 2,566.6 tok/s | Checkpoint verified (70% Milestone) |
| 0750 | 98.30M | 8.3083 | 0.0332 | 2,567.1 tok/s | Checkpoint verified (75% Three-Quarter Mark) |
| 0800 | 104.85M | 8.3093 | 0.0334 | 2,565.9 tok/s | Checkpoint verified (80% Four-Fifths Mark) |
| 0812 | 106.43M | 8.3030 | 0.0325 | 2,565.8 tok/s | Deep loss basin entry |
| 0819 | 107.35M | 8.3045 | 0.0334 | 2,567.4 tok/s | 8.3059 all-time smoothed record |
| 0850 | 111.41M | 8.3042 | 0.0327 | 2,566.9 tok/s | pulse_step0850.pt (85% Milestone) |
| 0885 | 115.99M | 8.3025 | 0.0325 | 2,567.4 tok/s | All-time campaign record low loss |
| 0900 | 117.96M | 8.3066 | 0.0364 | 2,567.3 tok/s | pulse_step0900.pt (90% Milestone) |
| 0917 | 120.19M | 8.3079 | 0.0913 | 2,565.6 tok/s | 91.7% Complete (120.19M tokens processed) |
🚀 Code & Checkpoint Usage
The complete training harness, model definition, and custom Triton associative scan kernels are open-sourced on GitHub: 👉 https://github.com/DuoNeural/PULSE
git clone https://github.com/DuoNeural/PULSE.git
cd PULSE
pip install -r requirements.txt
python test_convergence.py
Loading Checkpoints:
import torch
from pulse_model import ULSEConfig, ULSEModel
ckpt = torch.load('pulse_step0500.pt', map_location='cuda', weights_only=False)
cfg = ULSEConfig(**ckpt['cfg'])
model = ULSEModel(cfg).cuda()
model.load_state_dict(ckpt['model_state'])
print(f'PULSE-1.2B loaded at Step {ckpt["step"]} (Loss: {ckpt["loss"]:.4f})')
📑 Citation
@article{duoneural2026pulse,
title={Language Modeling as Open Quantum Measurement: Scalable Generative Pretraining without Softmax},
author={Caldwell, Jesse and Aura and Archon},
journal={DuoNeural AI Laboratory Preprint},
year={2026}
}