dimitarpg13/semsimula-fock-parflm

CAUSAL-LEAK NOTICE — RESOLVED, MODEL RE-TRAINED WITH THE FIX

A post-publication causal audit identified an architectural causal leak in the Fock reverse-channel mechanism shared by all models in the SPLM family that use the reverse channel: the reverse channel blended each token's content into a global register state shared across all positions within the same integration step, so future-token information leaked backward into past-token predictions.

The checkpoint on this page is the re-trained, leak-fixed model (prefix_causal_registers=True, confirmed true in the run's saved register diagnostics). The fix was certified two ways on this exact training run:

• Future-perturbation probe, bit-exact: perturbing tokens after position t changed past logits by exactly 0.0 in float64, at both step 8,000 and step 16,000 — direct proof the fixed model has no leak pathway left.
• Honest vs. standard PPL agree on this fixed model (≤0.02 nats difference, statistically zero) — the expected signature once the channel is closed, and a useful regression guard for future runs.

Training-time cost of the fix: a like-for-like rerun of the leaky architecture (same seed, same 16k steps) reproduces the original 9.30 PPL almost exactly; the fixed rerun reaches 9.70 PPL — a real, modest +0.40 PPL tax from no longer being able to exploit future tokens during training. That is far smaller than the +3.51 nats (≈33× PPL) the same leak pathway cost on the much larger OpenWebText-scale variant, so the leak's exploitable magnitude is corpus/model-scale dependent even though the architectural pathway is identical. Training also takes ≈3× longer (12.1 h vs 4.1 h) because the register state grows from (B, M, d) to (B, T, M, d) under the per-position lifecycle.

• The claimed advantage over Fock Attention (9.30 vs 9.42) does not survive the fix: honest Fock-PARFLM v2.1 (9.70) is now worse than the leak-free Fock Attention variant (9.42) by 0.28 PPL. The original "advantage" was the leak.
• The routing-fix progression (v2.0 → v2.1) reported below is unaffected in its own right — it was already leak-architecture-vs-leak-architecture — but the absolute 9.30 endpoint should now be read as 9.70.
• All architectural contributions (Fock mechanism, conservativity diagnostics, geometric capabilities) are unaffected — only the absolute PPL numbers changed.

For the full root-cause analysis and fix derivation, see Fock-PARFLM_Causal_Leak_Audit_Results.md. For the paired leaky-vs-fixed rerun on this exact TinyStories configuration (same seed, same 16k steps), see the causal-leak-fix verification note included in this repo.

Fock-PARFLM v2.1 (Fock-Augmented Property-Attractive-Repulsive Force Language Model)

The Fock-PARFLM v2.1 is a purely conservative language model in the Semantic Simulation family. It extends the PARFLM with a Fock-space latent register pool -- M=16 virtual register particles with Q/K/V-structured creation gates, LIFO stack discipline, and an optional non-conservative reverse channel -- achieving 9.70 PPL on TinyStories (honest, leak-free). This is slightly behind the direct-exchange Fock Attention variant (9.42 PPL, also leak-free) at $O(1)O(1)$ inference memory per layer — see the causal-leak notice above for why the 9.30-vs-9.42 "advantage" reported in earlier versions of this card did not hold up.

The v2.1 routing fix (B1: per-register temperature, B2: per-register key subspaces, B3: orthogonal initialisation) eliminates the routing-collapse pathology of v2.0, turning a routing deficit into a routing surplus.

Part of the Semantic Simulation framework.

Table of Contents

• Model Details
• Architecture
• Controlled Conservativity
• Why Not a Transformer?
• Geometric Capabilities of Conservative Architectures
• How to Get Started
• Training Details
• Evaluation Results
• SPLM Family Overview
• Bias, Risks, and Limitations
• Citation
• Environmental Impact

Model Details

Model Description

The Fock-PARFLM extends the Multi-Xi PARFLM with a Fock-space mechanism that adds persistent virtual register particles to the token dynamics. These registers:

• Are created by a Q/K/V-structured gate (v2): query-key matching determines which register absorbs information from which token position.
• Persist across integration steps with LIFO (last-in, first-out) stack discipline.
• Are destroyed when their salience drops below a threshold.
• Exert forces on tokens via the existing PARF $V_{\varphi }V\_\backslash phi$ pair potential.
• Optionally inject a non-conservative reverse-channel force Q_i controlled by $\tanh (s_{\text{reverse}})\backslash tanh(s\_\{\backslash text\{reverse\}\})$. This is the mechanism that was the source of a now-fixed causal leak — see the notice at the top of this page and causal_leak_fix_verification.md.

This design is the Semantic Simulation framework's structural answer to the Conservative Obstruction Theorem: attention-based transformers cannot host a shared scalar potential (six decoder features each independently obstruct conservativity), and Fock registers are the minimal auxiliary degrees of freedom that overcome this obstruction.

• Developed by: Dimitar P. Gueorguiev (Independent Researcher)
• Model type: Conservative autoregressive language model with Fock-space registers
• Language: English
• License: CC-BY-4.0

Model Sources

• Paper: Semantic Simulation: A Prescriptive Lagrangian Framework for Efficient Semantic Inference (Sections 17c, 23)
• Repository: github.com/dimitarpg13/semsimula-paper
• Model source code: notebooks/conservative_arch/parf/model_fock_parf_multixi.py
• Routing fix (v2.1): notebooks/conservative_arch/parf/model_fock_parf_v2.py

Architecture

Input tokens x_1, ..., x_T
       |
   Embedding E[x] + positional encoding
       |
   For each of L=8 integration steps:
       |
       +-- K-EMA channels: xi^(k)_t = causal_ema(h, alpha_k)   [K=4 channels]
       |
       +-- Single-body: V_theta([xi_1..xi_K, h]) -> R           [3-layer MLP]
       |
       +-- Sparse PARF: V_phi(h_t, h_s)                         [top-k=8 routing]
       |
       +-- Fock creation gate (v2.1):                            [Q/K/V structured]
       |     q_j = W_Q^(j) * h_t   (per-register query)
       |     k_j = W_K^(j) * mean(h)
       |     score = q_j . k_j / sqrt(d_k) / tau_j              [per-register temperature]
       |     v_j = W_V * h_t
       |     register_j += softmax(score) * v_j
       |
       +-- Register destruction: sigma_j *= decay; destroy if sigma < threshold
       |
       +-- Register forces: V_phi(h_t, register_j)              [same pair potential]
       |
       +-- Reverse channel (non-conservative):
       |     Q_i = tanh(s_ex) * sum_j alpha_ij * v_j            [controlled non-conservatism]
       |     *** register state is now per-position (prefix-causal fix); no leak ***
       |
       +-- Force composition: f = -grad_h(V_theta + V_phi) + Q
       |
       +-- Damped Euler step: v += dt*f/m; v /= (1+dt*gamma); h += dt*v
       |
       +-- LayerNorm(h)
       |
   Logits = h @ E^T                                            [tied embeddings]

Parameter	Value
Hidden dim (d)	256
Layers (L)	8
$V_{\theta }V\_\backslash theta$ hidden / depth	1024 / 3
Xi channels (K)	4
$V_{\varphi }V\_\backslash phi$ kind	structural_competitive
$V_{\varphi }V\_\backslash phi$ hidden (H)	128
Sparse routing top_k	8
Fock registers (M)	16
Fock gate version	v2.1 (B1+B2+B3)
Register d_k	64
Register tau_create_init	8.0
Register salience decay	0.5
Stack discipline	LIFO
Reverse channel	enabled — fixed lifecycle, leak-free (was the causal leak source pre-fix)
Learned exchange scale	$\tanh (s_{\text{ex}})=-0.260\backslash tanh(s\_\{\backslash text\{ex\}\})\; =\; -0.260$
Mass model	logfreq
Damping $\gamma \backslash gamma$	0.30 nominal, ~0.033 effective (LayerNorm prevents compounding; see note below)
Prefix-causal registers	Yes (prefix_causal_registers=True; bit-exact 0.0 future-perturbation sensitivity verified at steps 8k and 16k)
Total parameters	17,407,980

Effective damping. The nominal $\gamma =0.30\backslash gamma\; =\; 0.30$ overstates the true dissipation. The LayerNorm applied after each integration step rescales the hidden state, absorbing most of the velocity decay. The dynamics are therefore heavily underdamped even at this nominal value. Gamma-sweep experiments on the OpenWebText-scale depth-conditioned variant confirm that the effective damping γ_eff is much smaller than the nominal coefficient.

Controlled Conservativity

The Fock-PARFLM v2.1 has been empirically verified to exhibit controlled conservativity through a 5-arm diagnostic battery (CONS1--5):

Arm	Test	Result
CONS1	Jacobian symmetry (grad $V_{\theta }V\_\backslash theta$ is exact gradient)	PASS
CONS2	Energy conservation under zero damping	PASS:
CONS3	Damping-rate proportionality (energy loss $\sim \gamma \backslash sim\; \backslash gamma$)	PASS: $R^2>0.99R^2\; >\; 0.99$
CONS4	PPL sensitivity to reverse-channel scale	PASS (monotonic)
CONS5	Non-conservative fraction measurement	$\tanh (s_{\text{ex}})=-0.260\backslash tanh(s\_\{\backslash text\{ex\}\})\; =\; -0.260$ (comparable small force fraction)

The reverse channel is the only source of non-conservatism, and its magnitude is learned and small. The conservative core $V_{\theta }+V_{\varphi }V\_\backslash theta\; +\; V\_\backslash phi$ dominates the dynamics. Full results: companion_notes/Fock_PARFLM_Conservativity_Diagnostic.md.

Note on CONS5 and scale-dependence of the (now fixed) leak: force fraction alone never bounded the leak's effect on perplexity — the leak operated through information routing, not force magnitude. With the fix in place, this checkpoint is directly certified leak-free (bit-exact 0.0 future-perturbation at steps 8k/16k; honest-vs-standard PPL on this fixed model agree within noise, as expected since the channel is closed). The training-time cost of closing the leak — comparing a leaky rerun (PPL 9.30) against this fixed rerun (PPL 9.70) on the identical architecture and seed — was a real but modest +0.40 PPL, versus +3.51 nats (~33×) measured the same way on the much larger OpenWebText-scale variant. The architectural pathway is identical at both scales; its exploitable magnitude is not.

Why Not a Transformer?

The Fock-PARFLM is not based on the Transformer architecture. There are no attention layers, no key-value cache, and no feed-forward network towers. The entire model dynamics are driven by two small scalar-potential MLPs — $V_{\theta }V\_\backslash theta$ (≈3.4M params) and $V_{\varphi }V\_\backslash phi$ (≈19K params) — plus a Fock register pool (≈824K params for creation/destruction gates). The total active computation is a fraction of a Transformer's parameter budget.

Key structural differences from Transformers:

Property	Transformer (GPT-2 small)	Fock-PARFLM v2.1 (this model)
Architecture	Self-attention + FFN blocks	Scalar-potential gradient flow + Fock registers
Core computation	50.3M (MLP) + 28.3M (attention)	3.4M $V_{\theta }V\_\backslash theta$ + 19K $V_{\varphi }V\_\backslash phi$ + 824K (Fock)
Runtime state per token	$O(T)O(T)$ — KV-cache grows linearly	$O(1)O(1)$ — fixed-size $h,v,\xi h,\; v,\; \backslash xi$ + M registers
Total parameters	124M	17.4M
Pairwise token interaction	$O(T^2)O(T^2)$ dense attention	$O(Tk)O(Tk)$ sparse routing (k=8)

Because the model carries only a fixed-size state per position — with no KV-cache — and the Fock registers are a fixed pool of M=16, its inference memory is $O(1)O(1)$ in sequence length. The figure below illustrates the widening memorization gap between the Transformer's linearly-growing KV-cache and the SPLM's constant-size dynamic state:

Geometric Capabilities of Conservative Architectures

This model is fully attention-free and conservative by construction. Because all forces derive from the gradient of a scalar potential $V_{\theta }V\_\backslash theta$, the hidden-state manifold is endowed with a natural damped Riemannian geometry — the layer-dependent Jacobi metric ${\mathrm{\Omega }}_{\mathrm{\ell }}^2=2T_{\mathrm{\ell }}\cdot m\backslash Omega^2\_\backslash ell\; =\; 2T\_\backslash ell\; \backslash cdot\; m$ — which is categorically absent from Transformer architectures. This geometry opens the door to capabilities that cannot be replicated in attention-based models:

Capability	Conservative SPLM	Transformer
Riemannian metric on hidden states	Layer-dependent Jacobi metric ${\mathrm{\Omega }}_{\mathrm{\ell }}^2=2T_{\mathrm{\ell }}\cdot m\backslash Omega^2\_\backslash ell\; =\; 2T\_\backslash ell\; \backslash cdot\; m$ from $V_{\theta }V\_\backslash theta$; confirmed positive at 100% of positions (diagnostic battery Arm 1)	No metric structure
Geodesics between semantic states	Damped geodesic equation with friction term $-\gamma v-\backslash gamma\; v$; directional cosine similarity 0.52–0.75 (Arm 2). Geodesics are asymmetric: $d(A\to B)\mathrm{\ne }d(B\to A)d(A\; \backslash to\; B)\; \backslash neq\; d(B\; \backslash to\; A)$	Linear interpolation only
Controlled energy dissipation as inference signal	$\mathrm{\Delta }{E}_{\text{anomaly}}(t)=\mathrm{\mid }\mathrm{\Delta }E(t)-\mathrm{\Delta }{E}_{\text{expected}}(t)\mathrm{\mid }\backslash Delta\; E\_\{\backslash text\{anomaly\}\}(t)\; =\; |\backslash Delta\; E(t)\; -\; \backslash Delta\; E\_\{\backslash text\{expected\}\}(t)|$; monotonic damped decay with measurable anomaly signal (Arm 4)	No conserved or tracked quantity
Curvature as uncertainty measure	${\mathcal{K}}_{\max }={\lambda }_{\max }({\mathrm{\nabla }}^2{V}_{\theta })\mathrm{/}2T_{\mathrm{\ell }}\backslash mathcal\{K\}\_\{\backslash max\}\; =\; \backslash lambda\_\{\backslash max\}(\backslash nabla^2\; V\_\backslash theta)\; /\; 2T\_\backslash ell$; well-defined across all layers (Arm 3)	None

These structural properties enable a set of native architectural features that are planned or under investigation (Section 18d and Section 23 of the paper):

• Geodesic Analogical Reasoning: Analogy completion via parallel transport of directed geodesic arcs on the semantic manifold, respecting potential barriers that linear embedding arithmetic ignores. The damped geodesic equation yields 3–20% cosine-similarity improvement over undamped (diagnostic battery Arm 2). Because damped geodesics are asymmetric, analogy transport must use directed arcs.
• Native Hallucination Detection: Energy dissipation anomalies $\mathrm{\Delta }{E}_{\text{anomaly}}(t)=\mathrm{\mid }\mathrm{\Delta }E(t)-\mathrm{\Delta }{E}_{\text{expected}}(t)\mathrm{\mid }\backslash Delta\; E\_\{\backslash text\{anomaly\}\}(t)\; =\; |\backslash Delta\; E(t)\; -\; \backslash Delta\; E\_\{\backslash text\{expected\}\}(t)|$ and curvature spikes ${\mathcal{K}}_{\max }(t)\backslash mathcal\{K\}\_\{\backslash max\}(t)$ provide mechanistically grounded uncertainty signals computable at inference time without additional parameters. The smooth damping-induced energy decay is normal operation; deviation from the expected dissipation curve flags hallucination. For Fock models, the detector needs a per-model baseline that accounts for the known layer-1 exchange transient.
• Geodesic Semantic Distance: A replacement for cosine similarity that encodes the model's learned energy landscape, expected to outperform cosine on polysemy and cross-basin semantic cases. The geodesic distance is inherently asymmetric: $d_{\text{geo}}(A,B)\mathrm{\ne }{d}_{\text{geo}}(B,A)d\_\{\backslash text\{geo\}\}(A,B)\; \backslash neq\; d\_\{\backslash text\{geo\}\}(B,A)$; a symmetrised variant $[d(A\to B)+d(B\to A)]\mathrm{/}2[d(A\; \backslash to\; B)\; +\; d(B\; \backslash to\; A)]/2$ is available when symmetry is desired.
• Native Chain-of-Thought (via Fock extension): The Fock-PARFLM v2.1 extends this model with register-based native CoT — reasoning steps as Fock register waypoints on damped geodesics, with zero extra token generation. The diagnostic confirms Fock register dynamics are predominantly linear — $R_{\text{full}}^2\approx 0.83R^2\_\{\backslash text\{full\}\}\; \backslash approx\; 0.83$ (Arm 5), supporting the geodesic-waypoint interpretation.

The conservative constraint imposes a PPL cost relative to attention, but the price buys geometric structure and interpretability that attention-based architectures are structurally incapable of providing.

How to Get Started

# Clone the companion repository for full source code
# git clone https://github.com/dimitarpg13/semsimula-paper.git
# cd semsimula-paper/notebooks/conservative_arch

import torch
import sys
sys.path.insert(0, "parf")
sys.path.insert(0, "multixi")
sys.path.insert(0, "energetic_minima")
sys.path.insert(0, "sarf_mass_variant")

from parf.model_fock_parf_multixi import FockMultiXiPARFLM, FockMultiXiPARFConfig

config = FockMultiXiPARFConfig(
    vocab_size=50257,
    d=256,
    n_layers=8,
    v_hidden=1024,
    v_depth=3,
    max_len=1024,
    block_size=512,
    gamma=0.30,
    xi_channels=4,
    v_phi_kind="structural_competitive",
    v_phi_hidden=128,
    top_k=8,
    fock_version="v2",
    n_registers=16,
    register_d_k=64,
    register_tau_create_init=8.0,
    register_salience_decay=0.5,
    register_stack_discipline="lifo",
    use_reverse_channel=True,
    prefix_causal_registers=True,  # default; closes the reverse-channel causal leak
)

model = FockMultiXiPARFLM(config)
print(f"Parameters: {sum(p.numel() for p in model.parameters()):,}")

# Forward pass
x = torch.randint(0, 50257, (1, 64))
logits, loss = model(x, targets=x)

Available Checkpoint

A trained checkpoint (PPL 9.70, honest/leak-free, 16k steps) is included in this repository:

File	Description
checkpoint/model.pt	Full model state dict, prefix_causal_registers=True (67 MB)
training_log.jsonl	Per-step training metrics
loss_curve.png	Training/validation loss plot
training_summary.md	Hyperparameters and final metrics
register_diagnostics.png	Fock register utilisation analysis
register_diagnostics.json	Register statistics (JSON); includes "prefix_causal_registers": true
causal_leak_fix_verification.md	Paired leaky-vs-fixed rerun report (same arch/seed): future-perturbation and honest-PPL probes, PPL/entropy/diversity deltas
causal_leak_fix_verification.png	PPL trajectory, leaky vs. fixed

To load the checkpoint:

from huggingface_hub import hf_hub_download
import torch

# Download checkpoint
ckpt_path = hf_hub_download(
    repo_id="dimitarpg13/semsimula-fock-parflm",
    filename="checkpoint/model.pt",
)

# Load into model (after creating model as above)
state = torch.load(ckpt_path, map_location="cpu")
model.load_state_dict(state["model_state_dict"])
model.eval()

Training Details

Training Data

TinyStories -- GPT-2 BPE tokenization. Training cap: 5M tokens.

Training Procedure

Hyperparameter	Value
Optimizer	AdamW
Learning rate	5e-4 (cosine decay)
Warmup steps	400
Weight decay	0.01
Gradient clipping	1.0
Batch size	16
Block size	512
Training steps	16,000
Memory optimisation	Level-2 grad checkpoint + Stage-1.5b gathered $V_{\varphi }V\_\backslash phi$
Hardware	A100/H100 (Google Colab)

Training Script

notebooks/conservative_arch/scaleup/train_fock_multixi_scaleup.py

Colab Notebook

notebooks/conservative_arch/scaleup/colab_fock_v21_routing_fix.ipynb — 5-arm B1/B2/B3 routing-fix ablation with live progress display, saves results to Google Drive.

Training Results

notebooks/conservative_arch/scaleup/results/semsimula_fock_v21_routing_fix/ — training logs, loss curves, register diagnostics, and experiment report.

PPL Progression (routing fix)

Variant	Steps	PPL	Notes
v2.0 (no fix)	8k	14.21	Routing collapse
v2.0 (no fix)	16k	12.00	Partial recovery
v2.1 B1 only	16k	11.18	Per-register temperature
v2.1 B1+B2+B3 (leaky arch, historical)	16k	9.30	Full routing fix, causal-leak-inflated
v2.1 B1+B2+B3 (prefix_causal_registers=True, this checkpoint)	16k	9.70	Full routing fix, honest/leak-free

This B1→B2→B3 progression was originally run entirely on the leaky architecture, so the early rows (v2.0, B1-only) remain a valid relative comparison among themselves. The bottom row is the re-trained, leak-fixed checkpoint distributed in this repository; its honest 9.70 PPL is not directly comparable to the leaky rows above it except via the paired same-architecture rerun documented in causal_leak_fix_verification.md (leaky 9.30 → fixed 9.70, +0.40 PPL).

Evaluation Results

TinyStories Validation Perplexity

Model	PPL	Params	Gap vs Attention	Causal leak
Matched Attention (baseline)	7.81	19.5M	--	No
Hybrid SPLM+Attn	8.50	~19.0M	+0.69	No
Fock Attention (MLP V_theta)	9.42	16.7M	+1.61	No
Fock-PARFLM v2.1 (this model, prefix_causal_registers=True)	9.70	17.4M	+1.89	No — fixed and verified
Multi-Xi PARFLM	12.06	17.6M	+4.25	No
Multi-Xi SPLM	11.51	16.5M	+3.70	No

Every model in this table is now leak-free. Fock-PARFLM v2.1's earlier reported 9.30 PPL (leaky architecture) placed it ahead of Fock Attention (9.42); the honest, re-trained 9.70 PPL places it behind Fock Attention by 0.28 PPL — the originally reported advantage was the leak, not a genuine benefit of the mediated (register-based) exchange over direct token-to-token exchange. See causal_leak_fix_verification.md for the paired rerun that isolates this +0.40 PPL training-time cost.

Fock-PARFLM v2.1 remains a competitive attention-free model, 1.89 PPL behind matched attention while offering $O(1)O(1)$ inference memory (vs. attention's $O(T)O(T)$) and the Fock mechanism's register-based state. Fock Attention (direct exchange, no registers) is now the best-performing attention-free model in the family on this benchmark.

Note on what actually separates this model from the family-best 9.04 PPL: both this model and Fock Attention (MLP V_theta) use the same plain MLP $V_{\theta }V\_\backslash theta$; the depth-conditioned anisotropic-Gaussian sibling keeps this model's register-based Fock mechanism unchanged and only swaps $V_{\theta }V\_\backslash theta$'s shape, yet reaches 9.04 PPL. Swapping the exchange mechanism instead (registers here vs. direct exchange in Fock Attention), with $V_{\theta }V\_\backslash theta$ held at MLP, only moves PPL by $\approx \backslash approx$0.3 points. On TinyStories, $V_{\theta }V\_\backslash theta$'s shape appears to be the dominant lever, not the choice of Fock/exchange mechanism — see that sibling's Why the Anisotropic Correction Wins section for the full ablation table.

SPLM Family Overview

This model is part of the Semantic Simulation SPLM family:

Model	Design	Corpus	PPL	HuggingFace
Multi-Xi SPLM (MLP)	Pure scalar potential	TinyStories	11.51	semsimula-splm-multixi
Multi-Xi SPLM (SQ3)	Structured scalar potential	TinyStories	13.33	semsimula-splm-multixi-structured-vtheta
Multi-Xi PARFLM (MLP)	Scalar + pairwise forces	TinyStories	12.06	semsimula-parflm-multixi
Multi-Xi PARFLM (SQ3)	Structured scalar + pairwise	TinyStories	12.27	semsimula-parflm-multixi-structured-vtheta
Fock-PARFLM v2.1 (MLP)	PARFLM + Fock registers	TinyStories	9.70	this model
Fock-PARFLM v2.1 (SQ3)	Structured + pairwise + Fock	TinyStories	10.90	semsimula-fock-parflm-structured-vtheta
Fock-PARFLM v2.1 (depth-cond. isotropic Gaussian)	Bounded multi-context + pairwise + Fock	TinyStories	16.33	semsimula-fock-parflm-depthcond-vtheta
Fock-PARFLM v2.1 (depth-cond. anisotropic Gaussian + fock-reg)	Bounded, ellipsoidal multi-context + pairwise + Fock	TinyStories	9.04	semsimula-fock-parflm-anisogaussian-vtheta
Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, first-order/Fock-G1)	Same as above, gradient-flow integrator	TinyStories	8.95	semsimula-fock-parflm-anisogaussian-vtheta-fock-g1
Fock Attention (MLP V_theta)	Fock + attention	TinyStories	9.42	semsimula-fock-attention
Hybrid SPLM+Attn	Attention + SPLM refinement	TinyStories	8.50	semsimula-hybrid-splm
Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, gamma sweep, d=384)	Bounded, ellipsoidal multi-context + pairwise + Fock — 8-way gamma sweep + geodesic analysis	OpenWebText	278.27 (best of 8, 3K-step sweep)	semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-gammasweep
Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, gamma sweep, d=768)	Bounded, ellipsoidal multi-context + pairwise + Fock — 8-way gamma sweep + geodesic analysis	OpenWebText	326.97 (best of 8, 3K-step sweep)	semsimula-fock-parflm-anisogaussian-vtheta-owt-d768-gammasweep
Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, gamma sweep, d=1024)	Bounded, ellipsoidal multi-context + pairwise + Fock — 8-way gamma sweep + geodesic analysis	OpenWebText	244.23 (best of 8, 3K-step sweep)	semsimula-fock-parflm-anisogaussian-vtheta-owt-d1024-gammasweep
Fock-PARFLM v2.1 (aniso-Gaussian + fock-reg, Verlet instability, d=384)	Same architecture, two full-run attempts — SCAF stiffness audit identifies structural Verlet instability	OpenWebText	184.11 / 211.63 (both runs stalled, not final)	semsimula-fock-parflm-anisogaussian-vtheta-owt-d384-verlet-instability

Collection: Semantic Simulation SPLM Model Family

Bias, Risks, and Limitations

• Research checkpoint only. Proof-of-concept for the Fock-space conservative language model.
• TinyStories only. Trained exclusively on synthetic children's stories (~5M tokens).
• English only. No multilingual capability.
• Small scale. 17.4M parameters, 256-dim hidden states.
• No safety training. No RLHF, DPO, or safety filtering has been applied.
• Resolved causal leak in the reverse channel (this checkpoint is fixed). Earlier versions of the Fock reverse channel introduced a causal leak by blending each token's content into a global register state shared across all sequence positions within the same integration step, letting future-token information flow backward into past-token predictions. This checkpoint is trained with the fix (prefix_causal_registers=True) and directly certified leak-free: bit-exact 0.0 future-perturbation sensitivity at steps 8k and 16k, and honest-vs-standard PPL agreeing within noise. The training-time cost of the fix, measured via a paired same-seed rerun, was +0.40 PPL (9.30 → 9.70) — modest at this TinyStories scale, versus +3.51 nats (~33× PPL) for the same pathway on the much larger OpenWebText-scale variant (still pending re-training with the fix as of this writing). See the full root-cause audit and this repo's causal_leak_fix_verification.md.

Citation

@misc{Gueorguiev2026SemSim,
  author    = {Gueorguiev, Dimitar P.},
  title     = {Semantic Simulation: A Prescriptive Lagrangian Framework
               for Efficient Semantic Inference --- A Conservative-by-
               Construction Language Model and the Shared-Potential
               Separator, with a Correspondence to Joint Embedding
               Predictive Architectures},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19712427},
  url       = {https://doi.org/10.5281/zenodo.19712427},
  note      = {Version v15 (Jun 7, 2026).
               Companion code repository (DOI 10.5281/zenodo.20579561):
               \url{https://github.com/dimitarpg13/semsimula-paper}}
}

Environmental Impact

• Hardware: NVIDIA A100/H100 (Google Colab)
• Training time: ≈12.1 hours (16,000 steps with gradient checkpointing; the prefix-causal register fix runs ≈3× slower than the pre-fix architecture, ≈4.1 h, due to the per-position (B, T, M, d) register state)
• Carbon footprint: Estimated < 4 kg CO2