mirrorethic/t3-124m-v36-pcloss

T³ 124M v3.6 — PCloss sibling

A sibling release to mirrorethic/t3-124m-v36. Same architecture, same training data, same step count, same hyperparameters — with one difference: the inter-stage predictive-coding (PC) loss is un-detached, letting the cross-stage primitive predictor's weights actually receive gradient signal.

This checkpoint exists to make a particular ablation reproducible: was the "long-standing torch.no_grad() wrapper in inter-stage PC" actually doing anything? The answer is "yes, the K-predictor learns a real map; no, it doesn't translate to downstream reasoning gains at this scale."

The science

In v3.1e through v3.5, the inter-stage predictive-coding loss was wrapped in with torch.no_grad(): at two call sites in t3v3_chain.py (lines 1184 and 1697). The forward pass computed the loss term correctly and the value was added to the total loss, but gradients never flowed back to the predictor's weights — the predictor stayed at identity-init for the entire training run. v3.6 removed those two wrappers.

This pcloss checkpoint is the first to actually train the inter-stage predictor with the no_grad fix in place. The result:

• K-predictor (Chronos primitive) learns a non-trivial cross-stage map. Correlation of S1→S2 K-predictor output with actual S2 K primitives jumps from r=0.26 (identity baseline) to r=0.59.
• Other primitives (E, I, F, V, C) stay at near-identity baselines — apparently the K axis is the only one with enough cross-stage signal to be worth modeling.
• Language modeling gets slightly worse: val PPL 28.53 vs 27.76 for the canonical v36 release (2.8% worse).
• Reasoning benchmarks are net-neutral: every task is within ±0.02 of the canonical sibling. Modest wins on COPA (+0.02) and BoolQ (+0.002); modest losses on ARC-Challenge (-0.008) and PIQA (-0.002).

Conclusion at this scale (124M, 2,500 steps): teaching the inter-stage PC predictor to learn doesn't translate into downstream task gains. The load-bearing PC signal might emerge at larger scales or longer training budgets — that's an open research question.

Quick start

from huggingface_hub import hf_hub_download
from t3 import T3Model

ckpt = hf_hub_download("mirrorethic/t3-124m-v36-pcloss", "pytorch_model.bin")
model = T3Model.from_checkpoint(ckpt)
model.eval()

For the full reference repo + benchmark reproduction code, see https://github.com/MirrorEthic/t3-reference.

Direct comparison vs canonical sibling

Task	Metric	This (pcloss)	Canonical (v36)	Δ
WikiText-103 (val)	perplexity	28.53	27.76	+0.77 (worse)
BoolQ	acc	0.6064	0.6046	+0.0018
ARC-Easy	acc	0.4398	0.4331	+0.0067
ARC-Challenge	acc	0.2099	0.2176	−0.0077
PIQA	acc	0.6028	0.6050	−0.0022
HellaSwag	acc	0.3029	0.3040	−0.0011
WinoGrande	acc	0.5075	0.5043	+0.0032
COPA	acc	0.6200	0.6000	+0.0200
RTE	acc	0.5271	0.5235	+0.0036

All numbers are full lm-eval-harness 0.4.x runs (no subset). For the comparison panels, see https://t3atlas.dev/benchmarks/.

Architecture

Identical to the canonical sibling. See mirrorethic/t3-124m-v36's model card for full architecture and training details, or docs/ARCHITECTURE.md for the technical specification.

The only differing config field is implicit (the torch.no_grad wrapper is in the source code, not in the config dict). All exposed hyperparameters and capability flags are bit-identical to the canonical sibling.

Capabilities probe

Same as the canonical sibling — the no_grad fix is a code-path change, not a config change.

{
  "has_coupling":       true,
  "has_trivectors":     false,
  "has_dyn_omega":      false,
  "has_inter_stage_pc": true,
  "has_scratchpad":     true,
  "n_primitives":       6,
  "null_cone_strength": 0.02,
  "hamiltonian_coupling": 0.02,
  "sigma_hidden":       16,
  "scratchpad_inject_entropy": [0.0, 0.0, 0.03]
}

Intended use

Same as the canonical sibling, with one additional use-case: studying the effect of working vs no-op inter-stage predictive-coding loss in hierarchical-prediction transformers. Pair this checkpoint with mirrorethic/t3-124m-v36 for the controlled comparison.

Citation

@misc{sutherland2026t3pcloss,
  author = {Sutherland, Garret},
  title  = {T³ v3.6 PCloss Sibling — Inter-Stage Predictive Coding Ablation},
  year   = {2026},
  publisher = {Hugging Face},
  url    = {https://huggingface.co/mirrorethic/t3-124m-v36-pcloss}
}

License

Apache-2.0. Both code (MirrorEthic/t3-reference) and weights (this repository).

Contact

Garret Sutherland (MirrorEthic LLC) — gsutherland@mirrorethic.com.

---

Released 2026-05-03. The pytorch_model.bin here is a stripped inference-ready copy (498 MB) of checkpoints_v36_pcloss/best.pt from the v3.6 training campaign. The optimizer state and data-loader state were dropped; everything T3Model needs at inference is preserved (model_state, ecology_state, config, and provenance metadata).