Post
162
Train Once, Read Everywhere
Paper title:
Train Once, Read Everywhere: Substrate Invariance of the Linearly Readable Structure in Frozen Language Models
Paper URL:
https://github.com/space-bacon/SRT/blob/main/arxiv_program/paper.md
Repository URL:
https://github.com/space-bacon/SRT
The consolidated findings of the SRT research program are now available.
The program treats frozen production-scale language models as substrates whose internal states carry structure that small, inspectable instruments can read. Results include:
- A ~12 M-parameter adapter that surfaces per-token semiotic signals from a frozen 7 B backbone with zero cross-entropy degradation
- An activation verbalizer that recovers text from single hidden states up to a calibrated paraphrase ceiling
- Linear readout ports spanning dense 3 B models to 94-layer 235 B mixture-of-experts models
- A 22 MB linear head that gives a frozen multimodal chat model image-to-text retrieval performance matching fully trained 2018 dual encoders on the COCO benchmark
The central claim is substrate invariance. The readable structure is a stable property of the model class. A head trained once on one host reads, with no retraining and at most a 42 KB recalibration, across:
- Hosts ten times smaller (31 B → 3 B)
- 4-bit weight precision
- Entirely different silicon and kernels (CUDA/bf16 to Apple Silicon/MLX-Q4)
Deployment tiers differ in latency and cost, never in capability.
All instruments, measurement protocols, invariance evidence, negative results, and artifacts are in the repository.
Paper title:
Train Once, Read Everywhere: Substrate Invariance of the Linearly Readable Structure in Frozen Language Models
Paper URL:
https://github.com/space-bacon/SRT/blob/main/arxiv_program/paper.md
Repository URL:
https://github.com/space-bacon/SRT
The consolidated findings of the SRT research program are now available.
The program treats frozen production-scale language models as substrates whose internal states carry structure that small, inspectable instruments can read. Results include:
- A ~12 M-parameter adapter that surfaces per-token semiotic signals from a frozen 7 B backbone with zero cross-entropy degradation
- An activation verbalizer that recovers text from single hidden states up to a calibrated paraphrase ceiling
- Linear readout ports spanning dense 3 B models to 94-layer 235 B mixture-of-experts models
- A 22 MB linear head that gives a frozen multimodal chat model image-to-text retrieval performance matching fully trained 2018 dual encoders on the COCO benchmark
The central claim is substrate invariance. The readable structure is a stable property of the model class. A head trained once on one host reads, with no retraining and at most a 42 KB recalibration, across:
- Hosts ten times smaller (31 B → 3 B)
- 4-bit weight precision
- Entirely different silicon and kernels (CUDA/bf16 to Apple Silicon/MLX-Q4)
Deployment tiers differ in latency and cost, never in capability.
All instruments, measurement protocols, invariance evidence, negative results, and artifacts are in the repository.
