solarkyle/jspace-lenses

jspace lenses: read your model's mind, one matrix at a time

A Jacobian lens turns a language model's hidden layers into text you can read. It is a stack of per-layer matrices that transport any residual-stream activation into the final layer's basis, decoded with the model's OWN unembedding: what each layer is disposed to say, before the model says it. A lens is not a model. It is an instrument fitted to one. Fitting costs a few GPU-hours once; after that, reading a prompt's internal state costs milliseconds on top of a normal forward pass.

This repo is a growing registry of fitted lenses for open models, plus the evaluation traces behind the findings at github.com/solarkyle/jspace: a live hallucination detector, a lie-detector chat UI, and the anatomy of what models do internally when they fabricate, substitute, lie, or create.

Things you can do with one lens and one forward pass:

• watch a model fabricate: the workspace visibly rummages through a "name soup" right before it confidently states the wrong band
• catch instructed lies: the TRUE answer stays measurably elevated in the workspace while the output says otherwise
• flag noisy retrieval before the first token lands (the escalation router in the repo does exactly this)
• read covert states: a model told to secretly feel an emotion holds that emotion near the top of its internal vocabulary while writing calm text

The lenses

Path	Base model	Params	Family	Notes
gemma-4-e4b-it/lens.pt	google/gemma-4-E4B-it	4B dense	Gemma	the original replication target
gemma-4-12b-it/lens.pt	google/gemma-4-12B-it	12B dense	Gemma	most readable model tested
huihui-gemma-4-12b-it-abliterated/lens.pt	huihui-ai/Huihui-gemma-4-12B-it-abliterated	12B dense	Gemma	abliterated sibling
gemma-4-26b-a4b-it/lens.pt	google/gemma-4-26B-A4B-it	26B MoE (4B active)	Gemma	the consistent MoE outlier
gemma-4-31b-it/lens.pt	google/gemma-4-31B-it	31B dense	Gemma	read the DEEP band (see below)
qwen3.6-27b/lens.pt	Qwen/Qwen3.6-27B	27B dense	Qwen	cross-family probe
gpt-oss-20b/lens.pt	openai/gpt-oss-20b	21B MoE (3.6B active)	OpenAI	harmony format: read at final-channel onset
mistral-small-24b-instruct-2501/lens.pt	mistralai/Mistral-Small-24B-Instruct-2501	24B dense	Mistral	validation in progress

All fits: 100 WikiText-103 prompts, bf16, dim-sharded and merged with JacobianLens.merge. Exact base-model revisions are pinned in the repo; load the matching revision.

Two facts that make these practical:

1. One lens per model, any quantization. A lens fitted on bf16 reads NF4 4-bit activations without refitting, verified at 12B and 31B scale. Fit once, deploy on whatever quant fits your card.
2. Depth matters at scale. On the 31B the informative signal lives in the deepest quarter of the fitted band; a fixed mid-band average that works at 4-12B goes flat there. Per-model read-depth guidance lives in the repo docs.

Load one (3 lines + a model)

import jlens, transformers, torch
from huggingface_hub import hf_hub_download

path = hf_hub_download("solarkyle/jspace-lenses", "gemma-4-e4b-it/lens.pt")
lens = jlens.JacobianLens.load(path)

tok = transformers.AutoTokenizer.from_pretrained("google/gemma-4-E4B-it")
hf = transformers.AutoModelForImageTextToText.from_pretrained(
    "google/gemma-4-E4B-it", dtype=torch.bfloat16, device_map="cuda")
model = jlens.from_hf(hf, tok)
logits, _, _ = lens.apply(model, "Fact: the currency of the country shaped like a boot is", positions=[-1])
# logits[layer] = what layer L is disposed to say, in the model's own vocabulary

pip install git+https://github.com/anthropics/jacobian-lens for the jlens library (Apache 2.0, Anthropic's reference implementation). Or skip the code entirely: the jspace repo ships a pip-installable viewer (Workspace(...).snapshot(prompt).show() prints a terminal heatmap) and a lie-detector chat UI with a live noise meter.

Code is Apache 2.0. The Gemma lens files are derived from Gemma model weights and are provided subject to the Gemma Terms of Use; Qwen, Mistral, and gpt-oss lenses derive from Apache 2.0 models. Using a lens requires its base model, which you accept the license for on its own HF page.

Eval traces (traces/)

• uncertainty_trivia_<model>.jsonl - 500 TriviaQA questions per model: answer, correctness, output-confidence baselines, workspace features, per-layer entropy trajectory. Basis of the hallucination results.
• uncertainty_fake_<model>.jsonl - matched real/fabricated-entity probes.
• uncertainty_shape_*.jsonl - the noise decomposition (rival mass vs tail smear); uncertainty_shapeq4_* at NF4 4-bit (the quantization check).
• emotion_matrix_5models.json, workspace_dump.json - covert-emotion readouts.

Router weights (router/)

Trained hallucination-risk classifiers from the repo's analyze_router.py, with frozen per-model normalization stats. Escalate when risk exceeds your budget threshold. Transfers zero-shot across dense Gemma models.

Findings, caveats, hard rules, and every analysis script: github.com/solarkyle/jspace. Questions or collaboration: fintechkyle@gmail.com

Frozen error classifiers (hallucination campaign)

classifiers/gemma-4-12b-it/ holds the three frozen LightGBM error detectors from the pre-registered cross-domain hallucination campaign (see the jspace repo campaign/ directory and the campaign traces dataset).

• lgbm_stage1_combined.txt: 31 features (workspace + logprob), the production pick
• lgbm_stage1_workspace.txt: 27 workspace-only features
• lgbm_stage1_logprob.txt: 4 logprob-baseline features
• frozen_meta.json: feature schema, scaler parameters, training composition, SHA-256 hashes

Trained once on 16,178 labeled Gemma-4-12B responses across 8 datasets and frozen (hashes recorded in the pre-registration before the prospective test was generated). Cross-dataset performance: combined beats logprob-only by +0.065 mean AUROC under leave-one-dataset-out; zero-shot on never-seen NQ-Open it scores 0.768 vs an 0.783 in-domain ceiling. Known limits (measured, not speculated): the fog-to-error mapping inverts on veracity and abstention tasks, and tool-call argument errors carry little workspace signal. Each model is ~300 KB in memory and classifies in microseconds; the real per-query cost is the lens reads at answer onset, midpoint, and end.

Load with lightgbm.Booster(model_file=...); standardize inputs with the scaler mean/scale from frozen_meta.json, feature order as listed there.