README.md

metadata

license: other
license_name: unlicense
license_link: LICENSE
tags:
  - interpretability
  - mechanistic-interpretability
  - jacobian-lens
  - miru-tracer
  - adapters

Miru Tracer Jacobian-Lens Adapters

This repository hosts pre-fitted Jacobian-lens (J-Lens) adapters for Miru Tracer. The files are created with miru-tracer-fit-lens and let you use Miru's Jacobian-lens views without spending hours fitting a lens for each model yourself.

These are interpretability adapters, not LoRA adapters, fine-tunes, model weights, or generation plugins. They do not change what a language model has learned. They provide per-layer transformations that Miru uses to inspect—and experimentally intervene on—the model's internal residual stream.

What is Miru Tracer?

Miru Tracer is an experimental, open-source workbench for examining language-model generation token by token. Its Gradio interface can:

• step through generation and inspect token probabilities;
• override, undo, or continue individual generation steps;
• record and visualize complete generation traces;
• apply a standard logit lens or a fitted Jacobian lens at intermediate layers;
• compare logit- and Jacobian-lens readouts; and
• experiment with steer, swap, and ablate interventions on readout directions.

See the Miru Tracer repository for installation instructions, supported model families, and the full lens tutorial.

What is a Jacobian lens?

A transformer builds its prediction through a sequence of residual-stream states. Let h_l be the residual state at layer l, and let h_final be the state immediately before the model's final readout.

A logit lens sends h_l directly through the model's final normalization and unembedding. This is useful, but it implicitly assumes that an intermediate state already lives in the same representational basis as the final state. That assumption is often weakest in early and middle layers.

A Jacobian lens first transports the intermediate state toward the final layer's basis with a fitted matrix:

J_l = E[∂h_final / ∂h_l]
readout_l = unembed(J_l h_l)

J_l is the average Jacobian of the final residual state with respect to the state at layer l, estimated over many calibration prompts and token positions. The transported state can then be decoded with the model's own unembedding. This often gives more meaningful early- and middle-layer readouts than applying the unembedding directly.

Each adapter in this repository contains the fitted matrices for one exact base model, plus fit metadata. The matrices are stored as safetensors; they do not contain executable pickle payloads.

What fitting does

miru-tracer-fit-lens runs the base model over a calibration corpus, computes Jacobians with repeated backward passes, and maintains a running average for each fitted layer. Miru's default fitter:

• considers up to 1,000 prompts;
• uses sequences of at most 128 tokens;
• waits for at least 100 successful prompts before early stopping;
• tracks the mean relative change over the latest 10 successful updates; and
• declares convergence when that rolling mean falls below 0.002.

The prompt count shown below is the number of successful prompts actually included in the adapter—not merely the requested budget. Reaching the convergence threshold means that the running Jacobian estimate stabilized under this criterion; it does not by itself prove that every readout is semantically correct.

Model compatibility

J-Lens adapters are model-specific. Use an adapter only with the exact model it was fitted for. Two models can have the same hidden size and layer count while still having different weights, tokenizers, and internal representations; a shape match alone does not make their lenses interchangeable.

Available adapters

Base model	Adapter file	Prompts averaged	Convergence	Contributor
Qwen/Qwen3-0.6B	miru-tracer/Qwen/Qwen3-0.6B.safetensors	376	Yes (default early-stopping criterion reached)	Rodrigo Laneth (@rlaneth)
Qwen/Qwen3-4B	miru-tracer/Qwen/Qwen3-4B.safetensors	479	Yes (default early-stopping criterion reached)	Rodrigo Laneth (@rlaneth)

More models and community contributors will be added over time.

Using an adapter in Miru Tracer

1. Install and launch Miru Tracer.
2. Load the exact base model named in the table above.
3. Download its .safetensors adapter.
4. Open Miru's Lens tab and install the file in the Fit file section.
5. Select Jacobian or Compare (Jacobian / Logit) and generate or analyze a sequence.

You can also install a file directly into Miru's lens cache. Miru converts the slash in a Hugging Face model ID to -- for the cache directory, and expects the installed artifact to be named lens.safetensors:

~/.cache/miru-tracer/lenses/Qwen--Qwen3-0.6B/lens.safetensors
~/.cache/miru-tracer/lenses/Qwen--Qwen3-4B/lens.safetensors

Set MIRU_LENS_DIR to use a different cache root. This is convenient in a container, where the directory can be mounted as a persistent volume.

Fitting an adapter

Fitting is compute-intensive and is best done on a CUDA GPU. After installing Miru Tracer, a typical run is:

miru-tracer-fit-lens Qwen/Qwen3-0.6B --dim-batch 32

The fitter checkpoints after every successful prompt, so an interrupted run can resume. The current partial average is also written as a usable lens.safetensors file. Use --stop-at-delta 0 if you want to disable convergence-based early stopping and force the full prompt budget.

Run the following for all options:

miru-tracer-fit-lens --help

Contributing an adapter

Community contributions are welcome. Please submit adapters as safe .safetensors files using this path convention:

miru-tracer/<model-organization>/<model-name>.safetensors

For each contribution, include:

• the exact Hugging Face model ID and revision, when available;
• the number of successful prompts averaged;
• whether the convergence criterion was reached or the run used a fixed budget;
• any non-default fitting settings or calibration corpus;
• the Miru Tracer version used to create the file; and
• the contributor name and Hugging Face profile to display in the table.

Adapters generated by recent Miru Tracer versions embed model, tokenizer, corpus, and convergence provenance where available. Please preserve that metadata when uploading the artifact.

Public-domain contribution requirement

This repository is released under the Unlicense, which dedicates the repository's contents to the public domain. Every contributed adapter and its accompanying metadata must be submitted under the same terms. A contribution cannot be accepted under a more restrictive or additional license.

Because an adapter may be derived from a third-party base model or calibration corpus, a contributor can dedicate only the rights they actually own. The Unlicense does not cancel the base model's license, dataset terms, trademarks, or any other third-party rights. Contributors are responsible for checking those terms before submitting an artifact.

Every adapter contribution must include the following certification in its Hugging Face pull request or discussion. Replace the final line with the contributor's real name, Hugging Face username, and date:

Contributor Public-Domain Certification

I certify that I created this contribution or otherwise have the authority to submit it. To the extent that I own copyright or related rights in the adapter and its accompanying metadata, I permanently dedicate those rights to the public domain under the Unlicense. Where a public-domain dedication is not legally recognized, I make the contribution available under all permissions and disclaimers stated by the Unlicense. I have disclosed the base model and calibration sources, and I am not knowingly submitting material that I lack permission to distribute. If I am contributing as part of my employment or for another organization, I certify that I am authorized to make this dedication on its behalf.

Signed-off-by: Full Name (@huggingface-username), YYYY-MM-DD

The Signed-off-by line records the contributor's affirmative agreement to this certification; merely uploading a file is not sufficient. Maintainers may request a separately signed waiver or proof of organizational authorization for substantial or employer-owned contributions. The certification and sign-off should be retained in the repository's contribution history.

Further reading

• Miru Tracer
• Jacobian Lens research workspace
• Anthropic's reference Jacobian-lens implementation

Experimental status

Jacobians, lens readouts, and activation interventions are research tools. Fit quality depends on the base checkpoint, calibration corpus, prompt count, and fitting settings. Cross-check important conclusions with multiple methods and do not treat an individual decoded token as a definitive explanation of a model's internal computation.