README.md

---
license: other
license_name: unlicense
license_link: LICENSE
tags:
  - interpretability
  - mechanistic-interpretability
  - jacobian-lens
  - miru-tracer
  - adapters
---

# Miru Tracer Jacobian-Lens Adapters

This repository hosts pre-fitted **Jacobian-lens (J-Lens) adapters** for
[Miru Tracer](https://github.com/returnmoe/miru-tracer). The files are created
with `miru-tracer-fit-lens` and let you use Miru's Jacobian-lens views without
spending hours fitting a lens for each model yourself.

> [!IMPORTANT]
> These are interpretability adapters, not LoRA adapters, fine-tunes, model
> weights, or generation plugins. They do not change what a language model has
> learned. They provide per-layer transformations that Miru uses to inspect—and
> experimentally intervene on—the model's internal residual stream.

## What is Miru Tracer?

[Miru Tracer](https://github.com/returnmoe/miru-tracer) is an experimental,
open-source workbench for examining language-model generation token by token.
Its Gradio interface can:

- step through generation and inspect token probabilities;
- override, undo, or continue individual generation steps;
- record and visualize complete generation traces;
- apply a standard logit lens or a fitted Jacobian lens at intermediate layers;
- compare logit- and Jacobian-lens readouts; and
- experiment with steer, swap, and ablate interventions on readout directions.

See the [Miru Tracer repository](https://github.com/returnmoe/miru-tracer) for
installation instructions, supported model families, and the full lens
tutorial.

## What is a Jacobian lens?

A transformer builds its prediction through a sequence of residual-stream
states. Let `h_l` be the residual state at layer `l`, and let `h_final` be the
state immediately before the model's final readout.

A **logit lens** sends `h_l` directly through the model's final normalization
and unembedding. This is useful, but it implicitly assumes that an intermediate
state already lives in the same representational basis as the final state. That
assumption is often weakest in early and middle layers.

A **Jacobian lens** first transports the intermediate state toward the final
layer's basis with a fitted matrix:

```text
J_l = E[∂h_final / ∂h_l]
readout_l = unembed(J_l h_l)
```

`J_l` is the average Jacobian of the final residual state with respect to the
state at layer `l`, estimated over many calibration prompts and token positions.
The transported state can then be decoded with the model's own unembedding.
This often gives more meaningful early- and middle-layer readouts than applying
the unembedding directly.

Each adapter in this repository contains the fitted matrices for one exact base
model, plus fit metadata. The matrices are stored as `safetensors`; they do not
contain executable pickle payloads.

### What fitting does

`miru-tracer-fit-lens` runs the base model over a calibration corpus, computes
Jacobians with repeated backward passes, and maintains a running average for
each fitted layer. Miru's default fitter:

- considers up to 1,000 prompts;
- uses sequences of at most 128 tokens;
- waits for at least 100 successful prompts before early stopping;
- tracks the mean relative change over the latest 10 successful updates; and
- declares convergence when that rolling mean falls below `0.002`.

The prompt count shown below is the number of successful prompts actually
included in the adapter—not merely the requested budget. Reaching the
convergence threshold means that the running Jacobian estimate stabilized under
this criterion; it does **not** by itself prove that every readout is
semantically correct.

### Model compatibility

J-Lens adapters are model-specific. Use an adapter only with the exact model it
was fitted for. Two models can have the same hidden size and layer count while
still having different weights, tokenizers, and internal representations; a
shape match alone does not make their lenses interchangeable.

## Available adapters

| Base model | Adapter file | Prompts averaged | Convergence | Contributor |
|---|---|---:|---|---|
| `Qwen/Qwen3-0.6B` | [`miru-tracer/Qwen/Qwen3-0.6B.safetensors`](https://huggingface.co/returnmoe/jlens-adapters/blob/main/miru-tracer/Qwen/Qwen3-0.6B.safetensors) | 376 | Yes (default early-stopping criterion reached) | [Rodrigo Laneth](https://rlaneth.com) ([@rlaneth](https://huggingface.co/rlaneth)) |
| `Qwen/Qwen3-4B` | [`miru-tracer/Qwen/Qwen3-4B.safetensors`](https://huggingface.co/returnmoe/jlens-adapters/blob/main/miru-tracer/Qwen/Qwen3-4B.safetensors) | 479 | Yes (default early-stopping criterion reached) | [Rodrigo Laneth](https://rlaneth.com) ([@rlaneth](https://huggingface.co/rlaneth)) |

More models and community contributors will be added over time.

## Using an adapter in Miru Tracer

1. Install and launch [Miru Tracer](https://github.com/returnmoe/miru-tracer).
2. Load the exact base model named in the table above.
3. Download its `.safetensors` adapter.
4. Open Miru's **Lens** tab and install the file in the **Fit file** section.
5. Select **Jacobian** or **Compare (Jacobian / Logit)** and generate or analyze
   a sequence.

You can also install a file directly into Miru's lens cache. Miru converts the
slash in a Hugging Face model ID to `--` for the cache directory, and expects
the installed artifact to be named `lens.safetensors`:

```text
~/.cache/miru-tracer/lenses/Qwen--Qwen3-0.6B/lens.safetensors
~/.cache/miru-tracer/lenses/Qwen--Qwen3-4B/lens.safetensors
```

Set `MIRU_LENS_DIR` to use a different cache root. This is convenient in a
container, where the directory can be mounted as a persistent volume.

## Fitting an adapter

Fitting is compute-intensive and is best done on a CUDA GPU. After installing
Miru Tracer, a typical run is:

```bash
miru-tracer-fit-lens Qwen/Qwen3-0.6B --dim-batch 32
```

The fitter checkpoints after every successful prompt, so an interrupted run can
resume. The current partial average is also written as a usable
`lens.safetensors` file. Use `--stop-at-delta 0` if you want to disable
convergence-based early stopping and force the full prompt budget.

Run the following for all options:

```bash
miru-tracer-fit-lens --help
```

## Contributing an adapter

Community contributions are welcome. Please submit adapters as safe
`.safetensors` files using this path convention:

```text
miru-tracer/<model-organization>/<model-name>.safetensors
```

For each contribution, include:

- the exact Hugging Face model ID and revision, when available;
- the number of successful prompts averaged;
- whether the convergence criterion was reached or the run used a fixed budget;
- any non-default fitting settings or calibration corpus;
- the Miru Tracer version used to create the file; and
- the contributor name and Hugging Face profile to display in the table.

Adapters generated by recent Miru Tracer versions embed model, tokenizer,
corpus, and convergence provenance where available. Please preserve that
metadata when uploading the artifact.

### Public-domain contribution requirement

This repository is released under
[the Unlicense](https://unlicense.org/), which dedicates the repository's
contents to the public domain. Every contributed adapter and its accompanying
metadata must be submitted under the same terms. A contribution cannot be
accepted under a more restrictive or additional license.

Because an adapter may be derived from a third-party base model or calibration
corpus, a contributor can dedicate only the rights they actually own. The
Unlicense does not cancel the base model's license, dataset terms, trademarks,
or any other third-party rights. Contributors are responsible for checking
those terms before submitting an artifact.

Every adapter contribution must include the following certification in its
Hugging Face pull request or discussion. Replace the final line with the
contributor's real name, Hugging Face username, and date:

> **Contributor Public-Domain Certification**
>
> I certify that I created this contribution or otherwise have the authority
> to submit it. To the extent that I own copyright or related rights in the
> adapter and its accompanying metadata, I permanently dedicate those rights to
> the public domain under the Unlicense. Where a public-domain dedication is
> not legally recognized, I make the contribution available under all
> permissions and disclaimers stated by the Unlicense. I have disclosed the
> base model and calibration sources, and I am not knowingly submitting
> material that I lack permission to distribute. If I am contributing as part
> of my employment or for another organization, I certify that I am authorized
> to make this dedication on its behalf.
>
> `Signed-off-by: Full Name (@huggingface-username), YYYY-MM-DD`

The `Signed-off-by` line records the contributor's affirmative agreement to
this certification; merely uploading a file is not sufficient. Maintainers may
request a separately signed waiver or proof of organizational authorization
for substantial or employer-owned contributions. The certification and
sign-off should be retained in the repository's contribution history.

## Further reading

- [Miru Tracer](https://github.com/returnmoe/miru-tracer)
- [Jacobian Lens research workspace](https://transformer-circuits.pub/2026/workspace/index.html)
- [Anthropic's reference Jacobian-lens implementation](https://github.com/anthropics/jacobian-lens)

## Experimental status

Jacobians, lens readouts, and activation interventions are research tools. Fit
quality depends on the base checkpoint, calibration corpus, prompt count, and
fitting settings. Cross-check important conclusions with multiple methods and
do not treat an individual decoded token as a definitive explanation of a
model's internal computation.