---
title: Bertographer
emoji: 🔭
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 5.50.0
app_file: app.py
pinned: false
license: mit
short_description: Mechanistic-interpretability cockpit for encoder models
---

# Bertographer

A mechanistic-interpretability **cockpit for encoder *classifiers*** (BERT /
RoBERTa / DeBERTa / DistilBERT / ELECTRA), framed in **production-observability**
language rather than lab notation.

Where a decoder probe reads a model's next-token futures, Bertographer reads an
encoder's **decision**: for every layer and attention head it projects the
pooling token's register into two spaces —

- **vocab** (logit-lens via the tied word embeddings) — what token this register "says"
- **class** (the model's own classification head) — which label this head votes for

…and lets an operator intervene (**mute / scale a head**) and watch the
prediction move. The framing is the engine room, not the notebook:

- input = a **trace** · the layer stack = the **waterfall** · heads = **spans / voters**
- mute / scale = the **counterfactual** ("what if the model hadn't seen this evidence?")
- compare = a **trace‑vs‑trace diff** ("what differed between this request and the last?")

Default model: `cardiffnlp/twitter-roberta-base-sentiment-latest` (3‑class
sentiment). Architecture is auto‑discovered; per‑head views degrade gracefully
when a model's attention output isn't a clean per‑head concatenation.

---

Built by **James J. Davison**, with Claude (Anthropic) as coding collaborator. **Responsibility for the code and its design choices rests with the human author.** MIT licensed. © 2026 James J. Davison.