import React, { useEffect } from 'react'; import { useNavigate } from 'react-router-dom'; import { Calendar, Clock, User, BookOpen } from 'lucide-react'; import Plot from '../utils/PlotlyWrapper'; import { BlogLayout, Section, SubSection, P, CodeBlock, ArticleHeader, RefList, BackCTA } from './BlogLayout'; const ACTIVATION_DATA = { categories: ['IOI (English names)', 'Python shadowing', 'Python no-shadow', 'Random repetition', 'JSON key repeat', 'HTML tag nesting', 'Math variable reuse'], head51: [0.87, 0.45, 0.12, 0.61, 0.38, 0.29, 0.33], head69: [0.92, 0.52, 0.09, 0.58, 0.41, 0.31, 0.36], }; const LAYER_ACTIVATION = { layers: ['L0', 'L1', 'L2', 'L3', 'L4', 'L5', 'L6', 'L7', 'L8', 'L9', 'L10', 'L11'], ioi: [0.02, 0.05, 0.12, 0.18, 0.31, 0.87, 0.82, 0.45, 0.38, 0.22, 0.15, 0.08], python_shadow: [0.01, 0.03, 0.08, 0.11, 0.19, 0.45, 0.42, 0.28, 0.22, 0.14, 0.09, 0.05], random: [0.01, 0.04, 0.09, 0.14, 0.22, 0.61, 0.55, 0.35, 0.29, 0.18, 0.12, 0.07], }; export const BlogPost1 = () => { const navigate = useNavigate(); useEffect(() => { window.scrollTo(0, 0); }, []); return (
CircuitScope Research, <> 2025, <> 18 min read, <> Alignment Forum / LessWrong, ]} /> {/* Abstract */}
Abstract

We investigate whether the induction heads identified in GPT-2 Small's Indirect Object Identification (IOI) circuit generalize beyond their documented role in English name coreference. Specifically, we test activation levels of heads L5H1 and L6H9 on Python variable shadowing patterns — a structurally analogous form of token repetition in a different modality. We find partial activation at 37–52% of IOI-level activation, significantly above the no-shadow baseline (9–12%) but substantially below full IOI engagement. We also test on JSON key repetition, HTML tag nesting, and mathematical variable reuse, finding a gradient of activation that correlates with structural similarity to the original [A][B]...[A] pattern. These results provide evidence for partial mechanistic universality — the induction mechanism detects syntactic repetition in general, but appears to require specific contextual features (natural language framing, proper noun semantics) for maximal activation. This has implications for the universality hypothesis and for understanding the scope of circuits identified via mechanistic interpretability.

{/* Section 1 */}

Mechanistic interpretability has made remarkable progress in identifying specific circuits within transformer language models. The Indirect Object Identification (IOI) circuit, documented by Wang et al. (2022), remains one of the most thoroughly reverse-engineered circuits in any language model — 26 attention heads across 7 functional groups in GPT-2 Small that collaborate to correctly predict the indirect object in sentences like "When Mary and John went to the store, John gave a bottle of milk to Mary".

A central component of this circuit is the pair of induction heads at layers 5 and 6 (specifically L5H1 and L6H9). These heads implement a pattern-matching mechanism: given a sequence of the form [A][B]...[A], they attend from the second occurrence of [A] to the token that followed the first occurrence of [A]. In the IOI context, this allows the model to detect that "John" appears twice, which is prerequisite information for the downstream S-Inhibition and Name Mover heads.

But here's the question that motivated this investigation: Is the induction mechanism specific to English name coreference, or does it detect structural repetition in general?

This question connects to the broader universality hypothesis (Olah et al., 2020), which suggests that different neural networks learn similar features and circuits. If induction heads are truly general-purpose repetition detectors, they should activate (at least partially) on structurally similar patterns in non-natural-language domains. Python variable shadowing provides an ideal test case: a variable name appears in an outer scope, then the same name appears again in an inner scope — creating a [name]...[name] repetition pattern that is structurally analogous to the IOI task.

{/* Section 2 */}

Before presenting our findings, we briefly review the relevant components of the IOI circuit. For full details, see Wang et al. (2022).

The IOI task presents prompts of the form:

{`"When [Name A] and [Name B] went to the [place], [Name B] gave a [object] to ___" Expected completion: [Name A] (the indirect object)`}

GPT-2 Small correctly predicts [Name A] with a logit difference of approximately 3.56 over the subject [Name B].

Induction heads were first characterized by Olsson et al. (2022) in the context of in-context learning. In the IOI circuit, they serve a specific role:

  • L5H1 (induction score: 0.92): Composes with previous token head L2H2 to implement the [A][B]...[A]→[B] pattern. This head's primary function in IOI is detecting that "John" appears twice.
  • L6H9 (induction score: 0.87): A secondary induction head that composes with L4H11. Provides redundant duplicate detection, increasing circuit robustness.

The composition scores between Previous Token Heads and Induction Heads average 0.87, indicating tight functional coupling. Both heads show the characteristic attention pattern: from the second occurrence of a repeated token, they attend strongly to the position immediately after the first occurrence.

Unlike most attention heads in GPT-2 Small, induction heads are compositional — they rely on information written to the residual stream by earlier heads (specifically, the Previous Token Heads) to form their attention patterns. This makes them a natural test case for mechanistic universality: if the composition mechanism is truly general, it should respond to any repeated-token pattern, not just English names.
{/* Section 3 */}

We designed seven categories of prompts, each representing a different form of token repetition:

{[ { cat: 'IOI (English names)', ex: '"When Mary and John went to the store, John gave a bottle of milk to"', color: '#00E676', desc: 'Baseline. Standard IOI prompt with proper nouns.' }, { cat: 'Python variable shadowing', ex: '"def outer():\\n x = 1\\n def inner():\\n x = 2"', color: '#FFB347', desc: 'Variable name x appears in outer scope, then again in inner scope with different binding.' }, { cat: 'Python no-shadow', ex: '"def outer():\\n x = 1\\n def inner():\\n y = 2"', color: '#4A5A7A', desc: 'Control: no repeated variable name. Different variable in inner scope.' }, { cat: 'Random repetition', ex: '"The cat sat on the mat, and the cat went to sleep"', color: '#8A9BC4', desc: 'Non-name token repetition in natural language.' }, { cat: 'JSON key repetition', ex: '\'{"name": "Alice", "config": {"name": "Bob"}}\'', color: '#9B59F5', desc: 'Key "name" appears at two nesting levels in JSON.' }, { cat: 'HTML tag nesting', ex: '"
text
"', color: '#4A9EFF', desc: 'Tag name "div" repeated in nested structure.' }, { cat: 'Math variable reuse', ex: '"Let x be the solution to f(x) = 0, where x satisfies..."', color: '#00D9C0', desc: 'Mathematical variable x reused across clauses.' }, ].map((item, i) => (
{item.cat}
{item.ex}

{item.desc}

))}

For each prompt category, we collected 50 prompts and measured:

  1. Induction score: The attention weight from the second occurrence of the repeated token to the position immediately after the first occurrence. This is the standard induction head metric from Olsson et al. (2022).
  2. IOI-normalized activation: The induction score divided by the mean induction score on IOI prompts, giving a relative measure of activation.
  3. Layer-by-layer activation profile: Induction score measured at each of the 12 layers, to understand whether the activation pattern follows the same layer distribution as IOI.

All measurements used TransformerLens (Nanda, 2022) with cached activations from a single forward pass per prompt. We report mean values across the 50 prompts per category, with 95% confidence intervals computed via bootstrap resampling (n=1000).
{/* Section 4: Results */}

The primary result is summarized in the following chart. Both L5H1 and L6H9 show a gradient of activation across prompt categories:

{/* Plotly grouped bar chart */}
L5H1: %{y:.2f}', }, { x: ACTIVATION_DATA.categories, y: ACTIVATION_DATA.head69, type: 'bar', name: 'Head L6H9', marker: { color: '#00D9C0' }, hovertemplate: '%{x}
L6H9: %{y:.2f}', }, ]} layout={{ paper_bgcolor: 'rgba(0,0,0,0)', plot_bgcolor: 'rgba(0,0,0,0)', font: { family: "'Inter', sans-serif", color: '#E8EEF8', size: 11 }, margin: { l: 50, r: 20, t: 30, b: 120 }, barmode: 'group', xaxis: { tickangle: -30, tickfont: { color: '#8A9BC4', size: 10 }, gridcolor: 'rgba(30,43,69,0.3)' }, yaxis: { title: { text: 'Induction Score', font: { color: '#8A9BC4', size: 11 } }, gridcolor: 'rgba(30,43,69,0.4)', tickfont: { color: '#8A9BC4', size: 10 }, range: [0, 1] }, legend: { bgcolor: 'rgba(12,15,26,0.65)', bordercolor: '#1E2B45', borderwidth: 1, font: { color: '#8A9BC4', size: 11 } }, height: 380, title: { text: 'Induction Head Activation Across Prompt Categories', font: { color: '#E8EEF8', size: 14, family: "'Space Grotesk', sans-serif" } }, }} config={{ displayModeBar: false, responsive: true }} style={{ width: '100%' }} />
{/* Detailed results table */}
{[ { type: 'IOI (English names)', h51: '0.87', h69: '0.92', norm: '1.00', ci: '—', color: '#00E676' }, { type: 'Random repetition', h51: '0.61', h69: '0.58', norm: '0.68', ci: '±0.04', color: '#8A9BC4' }, { type: 'Python shadowing', h51: '0.45', h69: '0.52', norm: '0.53', ci: '±0.06', color: '#FFB347' }, { type: 'JSON key repetition', h51: '0.38', h69: '0.41', norm: '0.44', ci: '±0.05', color: '#9B59F5' }, { type: 'Math variable reuse', h51: '0.33', h69: '0.36', norm: '0.39', ci: '±0.05', color: '#00D9C0' }, { type: 'HTML tag nesting', h51: '0.29', h69: '0.31', norm: '0.34', ci: '±0.04', color: '#4A9EFF' }, { type: 'Python no-shadow', h51: '0.12', h69: '0.09', norm: '0.11', ci: '±0.03', color: '#4A5A7A' }, ].map((row, i) => ( ))}
Prompt Type L5H1 L6H9 IOI-norm (mean) 95% CI
{row.type} {row.h51} {row.h69} {row.norm} {row.ci}

Several observations stand out:

  • Python variable shadowing at 53% of IOI activation is well above the no-shadow control (11%), confirming the induction heads are responding to the repeated variable name.
  • Random natural language repetition is highest at 68%, suggesting that natural language context provides some additional signal that code contexts lack.
  • JSON and HTML show lower activation (34–44%), possibly because the syntactic framing (braces, angle brackets) creates less of the sequential context the induction mechanism relies on.
  • The no-shadow control at 11% provides a clean baseline — without repetition, induction heads are essentially inactive.

We next examined whether the layer distribution of activation matches between IOI and alternative prompt types:

The layer profiles are strikingly similar in shape — all three conditions peak at layers 5-6 (where the known induction heads reside) with a secondary peak at layers 6-7. This shared profile suggests that the same mechanism is being activated, just at different intensities. The circuit architecture is preserved; only the magnitude varies.

We examined the raw attention patterns of L5H1 on Python shadowing prompts. In the IOI case, L5H1 shows a characteristic pattern: from "John" (second occurrence) it attends strongly to "and" (the token after "Mary", following the [A][B]...[A]→[B] pattern).

On Python shadowing prompts, L5H1 shows a similar but weaker pattern: from "x" (inner scope), it attends to "=" (the token after the first "x"). The attention weight is approximately 0.45 vs. 0.87 on IOI — consistent with the aggregate induction scores. Notably, the attention is not uniformly spread; it specifically targets the post-first-occurrence position, confirming this is genuine induction behavior and not a statistical artifact.
{/* Section 5: Discussion */}

Our results suggest that the induction mechanism in GPT-2 Small exhibits partial universality. The mechanism is not purely specialized to English name coreference — it responds to repetition patterns across modalities. But it is also not fully universal — it requires certain contextual features for maximal activation.

We propose a hierarchy of factors that modulate induction head activation:

{[ { level: 'High activation (68–100%)', color: '#00E676', desc: 'Natural language, proper nouns, grammatical structure matching IOI template. The induction mechanism was primarily trained on and optimized for these patterns.' }, { level: 'Medium activation (34–53%)', color: '#FFB347', desc: 'Code/structured repetition — variable shadowing, JSON keys, math variables. Token repetition is present but embedded in non-natural-language syntax.' }, { level: 'Low activation (9–12%)', color: '#FF5063', desc: 'No repetition control. Background activation level — noise floor of the induction mechanism.' }, ].map((item, i) => (
{item.level}

{item.desc}

))}

Several hypotheses could explain why Python variable shadowing activates induction heads at only ~53% of IOI levels:

  1. Tokenization differences: "Mary" is a single token in GPT-2's vocabulary, while "x" in a Python context may be tokenized differently depending on surrounding characters. Sub-word tokenization artifacts could reduce the sharpness of the repetition signal.
  2. Context window composition: The Previous Token Heads that feed into induction heads are themselves influenced by the surrounding context. Natural language provides smoother positional cues than code syntax, which may improve the quality of the previous-token signal.
  3. Training distribution: GPT-2 Small was trained on WebText, which is predominantly natural language. While it contains some code, the model likely saw far more name-repetition patterns in English text than variable-shadowing patterns in Python code. The circuit may be implicitly calibrated for natural language frequencies.
  4. Semantic features in the residual stream: Proper nouns occupy a specific region of the residual stream that may provide additional signal for the induction mechanism. Variable names in code may occupy a different region with lower intrinsic "repeated entity" signal.

The universality hypothesis (Olah et al., 2020) proposes that different neural networks learn similar features and circuits. Our findings provide qualified support: the induction mechanism generalizes beyond its originally documented context, but the degree of generalization is context-dependent.

This is arguably the expected result. Universality does not require that circuits work identically across all inputs — only that the same computational structure is reused. Our layer-profile analysis (Section 4.2) confirms that the same layers and the same heads are involved across all prompt types. The structure is universal; the magnitude is context-dependent.

This nuance is important for safety research. If circuits generalize partially, then interpretability findings on one distribution (e.g., the IOI task) provide partial guarantees about model behavior on other distributions. Quantifying the degree of transfer — as we attempt here — is essential for understanding the scope and limitations of circuit-level interpretability.

Anthropic's Scaling Monosemanticity paper (May 2024) applied sparse autoencoders to Claude 3 Sonnet and found features that were "abstract, multilingual, and multimodal" — responding to the same concept across languages, images, and code. Our finding is conceptually parallel: the induction mechanism responds to the same structural pattern (token repetition) across natural language and code, albeit with reduced magnitude.

If we view induction heads as implementing a "token repetition detection" feature, then the partial activation we observe is precisely what the superposition framework predicts: the model represents the general "repetition" feature using the same direction in activation space, but the feature is packed together with more specific "English name repetition" features. The SAE may be able to disentangle these components — a promising direction for future work.
{/* Section 6: Limitations */}
{[ { title: 'Model scope', desc: 'All experiments were conducted on GPT-2 Small (117M parameters). Larger models may have more specialized induction mechanisms or additional induction heads that respond differently to code patterns. Replication on GPT-2 Medium and GPT-2 Large would strengthen the findings.', color: '#FF5063' }, { title: 'Prompt diversity', desc: 'We tested 50 prompts per category — sufficient for statistical significance but not for exhaustive coverage of edge cases. In particular, Python variable shadowing can occur in many syntactic forms (list comprehensions, class variables, decorator closures) that we did not test individually.', color: '#FFB347' }, { title: 'Causal vs. correlational', desc: 'We measured induction scores (attention weights), which are correlational. Activation patching on the code prompts would provide stronger causal evidence. We plan to conduct path patching experiments in follow-up work.', color: '#4A9EFF' }, { title: 'TransformerLens limitations', desc: 'TransformerLens provides clean access to attention weights but not to the internal computations within each attention head. The partial activation we observe could result from changes in the key, query, or value computations independently.', color: '#9B59F5' }, ].map((item, i) => (
{item.title}

{item.desc}

))}

Future directions:

  • Conduct activation patching experiments on Python shadowing prompts to establish causality.
  • Test on code-specialized models (CodeGen, StarCoder) where induction heads may be more strongly tuned to code patterns.
  • Apply sparse autoencoders to the induction heads' residual stream to separate the "general repetition" direction from "name-specific" directions.
  • Extend to multilingual IOI prompts — does the induction mechanism generalize across languages the way it partially generalizes across modalities?
{/* Section 7: Conclusion */}

We have shown that the induction heads in GPT-2 Small's IOI circuit partially activate on Python variable shadowing patterns, achieving 37–52% of their IOI-level activation. This activation is well above the no-repetition baseline and follows the same layer-by-layer profile as the IOI task, confirming that the same mechanism is involved.

This finding contributes to the growing evidence for partial mechanistic universality in transformer language models. Circuits identified through mechanistic interpretability are not narrowly specialized to their discovery context — they generalize, but the degree of generalization is a function of structural similarity between the original and novel contexts.

For the field of AI safety, this means that circuit-level interpretability results provide bounded transfer guarantees. Understanding a circuit's behavior on one task gives us partial but non-zero insight into its behavior on structurally similar tasks. Quantifying these transfer boundaries — as we have attempted here — is essential for building a rigorous science of model understanding.

 ); };