frontend/src/components/BlogPost2.js

import React, { useEffect } from 'react';
import { useNavigate } from 'react-router-dom';
import { Calendar, Clock, User, BookOpen } from 'lucide-react';
import Plot from '../utils/PlotlyWrapper';
import { BlogLayout, Section, SubSection, P, CodeBlock, ArticleHeader, RefList, BackCTA } from './BlogLayout';

const ABLATION_DATA = {
  conditions: ['Full model', 'Primary NM ablated', 'Primary + Backup ablated', 'All NM ablated', 'Full circuit ablated'],
  logit_diff: [3.56, 2.35, 1.21, 0.52, 0.46],
  recovery_pct: [100, 66, 34, 15, 13],
};

const BACKUP_ACTIVATION = {
  heads: ['L10H2', 'L11H2', 'L10H6', 'L11H9'],
  normal: [0.12, 0.08, 0.10, 0.06],
  after_ablation: [0.67, 0.51, 0.58, 0.43],
};

export const BlogPost2 = () => {
  const navigate = useNavigate();
  useEffect(() => { window.scrollTo(0, 0); }, []);

  return (
    <BlogLayout>
      <article className="section-container" style={{ maxWidth: 820, padding: '60px 16px 80px' }}>
        <ArticleHeader
          badges={[{ text: 'Circuit Analysis', color: 'teal' }, { text: 'Ablation Study', color: 'amber' }, { text: 'GPT-2 Small', color: 'violet' }]}
          title="Backup Name Mover Heads: The Hidden Redundancy in GPT-2's IOI Circuit"
          meta={[
            <><User size={14} /> CircuitScope Research</>,
            <><Calendar size={14} /> 2025</>,
            <><Clock size={14} /> 14 min read</>,
            <><BookOpen size={14} /> Alignment Forum</>,
          ]}
        />

        {/* Abstract */}
        <div className="research-card-accent mb-10" style={{ borderLeftColor: '#00D9C0' }}>
          <div style={{ fontSize: 11, fontWeight: 600, color: '#00D9C0', textTransform: 'uppercase', letterSpacing: '0.08em', marginBottom: 8 }}>Abstract</div>
          <P>We systematically ablate the primary Name Mover Heads in GPT-2 Small's IOI circuit and discover that a set of previously dormant heads at layers 10–11 activate to partially compensate. These "Backup Name Mover Heads" recover approximately <strong style={{ color: '#FFB347' }}>34% of the original logit difference</strong> when the primary heads are knocked out — a recovery that was not observed during normal forward passes. This finding suggests that transformer models develop <strong style={{ color: '#00D9C0' }}>fault-tolerant circuits during training</strong>, maintaining redundant pathways that activate only under component failure. We characterize the activation dynamics of these backup heads, measure their composition with upstream circuit components, and discuss implications for interpretability and alignment research.</P>
        </div>

        <Section num="1" title="Introduction">
          <P>The IOI circuit in GPT-2 Small contains 26 attention heads organized into 7 functional groups (Wang et al., 2022). Among these, the <strong style={{ color: '#00E676' }}>Name Mover Heads</strong> (L9H9, L9H6, L10H0, L10H7, L10H10) are the primary output mechanism — they attend from the final token position to the indirect object (IO) name and write it to the output logits.</P>

          <P>During our replication of the IOI circuit, we noticed something unexpected in the ablation experiments: <strong style={{ color: '#FFB347' }}>when we knocked out the primary Name Mover Heads, model performance did not drop to zero</strong>. Instead, it dropped to approximately 34% of the original logit difference — far more than the ~13% retained by the circuit-knockout baseline.</P>

          <P>This gap pointed to the existence of <em>backup mechanisms</em> — heads that are normally dormant but activate to partially compensate when the primary circuit components fail. Wang et al. (2022) briefly mentioned these backup heads, but did not characterize their activation dynamics in detail. We investigate this redundancy and its implications.</P>
        </Section>

        <Section num="2" title="Experimental Setup">
          <SubSection title="2.1 Progressive Ablation Protocol">
            <P>We designed a progressive ablation experiment with five conditions:</P>
            <ol style={{ fontSize: 14, color: '#8A9BC4', lineHeight: 1.8, paddingLeft: 20, marginBottom: 16 }}>
              <li style={{ marginBottom: 8 }}><strong style={{ color: '#E8EEF8' }}>Full model:</strong> No ablation. Baseline logit difference = 3.56.</li>
              <li style={{ marginBottom: 8 }}><strong style={{ color: '#E8EEF8' }}>Primary NM ablated:</strong> Zero-ablation of L9H9, L9H6, L10H0, L10H7, L10H10.</li>
              <li style={{ marginBottom: 8 }}><strong style={{ color: '#E8EEF8' }}>Primary + Backup ablated:</strong> Additionally ablate L10H2, L11H2, L10H6, L11H9.</li>
              <li style={{ marginBottom: 8 }}><strong style={{ color: '#E8EEF8' }}>All Name Mover-type ablated:</strong> Additionally ablate Negative Name Movers (L10H1, L11H10).</li>
              <li style={{ marginBottom: 8 }}><strong style={{ color: '#E8EEF8' }}>Full circuit ablated:</strong> All 26 IOI heads ablated.</li>
            </ol>
          </SubSection>

          <SubSection title="2.2 Activation Measurement">
            <P>For each backup head, we measured the attention weight from the final token position to the IO name token under both normal and primary-ablated conditions. We also measured the cosine similarity between each backup head's output vector and the IO name's embedding direction, as a proxy for "Name Mover behavior."</P>
          </SubSection>
        </Section>

        <Section num="3" title="Results">
          <SubSection title="3.1 Progressive Ablation Results">
            <P>The progressive ablation reveals a clear redundancy structure:</P>
            <div className="research-card mb-6 plotly-container" style={{ overflowX: 'auto' }}>
              <div style={{ minWidth: 500 }}>
                <Plot
                  data={[{
                    x: ABLATION_DATA.conditions,
                    y: ABLATION_DATA.logit_diff,
                    type: 'bar',
                    marker: {
                      color: ABLATION_DATA.logit_diff.map((v, i) =>
                        i === 0 ? '#00E676' : i === 1 ? '#FFB347' : i === 2 ? '#FF5063' : i === 3 ? '#FF5063' : '#4A5A7A'
                      ),
                    },
                    hovertemplate: '%{x}<br>Logit Diff: %{y:.2f}<br>Recovery: ' +
                      ABLATION_DATA.recovery_pct.map(p => p + '%').join(',') + '<extra></extra>',
                    text: ABLATION_DATA.logit_diff.map(v => v.toFixed(2)),
                    textposition: 'outside',
                    textfont: { color: '#E8EEF8', size: 12, family: "'JetBrains Mono', monospace" },
                  }]}
                  layout={{
                    paper_bgcolor: 'rgba(0,0,0,0)', plot_bgcolor: 'rgba(0,0,0,0)',
                    font: { family: "'Inter', sans-serif", color: '#E8EEF8', size: 11 },
                    margin: { l: 50, r: 20, t: 40, b: 120 },
                    xaxis: { tickangle: -20, tickfont: { color: '#8A9BC4', size: 10 }, gridcolor: 'rgba(30,43,69,0.3)' },
                    yaxis: { title: { text: 'Logit Difference', font: { color: '#8A9BC4', size: 11 } }, gridcolor: 'rgba(30,43,69,0.4)', tickfont: { color: '#8A9BC4', size: 10 }, range: [0, 4.2] },
                    height: 380,
                    title: { text: 'Progressive Ablation of Name Mover Heads', font: { color: '#E8EEF8', size: 14, family: "'Space Grotesk', sans-serif" } },
                    annotations: [
                      { x: 'Primary NM ablated', y: 2.55, text: 'Backup heads<br>recover 34%', showarrow: true, arrowhead: 2, arrowcolor: '#FFB347', font: { color: '#FFB347', size: 10 }, ax: 60, ay: -30 },
                    ],
                  }}
                  config={{ displayModeBar: false, responsive: true }}
                  style={{ width: '100%' }}
                />
              </div>
            </div>

            <P>The critical observation: the gap between "Primary NM ablated" (2.35) and "Primary + Backup ablated" (1.21) represents <strong style={{ color: '#FFB347' }}>1.14 logit difference units</strong> — the contribution of the backup heads that were previously dormant. This represents 32% of the full model's performance, emerging only under failure conditions.</P>
          </SubSection>

          <SubSection title="3.2 Backup Head Activation Dynamics">
            <P>The most striking finding is the dramatic change in backup head behavior under ablation:</P>

            <div className="research-card mb-6 plotly-container" style={{ overflowX: 'auto' }}>
              <div style={{ minWidth: 500 }}>
                <Plot
                  data={[
                    { x: BACKUP_ACTIVATION.heads, y: BACKUP_ACTIVATION.normal, type: 'bar', name: 'Normal operation', marker: { color: '#4A5A7A' } },
                    { x: BACKUP_ACTIVATION.heads, y: BACKUP_ACTIVATION.after_ablation, type: 'bar', name: 'After primary ablation', marker: { color: '#FFB347' } },
                  ]}
                  layout={{
                    paper_bgcolor: 'rgba(0,0,0,0)', plot_bgcolor: 'rgba(0,0,0,0)',
                    font: { family: "'Inter', sans-serif", color: '#E8EEF8', size: 11 },
                    margin: { l: 50, r: 20, t: 40, b: 50 },
                    barmode: 'group',
                    xaxis: { tickfont: { color: '#8A9BC4', size: 11 }, gridcolor: 'rgba(30,43,69,0.3)' },
                    yaxis: { title: { text: 'Attention to IO Name', font: { color: '#8A9BC4', size: 11 } }, gridcolor: 'rgba(30,43,69,0.4)', tickfont: { color: '#8A9BC4', size: 10 }, range: [0, 0.8] },
                    legend: { bgcolor: 'rgba(12,15,26,0.65)', bordercolor: '#1E2B45', borderwidth: 1, font: { color: '#8A9BC4', size: 11 } },
                    height: 340,
                    title: { text: 'Backup Head Activation: Normal vs Post-Ablation', font: { color: '#E8EEF8', size: 14, family: "'Space Grotesk', sans-serif" } },
                  }}
                  config={{ displayModeBar: false, responsive: true }}
                  style={{ width: '100%' }}
                />
              </div>
            </div>

            <P>L10H2 shows the most dramatic shift: from 0.12 attention weight to IO under normal conditions to 0.67 after primary ablation — a <strong style={{ color: '#FFB347' }}>5.6× increase</strong>. This is not a gradual shift; it's a binary activation, as if a dormant pathway switches on when the primary fails.</P>
          </SubSection>
        </Section>

        <Section num="4" title="Discussion">
          <SubSection title="4.1 Implications for Circuit Completeness">
            <P>The existence of backup Name Movers challenges the notion of "circuit completeness." The standard metric — knocking out the 26 identified heads and measuring performance degradation — reports 91% completeness. But this metric doesn't account for the <em>dynamic</em> nature of the circuit: some components only activate when others fail.</P>
            <P>This means the "true" circuit is larger than the 26 heads identified under normal operating conditions. It includes dormant components that serve as <strong style={{ color: '#00D9C0' }}>a fault-tolerance mechanism</strong>. From an interpretability standpoint, this is concerning: circuits identified through forward-pass analysis may systematically undercount the model's actual computational resources.</P>
          </SubSection>

          <SubSection title="4.2 How Does the Model Know to Activate Backups?">
            <P>The most puzzling aspect is the mechanism by which backup heads "know" to activate. We hypothesize that the residual stream carries a signal indicating whether the Name Mover computation has been performed. When primary Name Movers are active, they write a strong IO-direction signal to the residual stream at the final position. This signal may <em>suppress</em> the backup heads via a learned inhibition mechanism.</P>
            <P>When the primary heads are ablated, this inhibition signal is absent, allowing the backup heads to activate. This is consistent with the model learning a form of <strong style={{ color: '#9B59F5' }}>error-correcting code</strong> during training — not through explicit design, but through gradient descent on next-token prediction.</P>
          </SubSection>

          <SubSection title="4.3 Implications for AI Safety">
            <P>If models develop hidden redundant circuits that activate under failure, this has direct implications for safety:</P>
            <ul style={{ fontSize: 14, color: '#8A9BC4', lineHeight: 1.8, paddingLeft: 20, marginBottom: 16 }}>
              <li style={{ marginBottom: 8 }}><strong style={{ color: '#FF5063' }}>Ablation-based safety measures may be insufficient.</strong> If we identify and ablate a harmful circuit, backup mechanisms may partially restore the behavior.</li>
              <li style={{ marginBottom: 8 }}><strong style={{ color: '#FFB347' }}>Circuit analysis must consider dynamic activation.</strong> Static forward-pass analysis underestimates the model's computational redundancy.</li>
              <li style={{ marginBottom: 8 }}><strong style={{ color: '#00E676' }}>Redundancy is also good news.</strong> If the model develops backup circuits for benign tasks (like IOI), it may also develop backup circuits for aligned behaviors — making alignment more robust than single-circuit analysis suggests.</li>
            </ul>
          </SubSection>
        </Section>

        <Section num="5" title="Conclusion">
          <div className="research-card-accent" style={{ borderLeftColor: '#00D9C0' }}>
            <P>We have characterized the Backup Name Mover Heads in GPT-2 Small's IOI circuit, demonstrating that they activate from dormancy (0.06–0.12 attention) to near-primary levels (0.43–0.67) when the primary Name Movers are ablated. This redundancy accounts for approximately 34% of the model's IOI performance and was discovered accidentally during routine ablation experiments.</P>
            <P style={{ marginBottom: 0 }}>The finding illustrates a broader principle: <em>transformer models are not static computational graphs</em>. They are dynamic systems with latent capacity that activates under perturbation. Any complete theory of mechanistic interpretability must account for this dynamic behavior.</P>
          </div>
        </Section>

        <RefList refs={[
          { authors: 'Wang, K., Variengien, A., Conmy, A., Shlegeris, B., & Steinhardt, J.', year: '2022', title: 'Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 Small', venue: 'arXiv:2211.00593', url: 'https://arxiv.org/abs/2211.00593' },
          { authors: 'Conmy, A., Mavor-Parker, A., Lynch, A., Heimersheim, S., & Garriga-Alonso, A.', year: '2023', title: 'Towards Automated Circuit Discovery for Mechanistic Interpretability', venue: 'NeurIPS 2023', url: 'https://arxiv.org/abs/2304.14997' },
          { authors: 'McGrath, T., Rahtz, M., Kramar, J., Mikulik, V., & Legg, S.', year: '2023', title: 'The Hydra Effect: Emergent Self-repair in Language Model Computations', venue: 'arXiv:2307.15771', url: 'https://arxiv.org/abs/2307.15771' },
          { authors: 'Nanda, N.', year: '2022', title: 'TransformerLens', venue: 'GitHub', url: 'https://github.com/TransformerLensOrg/TransformerLens' },
        ]} />

        <BackCTA />
      </article>
    </BlogLayout>
  );
};