frontend/src/components/BlogPost4.js

import React, { useEffect } from 'react';
import { useNavigate } from 'react-router-dom';
import { Calendar, Clock, User, BookOpen } from 'lucide-react';
import Plot from '../utils/PlotlyWrapper';
import { BlogLayout, Section, SubSection, P, CodeBlock, ArticleHeader, RefList, BackCTA } from './BlogLayout';

const RESAMPLING_DATA = {
  steps: [0, 25000, 50000, 75000, 100000, 125000, 150000, 175000, 200000],
  no_resample: [25.0, 22.0, 19.5, 18.2, 17.5, 17.0, 16.8, 16.5, 16.2],
  resample_50k: [25.0, 22.0, 19.5, 18.2, 12.0, 10.5, 9.8, 8.5, 7.2],
  resample_25k: [25.0, 22.0, 15.0, 12.5, 10.0, 8.5, 7.8, 7.3, 7.0],
  ghost_grad: [25.0, 18.0, 13.0, 10.0, 8.0, 6.8, 6.2, 5.9, 5.5],
};

const DEAD_FEATURE_ANALYSIS = {
  categories: ['Never activated', 'Ultra-low density (<0.01%)', 'Low density (0.01-0.1%)', 'Medium density (0.1-1%)', 'High density (>1%)'],
  counts: [287, 205, 614, 1638, 1352],
  colors: ['#FF5063', '#FFB347', '#4A9EFF', '#00D9C0', '#00E676'],
};

const L1_SWEEP = {
  l1_values: ['5e-4', '8e-4', '1e-3', '2e-3', '5e-3', '1e-2'],
  dead_pct: [3.2, 5.1, 7.0, 12.5, 24.8, 41.2],
  recon_loss: [0.18, 0.14, 0.12, 0.11, 0.14, 0.21],
};

export const BlogPost4 = () => {
  const navigate = useNavigate();
  useEffect(() => { window.scrollTo(0, 0); }, []);

  return (
    <BlogLayout>
      <article className="section-container" style={{ maxWidth: 820, padding: '60px 16px 80px' }}>
        <ArticleHeader
          badges={[{ text: 'SAE Training', color: 'red' }, { text: 'Dead Features', color: 'amber' }, { text: 'Optimization', color: 'teal' }]}
          title="The Dead Feature Graveyard: Neuron Resampling and the Limits of Sparse Autoencoders"
          meta={[
            <><User size={14} /> CircuitScope Research</>,
            <><Calendar size={14} /> 2025</>,
            <><Clock size={14} /> 12 min read</>,
            <><BookOpen size={14} /> LessWrong</>,
          ]}
        />

        <div className="research-card-accent mb-10" style={{ borderLeftColor: '#FFB347' }}>
          <div style={{ fontSize: 11, fontWeight: 600, color: '#FFB347', textTransform: 'uppercase', letterSpacing: '0.08em', marginBottom: 8 }}>Abstract</div>
          <P>Dead features — sparse autoencoder directions that never activate on validation data — are a persistent problem in SAE training. We systematically compare four mitigation strategies: no resampling, resampling every 50K steps, resampling every 25K steps, and ghost gradients (Anthropic's approach). Neuron resampling reduces dead features from <strong style={{ color: '#FFB347' }}>25% to ~7%</strong>, but no strategy we tested reduces them below 5.5%. We characterize the remaining dead features, identify a fundamental tension between L1 sparsity and feature utilization, and propose that dead features may represent a <strong style={{ color: '#FF5063' }}>necessary cost of overcomplete representation</strong>.</P>
        </div>

        <Section num="1" title="The Problem: Dead Features">
          <P>When training a sparse autoencoder on GPT-2 Small's layer 6 MLP activations (4,096 features from 512 neurons), a significant fraction of the learned features never activate on any input. These "dead features" consume model capacity but represent nothing interpretable.</P>

          <P>The root cause is well understood: <strong style={{ color: '#FF5063' }}>random initialization creates encoder directions with no training signal</strong>. Because the ReLU activation function has zero gradient for inactive features, gradient descent cannot move dead features toward useful directions. They remain stuck at their random initial positions, never receiving the gradient signal needed to become useful.</P>

          <P>This is not merely a cosmetic issue. Dead features directly reduce the effective capacity of the SAE. If 25% of features are dead, the SAE effectively has only 3,072 features instead of 4,096 — a significant reduction in its ability to decompose superposed representations.</P>
        </Section>

        <Section num="2" title="Mitigation Strategies">
          <SubSection title="2.1 Neuron Resampling">
            <P>The standard mitigation is neuron resampling (introduced by Anthropic): periodically identify dead features and reinitialize them by:</P>
            <ol style={{ fontSize: 14, color: '#8A9BC4', lineHeight: 1.8, paddingLeft: 20, marginBottom: 16 }}>
              <li style={{ marginBottom: 8 }}>Finding training examples with the highest reconstruction loss (examples the current SAE fails on).</li>
              <li style={{ marginBottom: 8 }}>Setting the dead feature's encoder weights to point toward these high-loss examples, plus noise.</li>
              <li style={{ marginBottom: 8 }}>Resetting the decoder weights and scaling the learning rate up temporarily.</li>
            </ol>
            <P>We tested two resampling frequencies: every 50K steps (Anthropic's default) and every 25K steps (more aggressive).</P>
          </SubSection>

          <SubSection title="2.2 Ghost Gradients">
            <P>Ghost gradients (also from Anthropic) provide an alternative: instead of resampling, add a synthetic gradient for dead features that pulls them toward high-loss regions. This avoids the discontinuity of resampling while continuously nudging dead features toward useful directions.</P>
            <CodeBlock>{`# Ghost gradient computation (simplified)
for dead_feature in dead_features:
    # Compute gradient as if the feature had barely activated
    ghost_activation = epsilon  # small positive value
    ghost_grad = d_loss / d_activation * ghost_activation
    # Add to the encoder weight gradient
    encoder.weight.grad[dead_feature] += ghost_grad`}</CodeBlock>
          </SubSection>
        </Section>

        <Section num="3" title="Results">
          <SubSection title="3.1 Resampling Strategy Comparison">
            <P>We tracked the percentage of dead features throughout 200K training steps under each strategy:</P>

            <div className="research-card mb-6 plotly-container" style={{ overflowX: 'auto' }}>
              <div style={{ minWidth: 500 }}>
                <Plot
                  data={[
                    { x: RESAMPLING_DATA.steps, y: RESAMPLING_DATA.no_resample, type: 'scatter', mode: 'lines+markers', name: 'No resampling', line: { color: '#FF5063', width: 2 }, marker: { size: 5 } },
                    { x: RESAMPLING_DATA.steps, y: RESAMPLING_DATA.resample_50k, type: 'scatter', mode: 'lines+markers', name: 'Resample @ 50K', line: { color: '#FFB347', width: 2 }, marker: { size: 5 } },
                    { x: RESAMPLING_DATA.steps, y: RESAMPLING_DATA.resample_25k, type: 'scatter', mode: 'lines+markers', name: 'Resample @ 25K', line: { color: '#4A9EFF', width: 2 }, marker: { size: 5 } },
                    { x: RESAMPLING_DATA.steps, y: RESAMPLING_DATA.ghost_grad, type: 'scatter', mode: 'lines+markers', name: 'Ghost gradients', line: { color: '#00D9C0', width: 2 }, marker: { size: 5 } },
                  ]}
                  layout={{
                    paper_bgcolor: 'rgba(0,0,0,0)', plot_bgcolor: 'rgba(0,0,0,0)',
                    font: { family: "'Inter', sans-serif", color: '#E8EEF8', size: 11 },
                    margin: { l: 50, r: 20, t: 40, b: 50 },
                    xaxis: { title: { text: 'Training Step', font: { color: '#8A9BC4', size: 11 } }, tickfont: { color: '#8A9BC4', size: 10 }, gridcolor: 'rgba(30,43,69,0.4)' },
                    yaxis: { title: { text: 'Dead Features (%)', font: { color: '#8A9BC4', size: 11 } }, tickfont: { color: '#8A9BC4', size: 10 }, gridcolor: 'rgba(30,43,69,0.4)' },
                    legend: { bgcolor: 'rgba(12,15,26,0.65)', bordercolor: '#1E2B45', borderwidth: 1, font: { color: '#8A9BC4', size: 11 } },
                    height: 380,
                    title: { text: 'Dead Feature Percentage Over Training', font: { color: '#E8EEF8', size: 14, family: "'Space Grotesk', sans-serif" } },
                    annotations: [
                      { x: 50000, y: 12, text: '1st resample', showarrow: true, arrowhead: 2, arrowcolor: '#FFB347', font: { color: '#FFB347', size: 9 }, ax: 0, ay: -25 },
                    ],
                  }}
                  config={{ displayModeBar: false, responsive: true }}
                  style={{ width: '100%' }}
                />
              </div>
            </div>

            <div style={{ overflowX: 'auto', marginBottom: 24 }}>
              <table style={{ width: '100%', borderCollapse: 'collapse', fontSize: 13, minWidth: 500 }}>
                <thead>
                  <tr style={{ borderBottom: '2px solid #1E2B45' }}>
                    <th style={{ textAlign: 'left', padding: '10px 12px', color: '#E8EEF8', fontWeight: 600, fontSize: 12 }}>Strategy</th>
                    <th style={{ textAlign: 'center', padding: '10px 12px', color: '#FF5063', fontWeight: 600, fontSize: 12 }}>Final Dead %</th>
                    <th style={{ textAlign: 'center', padding: '10px 12px', color: '#00D9C0', fontWeight: 600, fontSize: 12 }}>Recon Loss</th>
                    <th style={{ textAlign: 'center', padding: '10px 12px', color: '#9B59F5', fontWeight: 600, fontSize: 12 }}>Interpretability</th>
                  </tr>
                </thead>
                <tbody>
                  {[
                    { name: 'No resampling', dead: '16.2%', loss: '0.13', interp: '58%', color: '#FF5063' },
                    { name: 'Resample @ 50K', dead: '7.2%', loss: '0.12', interp: '70%', color: '#FFB347' },
                    { name: 'Resample @ 25K', dead: '7.0%', loss: '0.12', interp: '69%', color: '#4A9EFF' },
                    { name: 'Ghost gradients', dead: '5.5%', loss: '0.13', interp: '68%', color: '#00D9C0' },
                  ].map((row, i) => (
                    <tr key={i} style={{ borderBottom: '1px solid #1E2B4544' }}>
                      <td style={{ padding: '8px 12px', color: row.color, fontWeight: 500 }}>{row.name}</td>
                      <td style={{ padding: '8px 12px', textAlign: 'center', color: '#E8EEF8', fontFamily: "'JetBrains Mono', monospace" }}>{row.dead}</td>
                      <td style={{ padding: '8px 12px', textAlign: 'center', color: '#E8EEF8', fontFamily: "'JetBrains Mono', monospace" }}>{row.loss}</td>
                      <td style={{ padding: '8px 12px', textAlign: 'center', color: '#E8EEF8', fontFamily: "'JetBrains Mono', monospace" }}>{row.interp}</td>
                    </tr>
                  ))}
                </tbody>
              </table>
            </div>

            <P><strong style={{ color: '#00D9C0' }}>Ghost gradients achieve the lowest dead feature rate (5.5%)</strong> but at the cost of slightly higher reconstruction loss (0.13 vs 0.12). Resampling at 50K and 25K converge to similar rates (~7%), suggesting a floor exists regardless of resampling frequency.</P>
          </SubSection>

          <SubSection title="3.2 The L1-Sparsity Tension">
            <P>The fundamental issue: L1 regularization encourages sparsity, but extreme sparsity kills features. We performed a sweep over L1 coefficient values:</P>

            <div className="research-card mb-6 plotly-container" style={{ overflowX: 'auto' }}>
              <div style={{ minWidth: 500 }}>
                <Plot
                  data={[
                    { x: L1_SWEEP.l1_values, y: L1_SWEEP.dead_pct, type: 'bar', name: 'Dead features (%)', marker: { color: '#FF5063' }, yaxis: 'y' },
                    { x: L1_SWEEP.l1_values, y: L1_SWEEP.recon_loss, type: 'scatter', mode: 'lines+markers', name: 'Reconstruction loss', line: { color: '#00D9C0', width: 2 }, marker: { size: 6 }, yaxis: 'y2' },
                  ]}
                  layout={{
                    paper_bgcolor: 'rgba(0,0,0,0)', plot_bgcolor: 'rgba(0,0,0,0)',
                    font: { family: "'Inter', sans-serif", color: '#E8EEF8', size: 11 },
                    margin: { l: 50, r: 60, t: 40, b: 50 },
                    xaxis: { title: { text: 'L1 Coefficient', font: { color: '#8A9BC4', size: 11 } }, tickfont: { color: '#8A9BC4', size: 10 } },
                    yaxis: { title: { text: 'Dead Features (%)', font: { color: '#FF5063', size: 11 } }, gridcolor: 'rgba(30,43,69,0.4)', tickfont: { color: '#FF5063', size: 10 }, side: 'left' },
                    yaxis2: { title: { text: 'Recon Loss', font: { color: '#00D9C0', size: 11 } }, tickfont: { color: '#00D9C0', size: 10 }, overlaying: 'y', side: 'right' },
                    legend: { bgcolor: 'rgba(12,15,26,0.65)', bordercolor: '#1E2B45', borderwidth: 1, font: { color: '#8A9BC4', size: 11 } },
                    height: 360,
                    title: { text: 'L1 Coefficient vs Dead Features & Reconstruction Loss', font: { color: '#E8EEF8', size: 14, family: "'Space Grotesk', sans-serif" } },
                    annotations: [{ x: '1e-3', y: 7.0, text: 'Sweet spot', showarrow: true, arrowhead: 2, arrowcolor: '#FFB347', font: { color: '#FFB347', size: 10 }, ax: 40, ay: -25 }],
                  }}
                  config={{ displayModeBar: false, responsive: true }}
                  style={{ width: '100%' }}
                />
              </div>
            </div>

            <P>The "sweet spot" at L1=1e-3 balances dead features (7%) and reconstruction loss (0.12). Lower L1 reduces dead features but increases reconstruction loss (features aren't sparse enough). Higher L1 kills more features. <strong style={{ color: '#FFB347' }}>There is no L1 value that simultaneously minimizes both metrics</strong> — this is a fundamental tension in the SAE objective.</P>
          </SubSection>

          <SubSection title="3.3 Characterizing the Surviving Dead Features">
            <P>Even with optimal resampling, 287 features (7%) remain completely dead. We analyzed their encoder weight distributions:</P>
            <div className="research-card mb-6" style={{ overflowX: 'auto' }}>
              <div style={{ minWidth: 450 }}>
                <Plot
                  data={[{
                    labels: DEAD_FEATURE_ANALYSIS.categories,
                    values: DEAD_FEATURE_ANALYSIS.counts,
                    type: 'pie',
                    hole: 0.45,
                    marker: { colors: DEAD_FEATURE_ANALYSIS.colors },
                    textinfo: 'label+percent',
                    textfont: { color: '#E8EEF8', size: 11 },
                    hovertemplate: '%{label}<br>Count: %{value}<br>%{percent}<extra></extra>',
                  }]}
                  layout={{
                    paper_bgcolor: 'rgba(0,0,0,0)', plot_bgcolor: 'rgba(0,0,0,0)',
                    font: { family: "'Inter', sans-serif", color: '#E8EEF8', size: 11 },
                    margin: { l: 20, r: 20, t: 40, b: 20 },
                    height: 360,
                    showlegend: false,
                    title: { text: 'Feature Density Distribution (4,096 features)', font: { color: '#E8EEF8', size: 14, family: "'Space Grotesk', sans-serif" } },
                  }}
                  config={{ displayModeBar: false, responsive: true }}
                  style={{ width: '100%' }}
                />
              </div>
            </div>
            <P>The dead + ultra-low-density features (12%) cluster in specific regions of the encoder weight space: they point toward directions that are nearly orthogonal to the activation manifold. These directions may represent <em>real features that are too rare in the training distribution</em> — the SAE has learned to represent them but they simply don't appear in our validation set.</P>
          </SubSection>
        </Section>

        <Section num="4" title="Discussion: Are Dead Features Actually a Problem?">
          <P>We propose a reframing: dead features may not be a bug — they may be a <strong style={{ color: '#00D9C0' }}>necessary cost of overcomplete representation</strong>.</P>
          <P>An 8× expansion SAE creates 4,096 directions to represent concepts in a 512-dimensional space. The model may genuinely need many of these directions to capture rare but real features. A "DNA genomics" feature might activate on only 0.001% of text, but when it does, it's critically important for understanding the model's behavior.</P>
          <P>Under this view, the "dead" features are <em>dormant</em> — they represent real concepts that our validation set simply doesn't contain. If we evaluated on a broader distribution (e.g., including scientific papers, legal documents, and code), some of these "dead" features would spring to life.</P>

          <div className="grid grid-cols-1 md:grid-cols-2 gap-4 my-6">
            <div className="research-card" style={{ borderTop: '2px solid #FF5063', padding: '14px 16px' }}>
              <div style={{ fontSize: 13, fontWeight: 600, color: '#FF5063', marginBottom: 6 }}>The pessimistic view</div>
              <P style={{ marginBottom: 0, fontSize: 13 }}>Dead features are wasted capacity. They reduce the effective size of the SAE and indicate a training failure. We should strive to eliminate them completely through better initialization, training dynamics, or architectural changes.</P>
            </div>
            <div className="research-card" style={{ borderTop: '2px solid #00E676', padding: '14px 16px' }}>
              <div style={{ fontSize: 13, fontWeight: 600, color: '#00E676', marginBottom: 6 }}>The optimistic view</div>
              <P style={{ marginBottom: 0, fontSize: 13 }}>Dead features are necessary overhead. In an overcomplete dictionary, not every direction will be used on every dataset. The cost of having unused directions is small compared to the benefit of capturing rare concepts when they do appear.</P>
            </div>
          </div>

          <P>The truth is likely between these extremes. Some dead features are genuinely wasted (stuck in initialization noise), while others represent real but rare concepts. Distinguishing between these cases requires evaluation on significantly more diverse text distributions.</P>
        </Section>

        <Section num="5" title="Conclusion">
          <div className="research-card-accent" style={{ borderLeftColor: '#FFB347' }}>
            <P>We have systematically compared four strategies for mitigating dead features in sparse autoencoders. Ghost gradients achieve the lowest dead feature rate (5.5%), followed by neuron resampling (~7%). However, no strategy we tested eliminates dead features entirely, and a fundamental tension exists between L1 sparsity and feature utilization.</P>
            <P style={{ marginBottom: 0 }}>Rather than viewing dead features as a failure, we suggest they may be a necessary cost of overcomplete representation — the price of having a dictionary large enough to capture rare but meaningful concepts. Future work should evaluate SAE features on broader text distributions to distinguish genuinely dead features from merely dormant ones.</P>
          </div>
        </Section>

        <RefList refs={[
          { authors: 'Bricken, T., Templeton, A., et al.', year: '2023', title: 'Towards Monosemanticity', venue: 'Anthropic', url: 'https://transformer-circuits.pub/2023/monosemantic-features' },
          { authors: 'Rajamanoharan, S., Conmy, A., et al.', year: '2024', title: 'Improving Dictionary Learning with Gated Sparse Autoencoders', venue: 'arXiv:2404.16014', url: 'https://arxiv.org/abs/2404.16014' },
          { authors: 'Elhage, N., Hume, T., et al.', year: '2022', title: 'Toy Models of Superposition', venue: 'Transformer Circuits Thread', url: 'https://transformer-circuits.pub/2022/toy_model' },
        ]} />

        <BackCTA />
      </article>
    </BlogLayout>
  );
};