README.md

---
title: Transformer Explanation Dashboard
emoji: 🧠
colorFrom: blue
colorTo: purple
sdk: docker
python_version: "3.10"
app_file: app.py
pinned: false
---

# Transformer Explanation Dashboard

A Dash-based interactive application for visualizing and analyzing the internal mechanics of Transformer-based Large Language Models (LLMs). This tool enables users to inspect the generation pipeline step-by-step and perform real-time experiments like ablation and attribution.

## Architecture

The project is structured around a central Dash application with modular components and utility libraries:

### Core Components
*   `app.py`: The main application entry point that orchestrates the layout and callbacks.
*   `components/`: Modular UI elements.
    *   `pipeline.py`: Implements the 5-stage visualization pipeline (Tokenization, Embedding, Attention, MLP, Output).
    *   `investigation_panel.py`: Handles the experimental interfaces (Ablation and Attribution).
    *   `ablation_panel.py`: Logic for the head ablation interface.
    *   `sidebar.py` & `model_selector.py`: Configuration and navigation controls.

### Utilities (`utils/`)
*   `model_patterns.py`: Core logic for hooking into PyTorch models to capture activations.
*   `model_config.py`: Registry for automatic detection of model families (LLaMA, GPT-2, OPT, etc.).
*   `head_detection.py`: Analysis logic for categorizing attention heads.
*   `beam_search.py`: Implementation of beam search for sequence generation analysis.
*   `token_attribution.py`: Integrated Gradients implementation for feature importance.

## Installation

### Prerequisites
*   Python 3.11+
*   PyTorch

### Steps

1.  Clone the repository:
    ```bash
    git clone <repository_url>
    cd <repository_directory>
    ```

2.  Install the required dependencies:
    ```bash
    pip install -r requirements.txt
    ```

## Usage

1.  **Launch the Dashboard**:
    ```bash
    python app.py
    ```

2.  **Access the Interface**:
    Open a web browser and navigate to `http://127.0.0.1:8050/`.

3.  **Workflow**:
    *   **Model Selection**: Choose a model from the dropdown or enter a HuggingFace model ID. The system automatically detects the architecture.
    *   **Analysis**: Enter a prompt and click "Analyze" to visualize the forward pass.
    *   **Pipeline Exploration**: Interact with the 5 pipeline stages to view detailed activation data.
    *   **Experiments**: Use the Investigation Panel at the bottom to run Ablation (disable heads) or Attribution (analyze token importance) experiments.

## Project Structure

*   `app.py`: Main application entry point and layout orchestration.
*   `components/`: Modular UI components.
    *   `pipeline.py`: The core 5-stage visualization.
    *   `investigation_panel.py`: Ablation and attribution interfaces.
    *   `ablation_panel.py`: Specific logic for head ablation UI.
*   `utils/`: Backend logic and helper functions.
    *   `model_patterns.py`: Activation capture and hooking logic.
    *   `model_config.py`: Model family definitions and auto-detection.
    *   `head_detection.py`: Attention head categorization logic.
    *   `beam_search.py`: Beam search implementation.
*   `tests/`: Comprehensive test suite ensuring stability.
*   `.context/`: Project memory — modules (architecture, conventions, education, product, testing) and data files (sessions, decisions, lessons).