README.md

metadata

title: Transformer Explanation Dashboard
emoji: 🧠
colorFrom: blue
colorTo: purple
sdk: docker
python_version: '3.10'
app_file: app.py
pinned: false

Transformer Explanation Dashboard

A Dash-based interactive application for visualizing and analyzing the internal mechanics of Transformer-based Large Language Models (LLMs). This tool enables users to inspect the generation pipeline step-by-step and perform real-time experiments like ablation and attribution.

Architecture

The project is structured around a central Dash application with modular components and utility libraries:

Core Components

• app.py: The main application entry point that orchestrates the layout and callbacks.
• components/: Modular UI elements.
  • pipeline.py: Implements the 5-stage visualization pipeline (Tokenization, Embedding, Attention, MLP, Output).
  • investigation_panel.py: Handles the experimental interfaces (Ablation and Attribution).
  • ablation_panel.py: Logic for the head ablation interface.
  • sidebar.py & model_selector.py: Configuration and navigation controls.

Utilities (utils/)

• model_patterns.py: Core logic for hooking into PyTorch models to capture activations.
• model_config.py: Registry for automatic detection of model families (LLaMA, GPT-2, OPT, etc.).
• head_detection.py: Analysis logic for categorizing attention heads.
• beam_search.py: Implementation of beam search for sequence generation analysis.
• token_attribution.py: Integrated Gradients implementation for feature importance.

Installation

Prerequisites

• Python 3.11+
• PyTorch

Steps

1. Clone the repository:

   git clone <repository_url>
   cd <repository_directory>
   

2. Install the required dependencies:

   pip install -r requirements.txt
   

Usage

1. Launch the Dashboard:

   python app.py
   

2. Access the Interface: Open a web browser and navigate to http://127.0.0.1:8050/.

3. Workflow:

   • Model Selection: Choose a model from the dropdown or enter a HuggingFace model ID. The system automatically detects the architecture.
   • Analysis: Enter a prompt and click "Analyze" to visualize the forward pass.
   • Pipeline Exploration: Interact with the 5 pipeline stages to view detailed activation data.
   • Experiments: Use the Investigation Panel at the bottom to run Ablation (disable heads) or Attribution (analyze token importance) experiments.

Project Structure

• app.py: Main application entry point and layout orchestration.
• components/: Modular UI components.
  • pipeline.py: The core 5-stage visualization.
  • investigation_panel.py: Ablation and attribution interfaces.
  • ablation_panel.py: Specific logic for head ablation UI.
• utils/: Backend logic and helper functions.
  • model_patterns.py: Activation capture and hooking logic.
  • model_config.py: Model family definitions and auto-detection.
  • head_detection.py: Attention head categorization logic.
  • beam_search.py: Beam search implementation.
• tests/: Comprehensive test suite ensuring stability.
• .context/: Project memory — modules (architecture, conventions, education, product, testing) and data files (sessions, decisions, lessons).