rag_docs/model_selector_guide.md

# Model Selector Guide

## How to Choose a Model

The dashboard supports seven transformer models from five architecture families. Select a model from the dropdown menu in the generator section at the top of the page.

### Available Models

| Model | Family | Params | Layers × Heads | Key Feature |
|-------|--------|--------|----------------|-------------|
| **GPT-2 (124M)** | GPT-2 | 85M | 12 × 12 | The MI classic — start here |
| **GPT-2 Medium (355M)** | GPT-2 | 302M | 24 × 16 | Scale comparison within GPT-2 |
| **GPT-Neo 125M** | GPT-Neo | 85M | 12 × 12 | Local attention in alternating layers |
| **Pythia-160M** | Pythia | 85M | 12 × 12 | Rotary PE, parallel attn+MLP |
| **Pythia-410M** | Pythia | 302M | 24 × 16 | Larger Pythia for scale comparison |
| **OPT-125M** | OPT | 85M | 12 × 12 | ReLU activation (unique contrast) |
| **Qwen2.5-0.5B (494M)** | Qwen2 | 391M | 24 × 14 | Modern: RMSNorm, SiLU, rotary PE |

### Architecture Comparisons

These models were chosen to highlight specific architectural differences:

- **Positional encoding**: GPT-2, GPT-Neo, and OPT use absolute positions. Pythia and Qwen use rotary (RoPE). Comparing the same prompt across both types shows how PE affects attention patterns.
- **Activation function**: Most models use GELU, but OPT uses ReLU and Qwen uses SiLU. This affects MLP behavior.
- **Normalization**: GPT-2, Neo, Pythia, and OPT use LayerNorm. Qwen uses RMSNorm.
- **Attention scope**: GPT-Neo alternates between local (256-token window) and global attention, unlike all other models.

### What Happens When You Load a Model

1. The model is downloaded from HuggingFace (this may take a moment the first time)
2. The dashboard **auto-detects** the model's architecture family
3. Internal hooks are automatically configured to capture attention patterns, MLP activations, and other data
4. The layer and head dropdowns are populated based on the model's structure

### Tips for Choosing

- **Start with GPT-2**: It's small, fast, and the most widely studied. Most educational resources reference GPT-2.
- **Compare same-size models**: GPT-2, GPT-Neo 125M, Pythia-160M, and OPT-125M all have 12 layers and 12 heads — differences in their attention patterns come from architecture, not scale.
- **Compare scale**: GPT-2 vs GPT-2 Medium (or Pythia-160M vs Pythia-410M) shows how more layers and heads change behavior.
- **Try Qwen2.5 for a modern perspective**: It uses an entirely different design philosophy from GPT-2.
- **Memory matters**: All dropdown models are small enough for interactive exploration. Larger models can be entered manually but may be slow.

### Generation Settings

After selecting a model and entering a prompt, you can configure:
- **Number of Generation Choices (Beams)**: 1-5 beams. More beams explore more paths but take longer.
- **Number of New Tokens**: 1-20 tokens to generate. Shorter is faster.

Click **Analyze** to run the model and see results in the pipeline and generation sections.