README.md

---
title: GPU SAE Service
emoji: 🧠
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 4.44.0
app_file: app.py
pinned: false
license: mit
hardware: t4-small
---

# GPU-Accelerated SAE Analysis Service

This HuggingFace Space provides GPU-accelerated Sparse Autoencoder (SAE) analysis for larger language models that are too slow to run on CPU.

## Supported Models

| Model | Parameters | Layers | SAE Features | Status |
|-------|------------|--------|--------------|--------|
| Gemma 2 9B | 9B | 42 | 16k | ✅ Ready |
| Mistral 7B | 7B | 32 | 65k | ⚠️ SAE pending |
| Pythia 1.4B | 1.4B | 24 | 32k | ⚠️ SAE pending |
| Llama 2 7B | 7B | 32 | - | ❌ No SAE |

## Usage

### As a Remote Backend

Set the `EXTERNAL_SAE_URL` environment variable in your Replit app to point to this Space:

```
EXTERNAL_SAE_URL=https://your-username-gpu-sae-service.hf.space
```

The Replit app will automatically route requests for larger models to this GPU backend.

### API Endpoints

- **POST /api/analyze** - Run SAE feature analysis
- **POST /api/discover** - Circuit discovery (input/output features)
- **POST /api/ablate** - Ablation testing (zero features, measure impact)
- **GET /api/models** - List available models
- **GET /api/health** - Check service status

### Example Request

```python
import requests

response = requests.post(
    "https://your-space.hf.space/api/analyze",
    json={
        "prompt": "The capital of France is",
        "model_id": "gemma-2-9b",
        "layer": 20,
        "top_k": 32
    }
)
print(response.json())
```

## Hardware Requirements

- **T4 (16GB VRAM)**: Runs 7B models with float16
- **A10G (24GB VRAM)**: Runs 9B-13B models comfortably
- **A100 (40/80GB VRAM)**: Required for 70B+ models

## Integration Architecture

```
┌─────────────────────────────────────────┐
│  Replit App (Frontend + API)            │
│  - UI for interpretability tools        │
│  - Routes to local or remote backend    │
└─────────────────┬───────────────────────┘
                  │
    ┌─────────────┴─────────────┐
    ▼                           ▼
┌───────────────┐    ┌─────────────────────┐
│ Local Service │    │ This HF Space (GPU) │
│ GPT-2, Pythia │    │ Gemma 9B, Mistral   │
│ Fast, Free    │    │ GPU-accelerated     │
└───────────────┘    └─────────────────────┘
```

## Deploying Your Own

1. Fork this Space
2. Select GPU hardware (T4-small recommended for cost)
3. Add HuggingFace token if using gated models (Llama, Gemma)
4. Copy the Space URL to your Replit app's `EXTERNAL_SAE_URL`

## Cost Estimates

- **T4 small**: ~$0.60/hour (pauses when idle)
- **A10G small**: ~$1.00/hour
- **A100 large**: ~$4.00/hour

Spaces with GPU auto-sleep after inactivity, so you only pay for active usage.