File size: 3,178 Bytes
6b2aec2 717dc83 6b2aec2 717dc83 6b2aec2 717dc83 6b2aec2 717dc83 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 | ---
title: GPU SAE Service
emoji: 🧠
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 4.44.0
app_file: app.py
pinned: false
license: mit
hardware: t4-small
---
# GPU-Accelerated SAE Analysis Service
This HuggingFace Space provides GPU-accelerated Sparse Autoencoder (SAE) analysis for larger language models that are too slow to run on CPU.
## Supported Models
| Model | Parameters | Layers | SAE Features | Status |
|-------|------------|--------|--------------|--------|
| Gemma 2 9B | 9B | 42 | 16k | ✅ Ready |
| Mistral 7B | 7B | 32 | 65k | ⚠️ SAE pending |
| Pythia 1.4B | 1.4B | 24 | 32k | ⚠️ SAE pending |
| Llama 2 7B | 7B | 32 | - | ❌ No SAE |
## Usage
### As a Remote Backend
Set the `EXTERNAL_SAE_URL` environment variable in your Replit app to point to this Space:
```
EXTERNAL_SAE_URL=https://your-username-gpu-sae-service.hf.space
```
The Replit app will automatically route requests for larger models to this GPU backend.
### API Endpoints
- **POST /api/analyze** - Run SAE feature analysis
- **POST /api/discover** - Circuit discovery (input/output features)
- **POST /api/ablate** - Ablation testing (zero features, measure impact)
- **GET /api/models** - List available models
- **GET /api/health** - Check service status
### Example Request
```python
import requests
response = requests.post(
"https://your-space.hf.space/api/analyze",
json={
"prompt": "The capital of France is",
"model_id": "gemma-2-9b",
"layer": 20,
"top_k": 32
}
)
print(response.json())
```
## Hardware Requirements
- **T4 (16GB VRAM)**: Runs 7B models with float16
- **A10G (24GB VRAM)**: Runs 9B-13B models comfortably
- **A100 (40/80GB VRAM)**: Required for 70B+ models
## Integration Architecture
```
┌─────────────────────────────────────────┐
│ Replit App (Frontend + API) │
│ - UI for interpretability tools │
│ - Routes to local or remote backend │
└─────────────────┬───────────────────────┘
│
┌─────────────┴─────────────┐
▼ ▼
┌───────────────┐ ┌─────────────────────┐
│ Local Service │ │ This HF Space (GPU) │
│ GPT-2, Pythia │ │ Gemma 9B, Mistral │
│ Fast, Free │ │ GPU-accelerated │
└───────────────┘ └─────────────────────┘
```
## Deploying Your Own
1. Fork this Space
2. Select GPU hardware (T4-small recommended for cost)
3. Add HuggingFace token if using gated models (Llama, Gemma)
4. Copy the Space URL to your Replit app's `EXTERNAL_SAE_URL`
## Cost Estimates
- **T4 small**: ~$0.60/hour (pauses when idle)
- **A10G small**: ~$1.00/hour
- **A100 large**: ~$4.00/hour
Spaces with GPU auto-sleep after inactivity, so you only pay for active usage.
|