# klaus-3 GPU services

Deploy these to `~/workspace/veil-pgd` on klaus-3. They provide every
model the Mac orchestrator needs, behind HTTP, so the Mac stays torch-free.

## Ports
- `8081` qwen-3.5-4b (llama.cpp, OpenAI-compatible `/v1`)
- `8082` gemma-4-4b (llama.cpp, OpenAI-compatible `/v1`)
- `8090` vision service (FastAPI: CLIP + all-MiniLM embeddings + LPIPS)

## Setup

```bash
cd ~/workspace/veil-pgd
python3 -m venv .venv && . .venv/bin/activate
# install torch matching the box CUDA first, then:
pip install -r services/requirements-klaus3.txt
```

## VRAM (8GB laptop 3070)
Kill the old gemma-4-12b llama-server first to free VRAM. Then two Q4_K_M 4B
VLMs (~2.6GB each) + CLIP (~1GB) coexist (~6GB). Do not also load the 12B.

## Run

```bash
# 1. surrogate VLMs
bash services/serve_vlms.sh

# 2. vision service (CLIP + embeddings + LPIPS)
. .venv/bin/activate
uvicorn services.vision_service:app --host 0.0.0.0 --port 8090
```

## Verify
```bash
curl localhost:8081/v1/models
curl localhost:8082/v1/models
curl localhost:8090/health
```