klaus-3 GPU services

Deploy these to ~/workspace/veil-pgd on klaus-3. They provide every model the Mac orchestrator needs, behind HTTP, so the Mac stays torch-free.

Ports

8081 qwen-3.5-4b (llama.cpp, OpenAI-compatible /v1)
8082 gemma-4-4b (llama.cpp, OpenAI-compatible /v1)
8090 vision service (FastAPI: CLIP + all-MiniLM embeddings + LPIPS)

Setup

cd ~/workspace/veil-pgd
python3 -m venv .venv && . .venv/bin/activate
# install torch matching the box CUDA first, then:
pip install -r services/requirements-klaus3.txt

VRAM (8GB laptop 3070)

Kill the old gemma-4-12b llama-server first to free VRAM. Then two Q4_K_M 4B VLMs (~~2.6GB each) + CLIP (~~1GB) coexist (~6GB). Do not also load the 12B.

Run

# 1. surrogate VLMs
bash services/serve_vlms.sh

# 2. vision service (CLIP + embeddings + LPIPS)
. .venv/bin/activate
uvicorn services.vision_service:app --host 0.0.0.0 --port 8090

Verify

curl localhost:8081/v1/models
curl localhost:8082/v1/models
curl localhost:8090/health