docs/OMNIROUTE_PROVIDER.md

# OmniRoute Provider Runbook

The Space is an OpenAI-compatible upstream:

```text
Base URL: https://abalanescu-flow.hf.space/v1
Models:   GET /v1/models
Chat:     POST /v1/chat/completions
Auth:     Authorization: Bearer <FLOW_API_KEY>
```

## Discover the exact model ID

```bash
curl -sS "$FLOW_BASE/v1/models" \
  -H "Authorization: Bearer $FLOW_API_KEY" | jq -r '.data[].id'
```

Use the exact returned ID. The verified models are:

```text
Qwen3.8-27B-Q6_K.gguf
gemma-4-26B-A4B-it-ultra-uncensored-heretic.i1-Q4_K_M.gguf
gemma-4-26B-A4B-it-ultra-uncensored-heretic.i1-Q6_K.gguf
```

## Smoke test

### Test Qwen3.8-27B Q6

```bash
curl -sS --max-time 180 "$FLOW_BASE/v1/chat/completions" \
  -H "Authorization: Bearer $FLOW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen3.8-27B-Q6_K.gguf",
    "messages": [{"role":"user","content":"Tell me a joke."}],
    "temperature": 0.7,
    "max_tokens": 64
  }'
```

### Test Gemma 4 26B Heretic

```bash
curl -sS --max-time 180 "$FLOW_BASE/v1/chat/completions" \
  -H "Authorization: Bearer $FLOW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma-4-26B-A4B-it-ultra-uncensored-heretic.i1-Q4_K_M.gguf",
    "messages": [{"role":"user","content":"Reply exactly OMNIROUTE_FLOW_OK"}],
    "temperature": 0.1,
    "max_tokens": 16
  }'
```

## OmniRoute configuration checklist

Add a provider named `hf-flow-zerogpu` in the OmniRoute provider configuration. The exact UI/API differs by the deployed OmniRoute version, so do not invent a config endpoint. Set:

- Base URL: `https://abalanescu-flow.hf.space/v1`
- API key: reference a homelab secret, never a literal token in git
- Model: exact ID from `/v1/models`
- Timeout: 180 seconds for cold starts and model loading
- Concurrency: 1 initially, because one cached `llama.cpp` model is shared
- Retries: transport/5xx only; do not replay a completed GPU request
- Health check: `/v1/models` with the bearer token

After adding it, run the smoke test through OmniRoute and record the request ID, latency, status, and selected upstream. Only then enable it in a combo.