Spaces:
Running on Zero
Running on Zero
File size: 2,124 Bytes
434c049 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | # OmniRoute Provider Runbook
The Space is an OpenAI-compatible upstream:
```text
Base URL: https://abalanescu-flow.hf.space/v1
Models: GET /v1/models
Chat: POST /v1/chat/completions
Auth: Authorization: Bearer <FLOW_API_KEY>
```
## Discover the exact model ID
```bash
curl -sS "$FLOW_BASE/v1/models" \
-H "Authorization: Bearer $FLOW_API_KEY" | jq -r '.data[].id'
```
Use the exact returned ID. The verified models are:
```text
Qwen3.8-27B-Q6_K.gguf
gemma-4-26B-A4B-it-ultra-uncensored-heretic.i1-Q4_K_M.gguf
gemma-4-26B-A4B-it-ultra-uncensored-heretic.i1-Q6_K.gguf
```
## Smoke test
### Test Qwen3.8-27B Q6
```bash
curl -sS --max-time 180 "$FLOW_BASE/v1/chat/completions" \
-H "Authorization: Bearer $FLOW_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "Qwen3.8-27B-Q6_K.gguf",
"messages": [{"role":"user","content":"Tell me a joke."}],
"temperature": 0.7,
"max_tokens": 64
}'
```
### Test Gemma 4 26B Heretic
```bash
curl -sS --max-time 180 "$FLOW_BASE/v1/chat/completions" \
-H "Authorization: Bearer $FLOW_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gemma-4-26B-A4B-it-ultra-uncensored-heretic.i1-Q4_K_M.gguf",
"messages": [{"role":"user","content":"Reply exactly OMNIROUTE_FLOW_OK"}],
"temperature": 0.1,
"max_tokens": 16
}'
```
## OmniRoute configuration checklist
Add a provider named `hf-flow-zerogpu` in the OmniRoute provider configuration. The exact UI/API differs by the deployed OmniRoute version, so do not invent a config endpoint. Set:
- Base URL: `https://abalanescu-flow.hf.space/v1`
- API key: reference a homelab secret, never a literal token in git
- Model: exact ID from `/v1/models`
- Timeout: 180 seconds for cold starts and model loading
- Concurrency: 1 initially, because one cached `llama.cpp` model is shared
- Retries: transport/5xx only; do not replay a completed GPU request
- Health check: `/v1/models` with the bearer token
After adding it, run the smoke test through OmniRoute and record the request ID, latency, status, and selected upstream. Only then enable it in a combo.
|