SYSTEM_GUIDE.md

OpenCode Space — System Architecture, Configuration & Troubleshooting Guide

This document serves as an exhaustive reference record for the fixes, configuration standards, and architectural design implemented for the OpenCode Space application stack.

---

📌 System Overview & Services

Service	Local Endpoint	External Endpoint Path	Description
Gateway Proxy	http://127.0.0.1:8000	https://jishnupg-opencode-cli.hf.space/	FastAPI ASGI Reverse Proxy & Multi-Service Hub
OmniRoute AI Gateway	http://127.0.0.1:20128	/v1/chat/completions & /dashboard	Multi-provider AI router managing connected models
Hermes Agent	http://127.0.0.1:8642	/hermes/	Autonomous AI agent backend & Telegram Bot integration
Telegram Streamer	http://127.0.0.1:8080	/tg_stream/	Telegram file & media streaming service
Jellyfin Media Server	http://127.0.0.1:8096	/jellyfin/	Media server application

---

🔑 Master Credentials & API Keys

• OmniRoute Master API Key: sk-6646a5f2024f6318-d27ff7-f3e152c8
• Connected Providers:
  • Antigravity: 3 Models (antigravity/gemini-3.6-flash-medium, antigravity/claude-sonnet-4-6, etc.)
  • NVIDIA NIM: 102 Free Models (meta/llama-3.3-70b-instruct, etc.)

---

🛠️ Critical Issues Fixed & Solutions Applied

1. Gateway Auto Model Resolver & Fallback Engine

• Problem: When Hermes Agent, Telegram Bot, or external clients requested model "auto" or "auto/best-fast", OmniRoute's internal Next.js server attempted public free web scrapers (grok-cli, gemini-web, opencode/mimo). These public scrapers failed with HTTP 429 Rate Limits or Playwright 500 Errors, returning 503 Service Unavailable and causing "The model provider failed after retries".
• Solution:
  • Updated gateway/omniroute.py and gateway/hermes_standalone.py to intercept "auto" and "auto/*" requests.
  • Automatically fetches the live model catalog from /v1/models, filters out auto/ and omniroute/ scrapers, and extracts real connected provider models.
  • Sequentially tries active models (antigravity/gemini-3.6-flash-medium, meta/llama-3.3-70b-instruct) and returns 200 OK on the first successful choice.

---

2. Telegram Bot API Base URL & Model Alignment

• Problem: Hermes Telegram Bot was pointing to http://127.0.0.1:20128/v1 directly, bypassing the Gateway auto-resolver.
• Solution:
  • Updated HERMES_API_BASE_URL and OPENAI_API_BASE in entrypoint.sh to point to http://127.0.0.1:8000/v1 (the Gateway auto-resolver).
  • Set default HERMES_MODEL="antigravity/gemini-3.6-flash-medium" in entrypoint.sh for instant, lag-free responses.

---

3. Open WebUI SQLite WAL Lock & Disk I/O Error

• Problem: Open WebUI crashed on startup with sqlite3.OperationalError: disk I/O error when executing PRAGMA journal_mode=WAL because SQLite WAL shared memory lock files (.db-wal / .db-shm) cannot lock across persistent volume container mounts.
• Solution:
  • Added WAL lock file purging (rm -f *.db-wal *.db-shm) in entrypoint.sh.
  • Added an automated python patch in entrypoint.sh that modifies /usr/local/lib/python*/dist-packages/open_webui/internal/db.py to replace PRAGMA journal_mode=WAL with PRAGMA journal_mode=DELETE.

---

4. OmniRoute Authentication & Transparent Errors

• Problem: gateway/omniroute.py was returning a fake HTTP 200 OK {"authenticated": false} JSON body on HTTP 401/403 responses.
• Solution:
  • Set OMNIROUTE_ALLOW_UNAUTHENTICATED=true in entrypoint.sh for local internal gateway proxy calls.
  • Restricted the fake unauthenticated JSON response in gateway/omniroute.py strictly to HTML dashboard paths, allowing API routes (/v1/chat/completions) to pass through raw status codes and choices transparently.

---

🧪 Verification & Testing Commands

1. Test via Python (test.py)

import urllib.request
import json
import ssl

url = "https://jishnupg-opencode-cli.hf.space/v1/chat/completions"
headers = {
    "Authorization": "Bearer sk-6646a5f2024f6318-d27ff7-f3e152c8",
    "Content-Type": "application/json"
}

payload = {
    "model": "auto",
    "messages": [{"role": "user", "content": "Explain quantum computing in one short sentence."}]
}

ctx = ssl.create_default_context()
req = urllib.request.Request(url, data=json.dumps(payload).encode('utf-8'), headers=headers, method='POST')

with urllib.request.urlopen(req, context=ctx) as resp:
    data = json.loads(resp.read().decode('utf-8'))
    print("Status:", resp.status)
    print("Model Used:", data.get("model"))
    print("Answer:", data["choices"][0]["message"]["content"])

2. Test via cURL

curl -X POST "https://jishnupg-opencode-cli.hf.space/v1/chat/completions" \
  -H "Authorization: Bearer sk-6646a5f2024f6318-d27ff7-f3e152c8" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

---

📋 Rules for Future Development

1. Do not set HERMES_API_BASE_URL directly to port 20128. Always point to Gateway port 8000 (http://127.0.0.1:8000/v1) so the Gateway auto-resolver maps "auto" to real provider models.
2. Preserve Master Key default: sk-6646a5f2024f6318-d27ff7-f3e152c8 must remain synced across entrypoint.sh, gateway/hermes_standalone.py, gateway/omniroute.py, and gateway/hermes.py.