Revert reasoning default to deepseek — Qwen3-4B-Thinking timed out live-testing at 90s

Add newer/smaller default models per role: Qwen3-4B-Thinking (reasoning), Dolphin3.0-3B (uncensored)

Add Wizard-Vicuna 7B uncensored model as a second uncensored option

Swap Dolphin 8B for Dolphin 7B (8B was too slow for free CPU tier, regularly hit the 90s self-heal timeout)

Fix: Qwen general-role model repo (official repo ships only sharded quants, was falling back to a broken FP16 shard)

Fix: queue concurrent requests instead of instant 429 rejection

Add role-based auto model assignment (SPACE_ROLE=general) + /jolly/refresh-model

Add 90s generation timeout that kills+restarts the worker instead of holding the busy-lock forever on an abandoned/oversized request

Reduce max_tokens cap to 80 (~15s) — 180 tokens was still hitting a client-side timeout

Fail fast with 429 when already generating, instead of silently queuing behind it

Run inference in a dedicated child process; tighten max_tokens cap to 180 for tolerable response times without streaming

Cap max_tokens at 400 — long CPU-only generations block the entire process, not just the request

Rewrite: load model directly with Llama class, drop buggy llama_cpp.server subprocess/proxy entirely

Fix empty-body proxy responses: reuse a long-lived httpx client instead of closing it before streaming finishes

Strip hop-by-hop headers in proxy forward — fixes empty 400s on all /v1/* routes

Fix health check: llama_cpp.server has no /health route, poll /v1/models instead

Pin llama-cpp-python to fix build timeout (Job timeout / BUILD_ERROR)

Update app.py

Update app.py

Update app.py

Create app.py