Update main.py

Update main.py

Update Dockerfile

Update main.py

Update main.py

Update main.py

Update main.py

Update Dockerfile

Update Dockerfile

Update Dockerfile

Update main.py

Revert to Llama-3.2-1B Q8_0 (proven 10-24s responses) — fastest working config"

Switch to SmolLM2-360M Q8_0 (386MB = 3.4x smaller = 3x faster inference)"

Use python:3.10-slim + compile llama-cpp 0.2.90 (proven working version)"

Revert main.py to exact original working code + n_batch=512 speedup"

Full revert to original working configuration"

main.py: use Q4_K_M path, optimized for OpenBLAS build"

Use python:3.11-slim + compile llama-cpp with OpenBLAS for CPU speed"

Revert main.py to working config with speed tweaks (n_batch=512, max_tokens=100, only last 2 msgs)"

Revert to working config: Q8_0 (was working before), pre-download for fast startup"

Ultra-minimal: max_tokens=40, n_ctx=256 for fastest possible inference"

Force restart

Switch to ghcr.io/abetlen/llama-cpp-python (has CPU optimizations) + Q4_K_M + pre-download

Add non-streaming /api/chat/sync endpoint + keep SSE with heartbeat"

Fix: send immediate SSE heartbeat to prevent proxy timeout"

main.py: Llama-3.2-1B Q4_K_M, optimized for speed (n_batch=512, max_tokens=100, streaming)"

Revert to Llama-3.2-1B Q4_K_M (proven working, no thinking issues) with speed optimizations"

Fix: pre-fill think block in prompt so model starts answering immediately"

Fix timeout: max_tokens=80, ensure response completes within proxy timeout

Fix: handle Qwen3 think mode — skip think tokens, emit only real content"

Fix: remove think block, use direct prompt without thinking mode"

Optimized main.py: Qwen3-0.6B, /no_think, n_batch=512, max_tokens=100

Speed overhaul: Qwen3-0.6B Q4_K_M (397MB, 3x faster), pre-built wheel, optimized config

Update Dockerfile

Update Dockerfile

Update main.py

Update main.py

Update main.py

Update main.py

Update Dockerfile

Update main.py

Update Dockerfile

Update main.py

Update main.py

Update main.py

Update main.py

Update main.py

Update main.py

Update main.py

Create main.py