Update main.py

Update main.py

Update main.py

Update main.py

Update main.py

Update main.py

Update main.py

Revert to Llama-3.2-1B Q8_0 (proven 10-24s responses) — fastest working config"

Switch to SmolLM2-360M Q8_0 (386MB = 3.4x smaller = 3x faster inference)"

Revert main.py to exact original working code + n_batch=512 speedup"

main.py: use Q4_K_M path, optimized for OpenBLAS build"

Revert main.py to working config with speed tweaks (n_batch=512, max_tokens=100, only last 2 msgs)"

Ultra-minimal: max_tokens=40, n_ctx=256 for fastest possible inference"

Add non-streaming /api/chat/sync endpoint + keep SSE with heartbeat"

Fix: send immediate SSE heartbeat to prevent proxy timeout"

main.py: Llama-3.2-1B Q4_K_M, optimized for speed (n_batch=512, max_tokens=100, streaming)"

Fix: pre-fill think block in prompt so model starts answering immediately"

Fix timeout: max_tokens=80, ensure response completes within proxy timeout

Fix: handle Qwen3 think mode — skip think tokens, emit only real content"

Fix: remove think block, use direct prompt without thinking mode"

Optimized main.py: Qwen3-0.6B, /no_think, n_batch=512, max_tokens=100

Update main.py

Update main.py

Update main.py

Update main.py

Update main.py

Update main.py

Update main.py

Update main.py

Update main.py

Update main.py

Update main.py

Update main.py

Create main.py

Delete main.py

Update main.py

Create main.py