README.md

metadata

title: Qwopus3.5-0.8B (serve)
emoji: ⚡
colorFrom: indigo
colorTo: green
sdk: docker
app_port: 7860
pinned: false

Qwopus3.5-0.8B — GGUF (live serve)

A Qwen3.5-0.8B model served as a pure model: this Space is a thin pass-through proxy to llama-server, which applies the model's own embedded chat template (Qwen3.5 ChatML).

On startup it:

1. Downloads Qwen3.5-0.8B.Q4_K_M.gguf from anon334test/qwopus

2. Serves it with prebuilt llama.cpp llama-server (release b9664, no compilation) behind an OpenAI-compatible FastAPI proxy + a multi-turn chat UI with a collapsible "Thinking" panel.

• Tiny + fast: 0.8B on CPU-basic is quick; great for low-latency chat.
• Chain-of-thought: <think> shown in the collapsible panel; the answer stays clean.
• Context: 32,768 tokens by default (model native up to 262,144 — raise N_CTX, slower on CPU).
• Output length: unlimited by default (MAX_TOKENS=-1) — generates until done or context is full.

Tunable via Space variables (no rebuild): MODEL_REPO, GGUF_FILE (e.g. ...Q5_K_M.gguf, ...Q8_0.gguf), N_CTX, MAX_TOKENS, NUM_THREADS, SYSTEM_PROMPT, LLAMA_EXTRA_ARGS.