README.md

---
title: Qwopus3.5-0.8B (serve)
emoji: ⚡
colorFrom: indigo
colorTo: green
sdk: docker
app_port: 7860
pinned: false
---

# Qwopus3.5-0.8B — GGUF (live serve)

A Qwen3.5-**0.8B** model served as a **pure model**: this Space is a thin pass-through proxy to
`llama-server`, which applies the model's **own embedded chat template** (Qwen3.5 ChatML).

On startup it:
1. Downloads `Qwen3.5-0.8B.Q4_K_M.gguf` from
   [`anon334test/qwopus`](https://huggingface.co/anon334test/qwopus)

2. Serves it with prebuilt llama.cpp `llama-server` (release `b9664`, no compilation) behind an
   OpenAI-compatible FastAPI proxy + a multi-turn chat UI with a collapsible "Thinking" panel.

- **Tiny + fast:** 0.8B on CPU-basic is quick; great for low-latency chat.
- **Chain-of-thought:** `<think>` shown in the collapsible panel; the answer stays clean.
- **Context:** 32,768 tokens by default (model native up to **262,144** — raise `N_CTX`, slower on CPU).
- **Output length:** unlimited by default (`MAX_TOKENS=-1`) — generates until done or context is full.

Tunable via Space variables (no rebuild): `MODEL_REPO`, `GGUF_FILE` (e.g. `...Q5_K_M.gguf`,
`...Q8_0.gguf`), `N_CTX`, `MAX_TOKENS`, `NUM_THREADS`, `SYSTEM_PROMPT`, `LLAMA_EXTRA_ARGS`.