Spaces:
Sleeping
Sleeping
| title: Voting Bot Llm | |
| emoji: 🗳️ | |
| colorFrom: yellow | |
| colorTo: green | |
| sdk: docker | |
| pinned: false | |
| app_port: 7860 | |
| # Voting Bot LLM Backend | |
| Self-hosted Qwen2.5-7B-Instruct (AWQ) served via vLLM, exposing an | |
| OpenAI-compatible API for tool-calling. | |
| This is a private backend service for a WhatsApp bot — not meant for | |
| interactive/public use. | |
| ## Endpoint | |
| Compatible with the OpenAI SDK — just point `baseURL` at this Space's URL | |
| with `/v1` appended. | |
| ## Model | |
| - **Model**: `Qwen/Qwen2.5-7B-Instruct-AWQ` | |
| - **Server**: vLLM (`vllm/vllm-openai:latest`) | |
| - **Tool calling**: enabled via `--enable-auto-tool-choice --tool-call-parser hermes` | |
| ## Notes | |
| - First boot downloads model weights from HF Hub — expect a few minutes cold start. | |
| - If Space sleep is enabled, subsequent wakes will re-download unless weights are baked into the image. | |