README.md

metadata

title: Voting Bot Llm
emoji: 🗳️
colorFrom: yellow
colorTo: green
sdk: docker
pinned: false
app_port: 7860

Voting Bot LLM Backend

Self-hosted Qwen2.5-7B-Instruct (AWQ) served via vLLM, exposing an OpenAI-compatible API for tool-calling.

This is a private backend service for a WhatsApp bot — not meant for interactive/public use.

Endpoint

Compatible with the OpenAI SDK — just point baseURL at this Space's URL with /v1 appended.

Model

• Model: Qwen/Qwen2.5-7B-Instruct-AWQ
• Server: vLLM (vllm/vllm-openai:latest)
• Tool calling: enabled via --enable-auto-tool-choice --tool-call-parser hermes

Notes

• First boot downloads model weights from HF Hub — expect a few minutes cold start.
• If Space sleep is enabled, subsequent wakes will re-download unless weights are baked into the image.