---
title: Voting Bot Llm
emoji: 🗳️
colorFrom: yellow
colorTo: green
sdk: docker
pinned: false
app_port: 7860
---

# Voting Bot LLM Backend

Self-hosted Qwen2.5-7B-Instruct (AWQ) served via vLLM, exposing an
OpenAI-compatible API for tool-calling.

This is a private backend service for a WhatsApp bot — not meant for
interactive/public use.

## Endpoint

Compatible with the OpenAI SDK — just point `baseURL` at this Space's URL
with `/v1` appended.

## Model

- **Model**: `Qwen/Qwen2.5-7B-Instruct-AWQ`
- **Server**: vLLM (`vllm/vllm-openai:latest`)
- **Tool calling**: enabled via `--enable-auto-tool-choice --tool-call-parser hermes`

## Notes

- First boot downloads model weights from HF Hub — expect a few minutes cold start.
- If Space sleep is enabled, subsequent wakes will re-download unless weights are baked into the image.