Spaces:
Sleeping
Sleeping
File size: 849 Bytes
4bfad11 7f28959 4bfad11 7f28959 4bfad11 7f28959 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | ---
title: Voting Bot Llm
emoji: 🗳️
colorFrom: yellow
colorTo: green
sdk: docker
pinned: false
app_port: 7860
---
# Voting Bot LLM Backend
Self-hosted Qwen2.5-7B-Instruct (AWQ) served via vLLM, exposing an
OpenAI-compatible API for tool-calling.
This is a private backend service for a WhatsApp bot — not meant for
interactive/public use.
## Endpoint
Compatible with the OpenAI SDK — just point `baseURL` at this Space's URL
with `/v1` appended.
## Model
- **Model**: `Qwen/Qwen2.5-7B-Instruct-AWQ`
- **Server**: vLLM (`vllm/vllm-openai:latest`)
- **Tool calling**: enabled via `--enable-auto-tool-choice --tool-call-parser hermes`
## Notes
- First boot downloads model weights from HF Hub — expect a few minutes cold start.
- If Space sleep is enabled, subsequent wakes will re-download unless weights are baked into the image.
|