jpanasuk/tabby-tavern-stack

README.md Markdown--- license: mit language: - en tags: - exl3 - tabbyapi - sillytavern - open-webui - docker - local-ai

Tabby-Tavern AI Stack

Complete containerized local AI infrastructure stack with TabbyAPI (EXL3), SillyTavern, Open WebUI, Ollama, and SearXNG.

---

System Requirements

• OS: Linux (Ubuntu/Debian recommended) or Windows via WSL2
• GPU: NVIDIA GPU with decent VRAM (RTX 4070 or better recommended)
• Software:
  • Docker + Docker Compose
  • NVIDIA Container Toolkit (for GPU passthrough)
  • Python 3.10+
  • huggingface_hub (pip install huggingface_hub)

---

Service Ports

Service	URL	Purpose
SillyTavern	http://localhost:8000	Chat / Roleplay frontend
Open WebUI	http://localhost:3000	LLM workspace
TabbyAPI	http://localhost:5000	EXL3 / ExLlamaV3 inference
Ollama	http://localhost:11434	GGUF backend
SearXNG	http://localhost:8080	Private metasearch

---

Quick Start

1. Clone the repo

git clone https://huggingface.co/jpanasuk/tabby-tavern-stack tabby-tavern
cd tabby-tavern
2. Download an EXL3 model
Bashcd tabby_models
hf download turboderp/Llama-3.1-8B-Instruct-exl3 --revision 6.0bpw --local-dir Llama-3.1-8B-Instruct-6.0bpw-exl3
cd ..
3. Launch the stack
Bashdocker compose down
docker compose up -d
4. Check status
Bashdocker compose ps
docker compose logs -f tabbyapi

Repository Structure
text├── docker-compose.yml      # Master orchestration file
├── tabby_models/           # EXL3 model weights
├── config/                 # Shared configs
├── tabby_config/           # TabbyAPI specific configs
└── DEVLOG.md               # Engineering notes & history

Notes

Default TabbyAPI admin credentials are set to admin / admin so the other containers can talk to it without whitelist issues.
Make sure your NVIDIA Container Toolkit is working before launching the stack.
See DEVLOG.md for the full engineering history and troubleshooting notes.


License: MIT