| README.md |
| Markdown--- |
| license: mit |
| language: |
| - en |
| tags: |
| - exl3 |
| - tabbyapi |
| - sillytavern |
| - open-webui |
| - docker |
| - local-ai |
| --- |
|
|
| # Tabby-Tavern AI Stack |
|
|
| []() |
| []() |
| []() |
|
|
| Complete containerized local AI infrastructure stack with **TabbyAPI (EXL3)**, **SillyTavern**, **Open WebUI**, **Ollama**, and **SearXNG**. |
|
|
| --- |
|
|
| ## System Requirements |
|
|
| - **OS:** Linux (Ubuntu/Debian recommended) or Windows via WSL2 |
| - **GPU:** NVIDIA GPU with decent VRAM (RTX 4070 or better recommended) |
| - **Software:** |
| - Docker + Docker Compose |
| - NVIDIA Container Toolkit (for GPU passthrough) |
| - Python 3.10+ |
| - `huggingface_hub` (`pip install huggingface_hub`) |
|
|
| --- |
|
|
| ## Service Ports |
|
|
| | Service | URL | Purpose | |
| |---------------|----------------------------|----------------------------------| |
| | SillyTavern | http://localhost:8000 | Chat / Roleplay frontend | |
| | Open WebUI | http://localhost:3000 | LLM workspace | |
| | TabbyAPI | http://localhost:5000 | EXL3 / ExLlamaV3 inference | |
| | Ollama | http://localhost:11434 | GGUF backend | |
| | SearXNG | http://localhost:8080 | Private metasearch | |
|
|
| --- |
|
|
| ## Quick Start |
|
|
| ### 1. Clone the repo |
| ```bash |
| git clone https://huggingface.co/jpanasuk/tabby-tavern-stack tabby-tavern |
| cd tabby-tavern |
| 2. Download an EXL3 model |
| Bashcd tabby_models |
| hf download turboderp/Llama-3.1-8B-Instruct-exl3 --revision 6.0bpw --local-dir Llama-3.1-8B-Instruct-6.0bpw-exl3 |
| cd .. |
| 3. Launch the stack |
| Bashdocker compose down |
| docker compose up -d |
| 4. Check status |
| Bashdocker compose ps |
| docker compose logs -f tabbyapi |
| |
| Repository Structure |
| text├── docker-compose.yml # Master orchestration file |
| ├── tabby_models/ # EXL3 model weights |
| ├── config/ # Shared configs |
| ├── tabby_config/ # TabbyAPI specific configs |
| └── DEVLOG.md # Engineering notes & history |
| |
| Notes |
| |
| Default TabbyAPI admin credentials are set to admin / admin so the other containers can talk to it without whitelist issues. |
| Make sure your NVIDIA Container Toolkit is working before launching the stack. |
| See DEVLOG.md for the full engineering history and troubleshooting notes. |
| |
| |
| License: MIT |
| |
