README.md
Markdown---
license: mit
language:
  - en
tags:
  - exl3
  - tabbyapi
  - sillytavern
  - open-webui
  - docker
  - local-ai
---

# Tabby-Tavern AI Stack

[![Status](https://img.shields.io/badge/Stack-EXL3%20Ready-success)]()
[![Engine](https://img.shields.io/badge/TabbyAPI-ExLlamaV3-blue)]()
[![License](https://img.shields.io/badge/License-MIT-green)]()

Complete containerized local AI infrastructure stack with **TabbyAPI (EXL3)**, **SillyTavern**, **Open WebUI**, **Ollama**, and **SearXNG**.

---

## System Requirements

- **OS:** Linux (Ubuntu/Debian recommended) or Windows via WSL2
- **GPU:** NVIDIA GPU with decent VRAM (RTX 4070 or better recommended)
- **Software:**
  - Docker + Docker Compose
  - NVIDIA Container Toolkit (for GPU passthrough)
  - Python 3.10+
  - `huggingface_hub` (`pip install huggingface_hub`)

---

## Service Ports

| Service       | URL                        | Purpose                          |
|---------------|----------------------------|----------------------------------|
| SillyTavern   | http://localhost:8000      | Chat / Roleplay frontend         |
| Open WebUI    | http://localhost:3000      | LLM workspace                    |
| TabbyAPI      | http://localhost:5000      | EXL3 / ExLlamaV3 inference       |
| Ollama        | http://localhost:11434     | GGUF backend                     |
| SearXNG       | http://localhost:8080      | Private metasearch               |

---

## Quick Start

### 1. Clone the repo
```bash
git clone https://huggingface.co/jpanasuk/tabby-tavern-stack tabby-tavern
cd tabby-tavern
2. Download an EXL3 model
Bashcd tabby_models
hf download turboderp/Llama-3.1-8B-Instruct-exl3 --revision 6.0bpw --local-dir Llama-3.1-8B-Instruct-6.0bpw-exl3
cd ..
3. Launch the stack
Bashdocker compose down
docker compose up -d
4. Check status
Bashdocker compose ps
docker compose logs -f tabbyapi

Repository Structure
text├── docker-compose.yml      # Master orchestration file
├── tabby_models/           # EXL3 model weights
├── config/                 # Shared configs
├── tabby_config/           # TabbyAPI specific configs
└── DEVLOG.md               # Engineering notes & history

Notes

Default TabbyAPI admin credentials are set to admin / admin so the other containers can talk to it without whitelist issues.
Make sure your NVIDIA Container Toolkit is working before launching the stack.
See DEVLOG.md for the full engineering history and troubleshooting notes.


License: MIT