---
title: README
emoji: 🪽
colorFrom: yellow
colorTo: gray
sdk: static
pinned: false
---

<p align="center">
  <img src="https://hal0.dev/og-default.png" alt="hal0 — local AI inference for AMD Strix Halo" width="640">
</p>

<h1 align="center">hal0 · Local AI for the Ultimate Homelab</h1>

<p align="center">
  <strong>Strix Halo native AI inference, image gen &amp; agents for homelabs.</strong>
</p>

<p align="center">
  <a href="https://hal0.dev"><img src="https://img.shields.io/badge/site-hal0.dev-feaf00?style=for-the-badge" alt="hal0.dev"></a>
  <a href="https://github.com/hal0ai/hal0"><img src="https://img.shields.io/badge/GitHub-hal0ai%2Fhal0-181717?style=for-the-badge&logo=github" alt="GitHub"></a>
  <a href="https://hal0.dev/docs/getting-started/"><img src="https://img.shields.io/badge/docs-getting%20started-444?style=for-the-badge" alt="Docs"></a>
  <img src="https://img.shields.io/badge/license-Apache--2.0-blue?style=for-the-badge" alt="Apache-2.0">
</p>

---

## Your Strix Halo box, running real `/v1/*` inference

**hal0 turns a Linux box — ideally a Ryzen AI Max+ 395 — into a private, OpenAI-compatible AI appliance.** One `/v1/*` API across every modality, with concurrent workloads the box manages for you. One command installs the lot.

```bash
curl -fsSL https://hal0.dev/install.sh | bash
```

> Not another llama-server wrapper — it's the orchestration around one. Stop running models from a chat tab; run one service for the whole local AI stack.

---

## What's in the org

We publish the models, quants, and artifacts that ship with hal0 — tuned for AMD Strix Halo (Ryzen AI Max) and the ROCm stack.

| Model | What it is |
|-------|------------|
| [**FastContext-Hal0-4B-ROCmFP4**](https://huggingface.co/Hal0ai/FastContext-Hal0-4B-ROCmFP4) | 4B fast-context chat model, quantized to **ROCmFP4** for Strix Halo iGPU inference. |

*More quants and companion models landing as hal0 ships. Watch the org to get pinged.*

---

## One `/v1/*` surface, five providers

Drop-in for any OpenAI SDK — point your client at `:8080/v1` and go. Chat, completions, embeddings, reranking, speech-to-text, text-to-speech, and image generation, all behind one API the box schedules for you.

| Provider | Backend | Workload |
|----------|---------|----------|
| **llama.cpp** | Vulkan / ROCm / CUDA | chat, embed, rerank, vision |
| **FLMv1** | AMD XDNA NPU | chat, embed |
| **FLM / Whisper v3 turbo** | XDNA NPU | speech-to-text |
| **Kokoro-82M** | CPU / Vulkan | text-to-speech (54 voices) |
| **ComfyUI v1** | ROCm | image gen (SDXL / SD 1.5 / Flux) |

**Strix Halo native. Not Strix-Halo-only** — also runs on Ryzen AI Max 385/390, NVIDIA RTX 30/40/50, AMD Radeon RX 7000, and CPU-only x86_64 fallback.

---

## The operator console

<p align="center">
  <img src="https://hal0.dev/screenshots/dashboard-overview.png" alt="hal0 dashboard — slots, throughput, and live service health" width="720">
</p>

Dark-by-default React admin UI with SSE-backed status and a live log tail — see slots, throughput, and service health at a glance.

<table>
  <tr>
    <td width="50%"><img src="https://hal0.dev/screenshots/slots-inference.png" alt="Slots view — per-slot state and the typed inference lifecycle"></td>
    <td width="50%"><img src="https://hal0.dev/screenshots/image-gen-comfyui.png" alt="ComfyUI image generation with the iGPU in exclusive image mode"></td>
  </tr>
  <tr>
    <td align="center"><sub><b>Slots</b> — per-slot state &amp; the typed inference lifecycle</sub></td>
    <td align="center"><sub><b>Image gen</b> — ComfyUI on the iGPU, inference slots paused</sub></td>
  </tr>
  <tr>
    <td width="50%"><img src="https://hal0.dev/screenshots/memory-graph.png" alt="Agent memory rendered as a navigable semantic and temporal knowledge graph"></td>
    <td width="50%"><img src="https://hal0.dev/agents/hermes.png" alt="Hermes — the bundled hal0 agent"></td>
  </tr>
  <tr>
    <td align="center"><sub><b>Memory graph</b> — semantic + temporal knowledge graph</sub></td>
    <td align="center"><sub><b>Hermes</b> — the bundled, self-bootstrapping agent</sub></td>
  </tr>
</table>

---

## Performance (Ryzen AI Max+ 395, 128 GB)

| Metric | Number |
|--------|--------|
| Primary + embed, concurrent | **258 tok/s** |
| Primary model serving | **142 tok/s** |
| Dispatch latency (p50) | **174 ms** |

---

## Meet Hermes

**Hermes installs and bootstraps itself on first run** — sandboxed under its own user, prewired to the local `/v1` API and your MCP servers, with tool-approval gating. The agent that comes home already plugged in.

---

## Get started

- 🚀 **Install:** `curl -fsSL https://hal0.dev/install.sh | bash`  *(Linux x86_64, Python ≥ 3.12)*
- 📖 **Docs:** [hal0.dev/docs/getting-started](https://hal0.dev/docs/getting-started/)
- 💻 **Source:** [github.com/hal0ai/hal0](https://github.com/hal0ai/hal0)
- ✉️ **Say hi:** [hello@hal0.dev](mailto:hello@hal0.dev)

<p align="center">
  <sub>Apache-2.0 · Linux + systemd · no telemetry by default · cosign-signed releases</sub>
</p>

<p align="center">
  <strong>★ Follow the org</strong> to get new models and quants in your feed as hal0 ships.
</p>