| --- |
| title: README |
| emoji: 🪽 |
| colorFrom: yellow |
| colorTo: gray |
| sdk: static |
| pinned: false |
| --- |
| |
| <p align="center"> |
| <img src="https://hal0.dev/og-default.png" alt="hal0 — local AI inference for AMD Strix Halo" width="640"> |
| </p> |
|
|
| <h1 align="center">hal0 · Local AI for the Ultimate Homelab</h1> |
|
|
| <p align="center"> |
| <strong>Strix Halo native AI inference, image gen & agents for homelabs.</strong> |
| </p> |
|
|
| <p align="center"> |
| <a href="https://hal0.dev"><img src="https://img.shields.io/badge/site-hal0.dev-feaf00?style=for-the-badge" alt="hal0.dev"></a> |
| <a href="https://github.com/hal0ai/hal0"><img src="https://img.shields.io/badge/GitHub-hal0ai%2Fhal0-181717?style=for-the-badge&logo=github" alt="GitHub"></a> |
| <a href="https://hal0.dev/docs/getting-started/"><img src="https://img.shields.io/badge/docs-getting%20started-444?style=for-the-badge" alt="Docs"></a> |
| <img src="https://img.shields.io/badge/license-Apache--2.0-blue?style=for-the-badge" alt="Apache-2.0"> |
| </p> |
|
|
| --- |
|
|
| ## Your Strix Halo box, running real `/v1/*` inference |
| |
| **hal0 turns a Linux box — ideally a Ryzen AI Max+ 395 — into a private, OpenAI-compatible AI appliance.** One `/v1/*` API across every modality, with concurrent workloads the box manages for you. One command installs the lot. |
|
|
| ```bash |
| curl -fsSL https://hal0.dev/install.sh | bash |
| ``` |
|
|
| > Not another llama-server wrapper — it's the orchestration around one. Stop running models from a chat tab; run one service for the whole local AI stack. |
|
|
| --- |
|
|
| ## What's in the org |
|
|
| We publish the models, quants, and artifacts that ship with hal0 — tuned for AMD Strix Halo (Ryzen AI Max) and the ROCm stack. |
|
|
| | Model | What it is | |
| |-------|------------| |
| | [**FastContext-Hal0-4B-ROCmFP4**](https://huggingface.co/Hal0ai/FastContext-Hal0-4B-ROCmFP4) | 4B fast-context chat model, quantized to **ROCmFP4** for Strix Halo iGPU inference. | |
|
|
| *More quants and companion models landing as hal0 ships. Watch the org to get pinged.* |
|
|
| --- |
|
|
| ## One `/v1/*` surface, five providers |
| |
| Drop-in for any OpenAI SDK — point your client at `:8080/v1` and go. Chat, completions, embeddings, reranking, speech-to-text, text-to-speech, and image generation, all behind one API the box schedules for you. |
| |
| | Provider | Backend | Workload | |
| |----------|---------|----------| |
| | **llama.cpp** | Vulkan / ROCm / CUDA | chat, embed, rerank, vision | |
| | **FLMv1** | AMD XDNA NPU | chat, embed | |
| | **FLM / Whisper v3 turbo** | XDNA NPU | speech-to-text | |
| | **Kokoro-82M** | CPU / Vulkan | text-to-speech (54 voices) | |
| | **ComfyUI v1** | ROCm | image gen (SDXL / SD 1.5 / Flux) | |
| |
| **Strix Halo native. Not Strix-Halo-only** — also runs on Ryzen AI Max 385/390, NVIDIA RTX 30/40/50, AMD Radeon RX 7000, and CPU-only x86_64 fallback. |
| |
| --- |
| |
| ## The operator console |
| |
| <p align="center"> |
| <img src="https://hal0.dev/screenshots/dashboard-overview.png" alt="hal0 dashboard — slots, throughput, and live service health" width="720"> |
| </p> |
| |
| Dark-by-default React admin UI with SSE-backed status and a live log tail — see slots, throughput, and service health at a glance. |
| |
| <table> |
| <tr> |
| <td width="50%"><img src="https://hal0.dev/screenshots/slots-inference.png" alt="Slots view — per-slot state and the typed inference lifecycle"></td> |
| <td width="50%"><img src="https://hal0.dev/screenshots/image-gen-comfyui.png" alt="ComfyUI image generation with the iGPU in exclusive image mode"></td> |
| </tr> |
| <tr> |
| <td align="center"><sub><b>Slots</b> — per-slot state & the typed inference lifecycle</sub></td> |
| <td align="center"><sub><b>Image gen</b> — ComfyUI on the iGPU, inference slots paused</sub></td> |
| </tr> |
| <tr> |
| <td width="50%"><img src="https://hal0.dev/screenshots/memory-graph.png" alt="Agent memory rendered as a navigable semantic and temporal knowledge graph"></td> |
| <td width="50%"><img src="https://hal0.dev/agents/hermes.png" alt="Hermes — the bundled hal0 agent"></td> |
| </tr> |
| <tr> |
| <td align="center"><sub><b>Memory graph</b> — semantic + temporal knowledge graph</sub></td> |
| <td align="center"><sub><b>Hermes</b> — the bundled, self-bootstrapping agent</sub></td> |
| </tr> |
| </table> |
| |
| --- |
| |
| ## Performance (Ryzen AI Max+ 395, 128 GB) |
| |
| | Metric | Number | |
| |--------|--------| |
| | Primary + embed, concurrent | **258 tok/s** | |
| | Primary model serving | **142 tok/s** | |
| | Dispatch latency (p50) | **174 ms** | |
| |
| --- |
| |
| ## Meet Hermes |
| |
| **Hermes installs and bootstraps itself on first run** — sandboxed under its own user, prewired to the local `/v1` API and your MCP servers, with tool-approval gating. The agent that comes home already plugged in. |
| |
| --- |
| |
| ## Get started |
| |
| - 🚀 **Install:** `curl -fsSL https://hal0.dev/install.sh | bash` *(Linux x86_64, Python ≥ 3.12)* |
| - 📖 **Docs:** [hal0.dev/docs/getting-started](https://hal0.dev/docs/getting-started/) |
| - 💻 **Source:** [github.com/hal0ai/hal0](https://github.com/hal0ai/hal0) |
| - ✉️ **Say hi:** [hello@hal0.dev](mailto:hello@hal0.dev) |
| |
| <p align="center"> |
| <sub>Apache-2.0 · Linux + systemd · no telemetry by default · cosign-signed releases</sub> |
| </p> |
| |
| <p align="center"> |
| <strong>★ Follow the org</strong> to get new models and quants in your feed as hal0 ships. |
| </p> |
| |
