Add hal0 organization card
Browse files
README.md
CHANGED
|
@@ -1,10 +1,127 @@
|
|
| 1 |
---
|
| 2 |
title: README
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
title: README
|
| 3 |
+
emoji: πͺ½
|
| 4 |
+
colorFrom: yellow
|
| 5 |
+
colorTo: gray
|
| 6 |
sdk: static
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
+
<p align="center">
|
| 11 |
+
<img src="https://hal0.dev/og-default.png" alt="hal0 β local AI inference for AMD Strix Halo" width="640">
|
| 12 |
+
</p>
|
| 13 |
+
|
| 14 |
+
<h1 align="center">hal0 Β· Local AI for the Homelab</h1>
|
| 15 |
+
|
| 16 |
+
<p align="center">
|
| 17 |
+
<strong>Strix Halo native AI inference, image gen & agents for homelabs.</strong>
|
| 18 |
+
</p>
|
| 19 |
+
|
| 20 |
+
<p align="center">
|
| 21 |
+
<a href="https://hal0.dev"><img src="https://img.shields.io/badge/site-hal0.dev-feaf00?style=for-the-badge" alt="hal0.dev"></a>
|
| 22 |
+
<a href="https://github.com/hal0ai/hal0"><img src="https://img.shields.io/badge/GitHub-hal0ai%2Fhal0-181717?style=for-the-badge&logo=github" alt="GitHub"></a>
|
| 23 |
+
<a href="https://hal0.dev/docs/getting-started/"><img src="https://img.shields.io/badge/docs-getting%20started-444?style=for-the-badge" alt="Docs"></a>
|
| 24 |
+
<img src="https://img.shields.io/badge/license-Apache--2.0-blue?style=for-the-badge" alt="Apache-2.0">
|
| 25 |
+
</p>
|
| 26 |
+
|
| 27 |
+
---
|
| 28 |
+
|
| 29 |
+
## Your Strix Halo box, running real `/v1/*` inference
|
| 30 |
+
|
| 31 |
+
**hal0 turns a Linux box β ideally a Ryzen AI Max+ 395 β into a private, OpenAI-compatible AI appliance.** One `/v1/*` API across every modality, with concurrent workloads the box manages for you. One command installs the lot.
|
| 32 |
+
|
| 33 |
+
```bash
|
| 34 |
+
curl -fsSL https://hal0.dev/install.sh | bash
|
| 35 |
+
```
|
| 36 |
+
|
| 37 |
+
> Not another llama-server wrapper β it's the orchestration around one. Stop running models from a chat tab; run one service for the whole local AI stack.
|
| 38 |
+
|
| 39 |
+
---
|
| 40 |
+
|
| 41 |
+
## What's in the org
|
| 42 |
+
|
| 43 |
+
We publish the models, quants, and artifacts that ship with hal0 β tuned for AMD Strix Halo (Ryzen AI Max) and the ROCm stack.
|
| 44 |
+
|
| 45 |
+
| Model | What it is |
|
| 46 |
+
|-------|------------|
|
| 47 |
+
| [**FastContext-Hal0-4B-ROCmFP4**](https://huggingface.co/Hal0ai/FastContext-Hal0-4B-ROCmFP4) | 4B fast-context chat model, quantized to **ROCmFP4** for Strix Halo iGPU inference. |
|
| 48 |
+
|
| 49 |
+
*More quants and companion models landing as hal0 ships. Watch the org to get pinged.*
|
| 50 |
+
|
| 51 |
+
---
|
| 52 |
+
|
| 53 |
+
## One `/v1/*` surface, five providers
|
| 54 |
+
|
| 55 |
+
Drop-in for any OpenAI SDK β point your client at `:8080/v1` and go. Chat, completions, embeddings, reranking, speech-to-text, text-to-speech, and image generation, all behind one API the box schedules for you.
|
| 56 |
+
|
| 57 |
+
| Provider | Backend | Workload |
|
| 58 |
+
|----------|---------|----------|
|
| 59 |
+
| **llama.cpp** | Vulkan / ROCm / CUDA | chat, embed, rerank, vision |
|
| 60 |
+
| **FLMv1** | AMD XDNA NPU | chat, embed |
|
| 61 |
+
| **FLM / Whisper v3 turbo** | XDNA NPU | speech-to-text |
|
| 62 |
+
| **Kokoro-82M** | CPU / Vulkan | text-to-speech (54 voices) |
|
| 63 |
+
| **ComfyUI v1** | ROCm | image gen (SDXL / SD 1.5 / Flux) |
|
| 64 |
+
|
| 65 |
+
**Strix Halo native. Not Strix-Halo-only** β also runs on Ryzen AI Max 385/390, NVIDIA RTX 30/40/50, AMD Radeon RX 7000, and CPU-only x86_64 fallback.
|
| 66 |
+
|
| 67 |
+
---
|
| 68 |
+
|
| 69 |
+
## The operator console
|
| 70 |
+
|
| 71 |
+
<p align="center">
|
| 72 |
+
<img src="https://hal0.dev/screenshots/dashboard-overview.png" alt="hal0 dashboard β slots, throughput, and live service health" width="720">
|
| 73 |
+
</p>
|
| 74 |
+
|
| 75 |
+
Dark-by-default React admin UI with SSE-backed status and a live log tail β see slots, throughput, and service health at a glance.
|
| 76 |
+
|
| 77 |
+
<table>
|
| 78 |
+
<tr>
|
| 79 |
+
<td width="50%"><img src="https://hal0.dev/screenshots/slots-inference.png" alt="Slots view β per-slot state and the typed inference lifecycle"></td>
|
| 80 |
+
<td width="50%"><img src="https://hal0.dev/screenshots/image-gen-comfyui.png" alt="ComfyUI image generation with the iGPU in exclusive image mode"></td>
|
| 81 |
+
</tr>
|
| 82 |
+
<tr>
|
| 83 |
+
<td align="center"><sub><b>Slots</b> β per-slot state & the typed inference lifecycle</sub></td>
|
| 84 |
+
<td align="center"><sub><b>Image gen</b> β ComfyUI on the iGPU, inference slots paused</sub></td>
|
| 85 |
+
</tr>
|
| 86 |
+
<tr>
|
| 87 |
+
<td width="50%"><img src="https://hal0.dev/screenshots/memory-graph.png" alt="Agent memory rendered as a navigable semantic and temporal knowledge graph"></td>
|
| 88 |
+
<td width="50%"><img src="https://hal0.dev/agents/hermes.png" alt="Hermes β the bundled hal0 agent"></td>
|
| 89 |
+
</tr>
|
| 90 |
+
<tr>
|
| 91 |
+
<td align="center"><sub><b>Memory graph</b> β semantic + temporal knowledge graph</sub></td>
|
| 92 |
+
<td align="center"><sub><b>Hermes</b> β the bundled, self-bootstrapping agent</sub></td>
|
| 93 |
+
</tr>
|
| 94 |
+
</table>
|
| 95 |
+
|
| 96 |
+
---
|
| 97 |
+
|
| 98 |
+
## Performance (Ryzen AI Max+ 395, 128 GB)
|
| 99 |
+
|
| 100 |
+
| Metric | Number |
|
| 101 |
+
|--------|--------|
|
| 102 |
+
| Primary + embed, concurrent | **258 tok/s** |
|
| 103 |
+
| Primary model serving | **142 tok/s** |
|
| 104 |
+
| Dispatch latency (p50) | **174 ms** |
|
| 105 |
+
|
| 106 |
+
---
|
| 107 |
+
|
| 108 |
+
## Meet Hermes
|
| 109 |
+
|
| 110 |
+
**Hermes installs and bootstraps itself on first run** β sandboxed under its own user, prewired to the local `/v1` API and your MCP servers, with tool-approval gating. The agent that comes home already plugged in.
|
| 111 |
+
|
| 112 |
+
---
|
| 113 |
+
|
| 114 |
+
## Get started
|
| 115 |
+
|
| 116 |
+
- π **Install:** `curl -fsSL https://hal0.dev/install.sh | bash` *(Linux x86_64, Python β₯ 3.12)*
|
| 117 |
+
- π **Docs:** [hal0.dev/docs/getting-started](https://hal0.dev/docs/getting-started/)
|
| 118 |
+
- π» **Source:** [github.com/hal0ai/hal0](https://github.com/hal0ai/hal0)
|
| 119 |
+
- βοΈ **Say hi:** [hello@hal0.dev](mailto:hello@hal0.dev)
|
| 120 |
+
|
| 121 |
+
<p align="center">
|
| 122 |
+
<sub>Apache-2.0 Β· Linux + systemd Β· no telemetry by default Β· cosign-signed releases</sub>
|
| 123 |
+
</p>
|
| 124 |
+
|
| 125 |
+
<p align="center">
|
| 126 |
+
<strong>β
Follow the org</strong> to get new models and quants in your feed as hal0 ships.
|
| 127 |
+
</p>
|