thinmint commited on
Commit
435a397
Β·
verified Β·
1 Parent(s): df12bd4

Add hal0 organization card

Browse files
Files changed (1) hide show
  1. README.md +121 -4
README.md CHANGED
@@ -1,10 +1,127 @@
1
  ---
2
  title: README
3
- emoji: 🐠
4
- colorFrom: gray
5
- colorTo: blue
6
  sdk: static
7
  pinned: false
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  title: README
3
+ emoji: πŸͺ½
4
+ colorFrom: yellow
5
+ colorTo: gray
6
  sdk: static
7
  pinned: false
8
  ---
9
 
10
+ <p align="center">
11
+ <img src="https://hal0.dev/og-default.png" alt="hal0 β€” local AI inference for AMD Strix Halo" width="640">
12
+ </p>
13
+
14
+ <h1 align="center">hal0 Β· Local AI for the Homelab</h1>
15
+
16
+ <p align="center">
17
+ <strong>Strix Halo native AI inference, image gen &amp; agents for homelabs.</strong>
18
+ </p>
19
+
20
+ <p align="center">
21
+ <a href="https://hal0.dev"><img src="https://img.shields.io/badge/site-hal0.dev-feaf00?style=for-the-badge" alt="hal0.dev"></a>
22
+ <a href="https://github.com/hal0ai/hal0"><img src="https://img.shields.io/badge/GitHub-hal0ai%2Fhal0-181717?style=for-the-badge&logo=github" alt="GitHub"></a>
23
+ <a href="https://hal0.dev/docs/getting-started/"><img src="https://img.shields.io/badge/docs-getting%20started-444?style=for-the-badge" alt="Docs"></a>
24
+ <img src="https://img.shields.io/badge/license-Apache--2.0-blue?style=for-the-badge" alt="Apache-2.0">
25
+ </p>
26
+
27
+ ---
28
+
29
+ ## Your Strix Halo box, running real `/v1/*` inference
30
+
31
+ **hal0 turns a Linux box β€” ideally a Ryzen AI Max+ 395 β€” into a private, OpenAI-compatible AI appliance.** One `/v1/*` API across every modality, with concurrent workloads the box manages for you. One command installs the lot.
32
+
33
+ ```bash
34
+ curl -fsSL https://hal0.dev/install.sh | bash
35
+ ```
36
+
37
+ > Not another llama-server wrapper β€” it's the orchestration around one. Stop running models from a chat tab; run one service for the whole local AI stack.
38
+
39
+ ---
40
+
41
+ ## What's in the org
42
+
43
+ We publish the models, quants, and artifacts that ship with hal0 β€” tuned for AMD Strix Halo (Ryzen AI Max) and the ROCm stack.
44
+
45
+ | Model | What it is |
46
+ |-------|------------|
47
+ | [**FastContext-Hal0-4B-ROCmFP4**](https://huggingface.co/Hal0ai/FastContext-Hal0-4B-ROCmFP4) | 4B fast-context chat model, quantized to **ROCmFP4** for Strix Halo iGPU inference. |
48
+
49
+ *More quants and companion models landing as hal0 ships. Watch the org to get pinged.*
50
+
51
+ ---
52
+
53
+ ## One `/v1/*` surface, five providers
54
+
55
+ Drop-in for any OpenAI SDK β€” point your client at `:8080/v1` and go. Chat, completions, embeddings, reranking, speech-to-text, text-to-speech, and image generation, all behind one API the box schedules for you.
56
+
57
+ | Provider | Backend | Workload |
58
+ |----------|---------|----------|
59
+ | **llama.cpp** | Vulkan / ROCm / CUDA | chat, embed, rerank, vision |
60
+ | **FLMv1** | AMD XDNA NPU | chat, embed |
61
+ | **FLM / Whisper v3 turbo** | XDNA NPU | speech-to-text |
62
+ | **Kokoro-82M** | CPU / Vulkan | text-to-speech (54 voices) |
63
+ | **ComfyUI v1** | ROCm | image gen (SDXL / SD 1.5 / Flux) |
64
+
65
+ **Strix Halo native. Not Strix-Halo-only** β€” also runs on Ryzen AI Max 385/390, NVIDIA RTX 30/40/50, AMD Radeon RX 7000, and CPU-only x86_64 fallback.
66
+
67
+ ---
68
+
69
+ ## The operator console
70
+
71
+ <p align="center">
72
+ <img src="https://hal0.dev/screenshots/dashboard-overview.png" alt="hal0 dashboard β€” slots, throughput, and live service health" width="720">
73
+ </p>
74
+
75
+ Dark-by-default React admin UI with SSE-backed status and a live log tail β€” see slots, throughput, and service health at a glance.
76
+
77
+ <table>
78
+ <tr>
79
+ <td width="50%"><img src="https://hal0.dev/screenshots/slots-inference.png" alt="Slots view β€” per-slot state and the typed inference lifecycle"></td>
80
+ <td width="50%"><img src="https://hal0.dev/screenshots/image-gen-comfyui.png" alt="ComfyUI image generation with the iGPU in exclusive image mode"></td>
81
+ </tr>
82
+ <tr>
83
+ <td align="center"><sub><b>Slots</b> β€” per-slot state &amp; the typed inference lifecycle</sub></td>
84
+ <td align="center"><sub><b>Image gen</b> β€” ComfyUI on the iGPU, inference slots paused</sub></td>
85
+ </tr>
86
+ <tr>
87
+ <td width="50%"><img src="https://hal0.dev/screenshots/memory-graph.png" alt="Agent memory rendered as a navigable semantic and temporal knowledge graph"></td>
88
+ <td width="50%"><img src="https://hal0.dev/agents/hermes.png" alt="Hermes β€” the bundled hal0 agent"></td>
89
+ </tr>
90
+ <tr>
91
+ <td align="center"><sub><b>Memory graph</b> β€” semantic + temporal knowledge graph</sub></td>
92
+ <td align="center"><sub><b>Hermes</b> β€” the bundled, self-bootstrapping agent</sub></td>
93
+ </tr>
94
+ </table>
95
+
96
+ ---
97
+
98
+ ## Performance (Ryzen AI Max+ 395, 128 GB)
99
+
100
+ | Metric | Number |
101
+ |--------|--------|
102
+ | Primary + embed, concurrent | **258 tok/s** |
103
+ | Primary model serving | **142 tok/s** |
104
+ | Dispatch latency (p50) | **174 ms** |
105
+
106
+ ---
107
+
108
+ ## Meet Hermes
109
+
110
+ **Hermes installs and bootstraps itself on first run** β€” sandboxed under its own user, prewired to the local `/v1` API and your MCP servers, with tool-approval gating. The agent that comes home already plugged in.
111
+
112
+ ---
113
+
114
+ ## Get started
115
+
116
+ - πŸš€ **Install:** `curl -fsSL https://hal0.dev/install.sh | bash` *(Linux x86_64, Python β‰₯ 3.12)*
117
+ - πŸ“– **Docs:** [hal0.dev/docs/getting-started](https://hal0.dev/docs/getting-started/)
118
+ - πŸ’» **Source:** [github.com/hal0ai/hal0](https://github.com/hal0ai/hal0)
119
+ - βœ‰οΈ **Say hi:** [hello@hal0.dev](mailto:hello@hal0.dev)
120
+
121
+ <p align="center">
122
+ <sub>Apache-2.0 Β· Linux + systemd Β· no telemetry by default Β· cosign-signed releases</sub>
123
+ </p>
124
+
125
+ <p align="center">
126
+ <strong>β˜… Follow the org</strong> to get new models and quants in your feed as hal0 ships.
127
+ </p>