Instructions to use Hal0ai/hal0-brain-sft-ROCmFPX-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Hal0ai/hal0-brain-sft-ROCmFPX-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Hal0ai/hal0-brain-sft-ROCmFPX-GGUF",
	filename="hal0-brain-sft-F16.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Hal0ai/hal0-brain-sft-ROCmFPX-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16
# Run inference directly in the terminal:
llama cli -hf Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16
# Run inference directly in the terminal:
./llama-cli -hf Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16

Use Docker

docker model run hf.co/Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16

LM Studio
Jan

vLLM

How to use Hal0ai/hal0-brain-sft-ROCmFPX-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Hal0ai/hal0-brain-sft-ROCmFPX-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Hal0ai/hal0-brain-sft-ROCmFPX-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16

Ollama
How to use Hal0ai/hal0-brain-sft-ROCmFPX-GGUF with Ollama:
```
ollama run hf.co/Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16
```

Unsloth Studio

How to use Hal0ai/hal0-brain-sft-ROCmFPX-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Hal0ai/hal0-brain-sft-ROCmFPX-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Hal0ai/hal0-brain-sft-ROCmFPX-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Hal0ai/hal0-brain-sft-ROCmFPX-GGUF to start chatting

How to use Hal0ai/hal0-brain-sft-ROCmFPX-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Hal0ai/hal0-brain-sft-ROCmFPX-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use Hal0ai/hal0-brain-sft-ROCmFPX-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use Hal0ai/hal0-brain-sft-ROCmFPX-GGUF with Docker Model Runner:
```
docker model run hf.co/Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16
```

Lemonade

How to use Hal0ai/hal0-brain-sft-ROCmFPX-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Hal0ai/hal0-brain-sft-ROCmFPX-GGUF:F16

Run and chat with the model

lemonade run user.hal0-brain-sft-ROCmFPX-GGUF-F16

List all available models

lemonade list

hal0-brain-sft-ROCmFPX-GGUF / README.md

thinmint

Publish verified F16 and ROCmFPX agent quants

29b8461 verified 8 days ago

preview code

Raw

History Blame Contribute Delete

7.29 kB

metadata

license: apache-2.0
pipeline_tag: text-generation
base_model: Hal0ai/hal0-brain-sft
base_model_relation: quantized
tags:
  - hal0
  - hal0-brain
  - gguf
  - rocmfpx
  - rocmfp4
  - rocmfp8
  - agent
  - tool-use
language:
  - en

HAL0 BRAIN — ROCmFPX GGUF

Advanced reasoning · Tool calling · Platform management

HAL0 BRAIN is a mini-agent administrator trained on the hal0 system: its services, profiles, slots, model lifecycle, diagnostics, and tool workflows. This repository packages the verified F16 reference plus two agent-oriented ROCmFPX quants for structured reasoning, tool calling, and platform management.

Files

File	Preset	Bytes	SHA-256	`general.file_type`	Observed tensor types
`hal0-brain-sft-F16.gguf`	F16 reference	2,166,552,096	`ed9d28c4eac1d7c291bc80d9410c243a3d28e655921ccaf90f2b6619aa24d2c3`	1	170 `F16`, 49 `F32`
`hal0-brain-sft-Q4_0_ROCMFP4_COHERENT.gguf`	`Q4_0_ROCMFP4_COHERENT`	664,952,352	`062219355646f09e184cfb30cbfdfc429d70275f3be4d996d32644ec2b20db44`	102	169 `Q4_0_ROCMFP4`, 1 `Q6_K`, 49 `F32`
`hal0-brain-sft-Q8_0_ROCMFPX_AGENT.gguf`	`Q8_0_ROCMFPX_AGENT`	1,144,682,016	`eeb38e45877d26a5923d2ec22e1c887a0b12f0c40f9df72015bd73f11d1263e4`	115	58 `Q8_0_ROCMFPX`, 112 protected `Q8_0`, 49 `F32`
`chat-long-context.hal0profile.json`	portable hal0 profile	—	profile checksum `sha256:241af4cd2636ac1da32a8a7ca0d856724445242cfcde88a208b702b155bdee47`	—	`hal0.profile` schema 1

The Agent presets deliberately retain higher-precision tensors where structured behavior is most sensitive. The filenames are descriptive, but the format claims above come from ROCmFPX-aware GGUF inspection—not filename inference.

Provenance

Base model: Hal0ai/hal0-brain-sft
Verified F16 GGUF source: Hal0ai/hal0-brain-sft-GGUF at revision 6b190df6e816cc806f7fa7ae3de7248f5551e00b
F16 source SHA-256: ed9d28c4eac1d7c291bc80d9410c243a3d28e655921ccaf90f2b6619aa24d2c3
Quantizer: charlie12345/ROCmFPX at commit 61f2f2d7bc4955e9bca821095ef69125837133b5
Architecture: Llama-compatible GGUF, approximately 1.08B parameters
License: Apache-2.0

Both quantized files were produced independently from F16. Neither was requantized from another low-bit artifact.

Runtime compatibility

Artifact	Stock llama.cpp	ROCmFPX CPU reference	ROCmFPX HIP/ROCm	ROCmFPX Vulkan
F16	✅	✅	✅	✅
Q4 ROCmFP4 Coherent	❌	✅	✅	✅
Q8 ROCmFPX Agent	❌	✅	✅	✅

The Q4 and Q8 files contain custom GGML tensor types and require a ROCmFPX-capable runner. Current stock llama.cpp rejects the Q4 tensor type ID 100 and Q8 tensor type ID 103 as invalid/unknown. LM Studio, Ollama, or another stock llama.cpp package must not be assumed compatible.

hal0 pins the unified runner by immutable digest:

ghcr.io/hal0ai/hal0-rocmfpx@sha256:fd6b02a720e633e402e929e19eedefff52aeec18e5de8f43e525689e523985f3

That runner contains the MiniCPM5 pre-tokenizer mapping required by this model and supports both ROCm/HIP and Vulkan ROCmFPX paths.

Download

Install the Hugging Face CLI and authenticate only if your environment requires it:

pip install -U huggingface_hub

F16:

hf download Hal0ai/hal0-brain-sft-ROCmFPX-GGUF \
  hal0-brain-sft-F16.gguf --local-dir .

ROCmFP4 Coherent Agent:

hf download Hal0ai/hal0-brain-sft-ROCmFPX-GGUF \
  hal0-brain-sft-Q4_0_ROCMFP4_COHERENT.gguf --local-dir .

ROCmFP8 Agent:

hf download Hal0ai/hal0-brain-sft-ROCmFPX-GGUF \
  hal0-brain-sft-Q8_0_ROCMFPX_AGENT.gguf --local-dir .

Portable hal0 long-context profile:

hf download Hal0ai/hal0-brain-sft-ROCmFPX-GGUF \
  chat-long-context.hal0profile.json --local-dir .

Run with ROCmFPX

ROCm/HIP:

llama-cli \
  -m hal0-brain-sft-Q8_0_ROCMFPX_AGENT.gguf \
  -dev ROCm0 -ngl 999 -fa on -c 8192 -b 512 -ub 512 --jinja

Vulkan:

llama-cli \
  -m hal0-brain-sft-Q4_0_ROCMFP4_COHERENT.gguf \
  -dev Vulkan0 -ngl 999 -fa on -c 8192 -b 512 -ub 512 --jinja

OpenAI-compatible server:

llama-server \
  -m hal0-brain-sft-Q8_0_ROCMFPX_AGENT.gguf \
  --host 0.0.0.0 --port 8080 \
  -dev Vulkan0 -ngl 999 -fa on -c 8192 -b 512 -ub 512 --jinja

Choose ROCm0 or Vulkan0 according to the devices reported by the ROCmFPX runner. Long context and large batches consume substantial memory; tune them for the host.

Tool-calling contract

hal0's catalog declares the hal0-function-xml prompt contract. A complete tool call has this shape:

<function name="memory_add">
  <param name="text">Remember that the brain slot uses the ROCmFPX runner.</param>
</function>

The hal0 integration validates the tool schema, allows at most one complete tool call per turn, and stops after </function>.

Portable hal0 profile

chat-long-context.hal0profile.json is a checksummed, portable hal0.profile schema-1 envelope. It includes:

-fa on -ctk q8_0 -ctv q8_0 -b 2048 -ub 512 --parallel 1
--no-mmap --no-context-shift --poll 100 --poll-batch 1 --metrics --no-webui

These are performance-oriented defaults, not a hardware guarantee. Reduce context/batch settings or use another profile when the host cannot sustain them.

Dashboard import

Open Profiles, choose Import, select chat-long-context.hal0profile.json, review the dry-run result, and commit the import. If chat-long-context already exists, choose another name rather than overwriting it implicitly.

REST API dry-run and import

Dry-run validates the envelope and checksum without changing the profile catalog:

PROFILE=chat-long-context.hal0profile.json
jq -n --slurpfile envelope "$PROFILE" \
  '{envelope:$envelope[0],name:"chat-long-context",dry_run:true}' |
  curl --fail-with-body -sS http://127.0.0.1:8080/api/profiles/import \
    -H 'content-type: application/json' --data-binary @-

Commit only after the dry-run reports valid: true, checksum_ok: true, and no unwanted collision:

PROFILE=chat-long-context.hal0profile.json
jq -n --slurpfile envelope "$PROFILE" \
  '{envelope:$envelope[0],name:"chat-long-context",dry_run:false}' |
  curl --fail-with-body -sS http://127.0.0.1:8080/api/profiles/import \
    -H 'content-type: application/json' --data-binary @-

Validation record

ROCmFPX-aware inspection confirmed every file type and tensor count in the table.
ROCmFPX CPU reference execution loaded and benchmarked both custom artifacts.
The catalog runner image includes the required MiniCPM5 tokenizer mapping and ROCmFPX ROCm/Vulkan kernels.
Stock llama.cpp e8e6c7af2456fd50bb62f7a2bbd642e6fb14ae77 rejected Q4 type 100 and Q8 type 103 before tensor loading.

Hal0ai
/

hal0-brain-sft-ROCmFPX-GGUF

HAL0 BRAIN — ROCmFPX GGUF

Files

Provenance

Runtime compatibility

Download

Run with ROCmFPX

Tool-calling contract

Portable hal0 profile

Dashboard import

REST API dry-run and import

Validation record

Related