Instructions to use vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16
# Run inference directly in the terminal:
./llama-cli -hf vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16

Use Docker

docker model run hf.co/vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16

LM Studio
Jan

vLLM

How to use vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16

Ollama
How to use vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF with Ollama:
```
ollama run hf.co/vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16
```

Unsloth Studio

How to use vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF to start chatting

How to use vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF with Docker Model Runner:
```
docker model run hf.co/vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16
```

Lemonade

How to use vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16

Run and chat with the model

lemonade run user.Muse-Glimmer-30B-heretic-ROCmFPX-GGUF-BF16

List all available models

lemonade list

Hermes Agent

How to use vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF:BF16

Run Hermes

hermes

Atomic Chat

Muse-Glimmer-30B-heretic-ROCmFPX-GGUF / README.md

vmlinux

Publish Muse Glimmer 30B Heretic ROCmFPX GGUF release

99f8189 verified 1 day ago

preview code

Raw

History Blame Contribute Delete

10.8 kB

	---
	base_model:
	- darkc0de/Muse-Glimmer-30B-heretic
	- meta-models/Muse-Glimmer-30B-assistant
	base_model_relation: quantized
	library_name: llama.cpp
	pipeline_tag: image-text-to-text
	license: apache-2.0
	tags:
	- gguf
	- llama.cpp
	- rocm
	- rocmfpx
	- amd
	- strix-halo
	- gfx1151
	- imatrix
	- multimodal
	- muse-glimmer
	- conversational
	- heretic
	- uncensored
	- decensored
	- abliterated
	- dflash
	- speculative-decoding
	---

	# Muse-Glimmer-30B Heretic ROCmFPX GGUF

	ROCmFP4 and ROCmFP8 builds of
	[darkc0de/Muse-Glimmer-30B-heretic](https://huggingface.co/darkc0de/Muse-Glimmer-30B-heretic),
	targeted and tested on AMD Strix Halo (`gfx1151`). The source is a reproducible
	Heretic v1.4.0 abliteration of Meta's Muse Glimmer 30B. Rawr. 🦖

	These are custom ROCmFPX formats, not ordinary llama.cpp Q4/Q8 files. Read the
	compatibility section before downloading.

	> Experimental runtime required: these GGUFs need a patched build of
	> [charlie12345/ROCmFPX](https://github.com/charlie12345/ROCmFPX). Stock
	> llama.cpp does not implement the ROCmFP4/ROCmFP8 tensor layouts, while the
	> pinned ROCmFPX base predates Muse Glimmer support. Apply the included
	> [`ROCmFPX-Muse-Glimmer.patch`](https://huggingface.co/vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF/blob/main/ROCmFPX-Muse-Glimmer.patch)
	> to ROCmFPX commit `00d54526e…`, then build that checkout.

	## Files

	Click a filename to download it directly from the Hub.

	\| File \| ROCmFPX preset \| Size \| BPW \| iMatrix \|
	\| --- \| --- \| ---: \| ---: \| --- \|
	\| [`Muse-Glimmer-30B-heretic-ROCmFP4.gguf`](https://huggingface.co/vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF/resolve/main/Muse-Glimmer-30B-heretic-ROCmFP4.gguf?download=true) \| `Q4_0_ROCMFP4_STRIX` \| 14.17 GiB \| 4.36 \| Yes \|
	\| [`Muse-Glimmer-30B-heretic-ROCmFP4-Q6-QUALITY.gguf`](https://huggingface.co/vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF/resolve/main/Muse-Glimmer-30B-heretic-ROCmFP4-Q6-QUALITY.gguf?download=true) \| `Q4_0_ROCMFP4_COHERENT` \| 14.93 GiB \| 4.60 \| Yes \|
	\| [`Muse-Glimmer-30B-heretic-ROCmFP8.gguf`](https://huggingface.co/vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF/resolve/main/Muse-Glimmer-30B-heretic-ROCmFP8.gguf?download=true) \| `Q8_0_ROCMFPX` \| 26.77 GiB \| 8.25 \| No \|
	\| [`mmproj-Muse-Glimmer-30B-heretic-BF16.gguf`](https://huggingface.co/vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF/resolve/main/mmproj-Muse-Glimmer-30B-heretic-BF16.gguf?download=true) \| BF16 vision projector \| 3.58 GiB \| — \| — \|
	\| [`Muse-Glimmer-30B-DFlash-ROCmFP4.gguf`](https://huggingface.co/vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF/resolve/main/Muse-Glimmer-30B-DFlash-ROCmFP4.gguf?download=true) \| `Q4_0_ROCMFP4_STRIX` drafter \| 1.39 GiB \| 4.63 \| No \|
	\| [`Muse-Glimmer-30B-DFlash-ROCmFP8.gguf`](https://huggingface.co/vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF/resolve/main/Muse-Glimmer-30B-DFlash-ROCmFP8.gguf?download=true) \| `Q8_0_ROCMFPX` drafter \| 2.47 GiB \| 8.25 \| No \|

	Suggested choices:

	- ROCmFP4: default Strix Halo speed/quality build. Fast FP4 body,
	dual-scale FP4 attention K/V, and Q6_K token embeddings.
	- ROCmFP4-Q6-QUALITY: coherence-biased build. Dual-scale FP4 throughout
	the body with Q6_K token embeddings.
	- ROCmFP8: high-fidelity 8.25-bpw reference build.
	- DFlash ROCmFP8: recommended drafter for this Heretic target on the
	tested host. It produced the best measured six-token speculative speed.
	- DFlash ROCmFP4: smaller drafter alternative.

	The fresh Heretic-labelled BF16 projector works with all three text models.

	## About the abliteration

	The source model was produced with
	[Heretic](https://heretic-project.org) v1.4.0 and retains Muse Glimmer's
	architecture, tokenizer, chat template, perception encoder, and native
	131,072-token context. The source author reports KL divergence `0.0743` from
	the original and 11 refusals out of 100, compared with 59/100 for the original.

	Published abliteration parameters:

	\| Parameter \| Value \|
	\| --- \| ---: \|
	\| `direction_index` \| 38.49 \|
	\| `attn.o_proj.max_weight` \| 1.46 \|
	\| `attn.o_proj.max_weight_position` \| 30.98 \|
	\| `attn.o_proj.min_weight` \| 0.75 \|
	\| `attn.o_proj.min_weight_distance` \| 29.88 \|
	\| `mlp.down_proj.max_weight` \| 1.49 \|
	\| `mlp.down_proj.max_weight_position` \| 31.58 \|
	\| `mlp.down_proj.min_weight` \| 0.32 \|
	\| `mlp.down_proj.min_weight_distance` \| 26.40 \|

	See the
	[source model card](https://huggingface.co/darkc0de/Muse-Glimmer-30B-heretic)
	for its reproduction recipe and the original author's description. These
	figures describe the BF16 source; this repository did not rerun that evaluation
	on each quantization.

	## iMatrix

	Both FP4 targets use the same GGUF importance matrix:

	- 500 chunks × 512 tokens (approximately 256k calibration tokens)
	- 416 tensor importance entries consumed by each quantizer
	- varied narrative/general-language calibration corpus
	- checkpoints saved every 100 chunks

	`Q8_0_ROCMFPX` does not consume importance weights, so the FP8 reference was
	intentionally built without an iMatrix.

	The two DFlash files are quantizations of Meta's official
	[Muse-Glimmer-30B-assistant](https://huggingface.co/meta-models/Muse-Glimmer-30B-assistant),
	not a separately trained Heretic assistant. Every proposal is still verified
	by the Heretic target. Compatibility and speed were measured rather than
	assumed; see Validation below.

	## Compatibility

	These files use experimental ROCmFPX tensor types and **will not load in stock
	llama.cpp**.

	The validated runtime was built from:

	- [ROCmFPX](https://github.com/charlie12345/ROCmFPX) base commit
	`00d54526e24e3aba4c76474e3147cbf9c7cc034c`
	- upstream llama.cpp Muse converter commit
	`d2f83055d6e3b379b5d34c4837122a918cf402c2`
	- the included Muse text, vision, DFlash, sparse-attention, and quantized-cache
	compatibility patch

	The runtime was built with ROCm and Vulkan backends. Reported generation tests
	used `ROCm0` on `gfx1151`.

	Minimal runtime setup:

	```bash
	git clone https://github.com/charlie12345/ROCmFPX.git
	cd ROCmFPX
	git checkout 00d54526e24e3aba4c76474e3147cbf9c7cc034c

	hf download vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF \
	ROCmFPX-Muse-Glimmer.patch \
	--local-dir /tmp/muse-glimmer-heretic-rocmfpx
	git apply /tmp/muse-glimmer-heretic-rocmfpx/ROCmFPX-Muse-Glimmer.patch

	BUILD_DIR=build-muse-rocmfpx \
	JOBS=16 \
	CMAKE_HIP_COMPILER=/opt/rocm-7.2.0/lib/llvm/bin/clang++ \
	GGML_HIP_ROCWMMA_FATTN=OFF \
	./scripts/build-strix-rocmfp4-mtp.sh
	```

	Adjust `CMAKE_HIP_COMPILER` for the installed ROCm version. The patch must be
	applied to the exact pinned commit.

	## Download and run

	```bash
	hf download vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF \
	Muse-Glimmer-30B-heretic-ROCmFP4.gguf \
	Muse-Glimmer-30B-DFlash-ROCmFP8.gguf \
	mmproj-Muse-Glimmer-30B-heretic-BF16.gguf \
	--local-dir ./Muse-Glimmer-30B-heretic-ROCmFPX
	```

	Text:

	```bash
	./llama-cli \
	-m ./Muse-Glimmer-30B-heretic-ROCmFPX/Muse-Glimmer-30B-heretic-ROCmFP4.gguf \
	-dev ROCm0 -ngl all -c 8192 -cnv
	```

	Vision:

	```bash
	./llama-cli \
	-m ./Muse-Glimmer-30B-heretic-ROCmFPX/Muse-Glimmer-30B-heretic-ROCmFP4.gguf \
	-mm ./Muse-Glimmer-30B-heretic-ROCmFPX/mmproj-Muse-Glimmer-30B-heretic-BF16.gguf \
	--image ./image.png \
	-p "Describe this image." \
	-dev ROCm0 -ngl all -c 8192 -cnv -st
	```

	DFlash speculative decoding (recommended starting point):

	```bash
	./llama-cli \
	-m ./Muse-Glimmer-30B-heretic-ROCmFPX/Muse-Glimmer-30B-heretic-ROCmFP4.gguf \
	--spec-draft-model ./Muse-Glimmer-30B-heretic-ROCmFPX/Muse-Glimmer-30B-DFlash-ROCmFP8.gguf \
	--spec-type draft-dflash \
	-dev ROCm0 -ngl all \
	--spec-draft-device ROCm0 --spec-draft-ngl all \
	-ctk q4_0 -ctv q4_0 \
	--spec-draft-type-k q4_0 --spec-draft-type-v q4_0 \
	--spec-draft-n-max 6 --spec-draft-n-min 0 \
	--spec-draft-p-min 0.0 --spec-draft-p-split 0.10 \
	--no-spec-draft-backend-sampling \
	-c 8192 -cnv
	```

	## Validation

	All three target quantizations loaded on `ROCm0` with every layer offloaded and
	generated tokens. The default FP4 also passed an end-to-end image test using
	the fresh projector; it correctly identified both the Hugging Face site and
	the repository shown in the test screenshot.

	Three deterministic 256-token runs compared native ROCmFP4 decoding with the
	official FP8 DFlash. Both used Q4_0 target KV cache, seed 42, temperature 0, a
	4,096-token context, six-token drafts, and the same prompts.

	\| Prompt \| Native tok/s \| DFlash tok/s \| Accepted / proposed \|
	\| --- \| ---: \| ---: \| ---: \|
	\| Technical explanation \| 13.8 \| 22.4 \| 155 / 595 \|
	\| Backup strategy \| 13.7 \| 24.1 \| 163 / 543 \|
	\| Fiction opening \| 13.8 \| 21.0 \| 149 / 628 \|
	\| Mean / aggregate \| 13.77 \| 22.50 \| 467 / 1,766 (26.44%) \|

	The measured mean speedup was 1.63×. At least one draft token was accepted
	in 199 of 297 verification rounds (67.0%), and the mean accepted span including
	the target token was 2.57 tokens. In a technical-prompt comparison, the smaller
	FP4 DFlash reached 20.3 tok/s with 140/684 proposals accepted, so FP8 is the
	recommended drafter on this host.

	Native and speculative greedy outputs were not byte-identical. With custom
	ROCmFP4 kernels, speculative verification changes target batch shapes; small
	floating-point differences can redirect the model's reasoning trace. Treat
	this cross-checkpoint pairing as a measured throughput option, not a claim of
	bit-exact decoding.

	Additional verification:

	- all 13 downloaded Heretic Safetensors shards passed the publisher's manifest
	- both FP4 targets consumed all 416 iMatrix entries
	- all six release files pass the published `SHA256SUMS`
	- source, intermediate, calibration, and patch hashes are retained in
	[`PROVENANCE_SHA256SUMS`](https://huggingface.co/vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF/blob/main/PROVENANCE_SHA256SUMS)
	- detailed build and validation report:
	[`BUILD_RESULTS.md`](https://huggingface.co/vmlinux/Muse-Glimmer-30B-heretic-ROCmFPX-GGUF/blob/main/BUILD_RESULTS.md)

	## DFlash, not MTP

	The companion files are converted from Meta's official five-layer
	`MuseGlimmerAssistantModel`, which uses DFlash block diffusion with a trained
	block size of 16. Run it with `--spec-type draft-dflash`; it is not an MTP
	checkpoint and should not be run with `draft-mtp`.

	## Provenance

	- Heretic source revision: `64a36ddcb9745b521bd9eb114465c93f860a594f`
	- Original target revision: `f84ecc3a0ea984a4c04542a84269e3d065350a6e`
	- DFlash source revision: `2c86316d689027b91123638739743fef1d425233`
	- ROCmFPX base: `00d54526e24e3aba4c76474e3147cbf9c7cc034c`
	- Conversion: upstream llama.cpp `d2f83055d6e3b379b5d34c4837122a918cf402c2`

	The source model's Apache 2.0 license and Muse Glimmer usage policy apply. This
	abliterated model may be more likely to produce unsafe, objectionable, or
	unreliable output; deployers should perform their own evaluation and add
	guardrails appropriate to their application.