Instructions to use WaveCut/Bonsai-web-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use WaveCut/Bonsai-web-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf WaveCut/Bonsai-web-GGUF:Q1_0
# Run inference directly in the terminal:
llama cli -hf WaveCut/Bonsai-web-GGUF:Q1_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf WaveCut/Bonsai-web-GGUF:Q1_0
# Run inference directly in the terminal:
llama cli -hf WaveCut/Bonsai-web-GGUF:Q1_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf WaveCut/Bonsai-web-GGUF:Q1_0
# Run inference directly in the terminal:
./llama-cli -hf WaveCut/Bonsai-web-GGUF:Q1_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf WaveCut/Bonsai-web-GGUF:Q1_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf WaveCut/Bonsai-web-GGUF:Q1_0

Use Docker

docker model run hf.co/WaveCut/Bonsai-web-GGUF:Q1_0

LM Studio
Jan

vLLM

How to use WaveCut/Bonsai-web-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "WaveCut/Bonsai-web-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "WaveCut/Bonsai-web-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/WaveCut/Bonsai-web-GGUF:Q1_0

Ollama
How to use WaveCut/Bonsai-web-GGUF with Ollama:
```
ollama run hf.co/WaveCut/Bonsai-web-GGUF:Q1_0
```

Unsloth Studio

How to use WaveCut/Bonsai-web-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for WaveCut/Bonsai-web-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for WaveCut/Bonsai-web-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for WaveCut/Bonsai-web-GGUF to start chatting

How to use WaveCut/Bonsai-web-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf WaveCut/Bonsai-web-GGUF:Q1_0

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "WaveCut/Bonsai-web-GGUF:Q1_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use WaveCut/Bonsai-web-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf WaveCut/Bonsai-web-GGUF:Q1_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default WaveCut/Bonsai-web-GGUF:Q1_0

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use WaveCut/Bonsai-web-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf WaveCut/Bonsai-web-GGUF:Q1_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "WaveCut/Bonsai-web-GGUF:Q1_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use WaveCut/Bonsai-web-GGUF with Docker Model Runner:
```
docker model run hf.co/WaveCut/Bonsai-web-GGUF:Q1_0
```

Lemonade

How to use WaveCut/Bonsai-web-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull WaveCut/Bonsai-web-GGUF:Q1_0

Run and chat with the model

lemonade run user.Bonsai-web-GGUF-Q1_0

List all available models

lemonade list

Bonsai-web-GGUF / README.md

WaveCut

Document DFlash compatibility boundary

0fd77bb verified 20 days ago

preview code

Raw

History Blame Contribute Delete

4.51 kB

	---
	license: apache-2.0
	library_name: llama.cpp
	pipeline_tag: text-generation
	tags:
	- gguf
	- 1-bit
	- q1_0
	- webgpu
	- wllama
	- on-device
	- dflash
	base_model:
	- prism-ml/Bonsai-1.7B-unpacked
	- prism-ml/Bonsai-4B-unpacked
	- prism-ml/Bonsai-8B-unpacked
	- Qwen/Qwen3.6-27B
	---

	# Bonsai Web GGUF

	Browser-ready, deterministic shards for the
	[`WaveCut/bonsai-webgpu-chat`](https://huggingface.co/spaces/WaveCut/bonsai-webgpu-chat)
	Space. The files are the pinned Prism ML Bonsai Q1_0 GGUF releases, re-sharded
	with `llama-gguf-split --split-max-size 512M` so every browser download stays
	below the current wllama single-file limit.

	## Contents

	\| Tier \| Architecture \| Download \| Shards \| CPU fallback \|
	\| --- \| --- \| ---: \| ---: \| --- \|
	\| `1_7b` \| Qwen3 dense \| 248,302,336 bytes \| 1 \| yes \|
	\| `4b` \| Qwen3 dense \| 572,270,816 bytes \| 2 \| yes \|
	\| `8b` \| Qwen3 dense \| 1,158,654,816 bytes \| 3 \| yes \|
	\| `27b` \| Qwen3.5 hybrid \| 3,803,453,440 bytes \| 8 \| no \|

	The repository also carries
	`dflash/Qwen3.6-27B-DFlash-Q8_0.gguf` (1,849,481,440 bytes) as a
	native correctness reference only. It is not part of a tier download and
	is not loaded by the browser Space.

	`manifest.json` is the machine-readable contract: immutable source provenance,
	ordered shard paths, bytes, SHA-256, context defaults, WebGPU limits, chat
	template evidence, and the default-off DFlash research status. `SHA256SUMS`
	is the independent shard checksum list.

	## Source provenance

	\| Tier \| Source repository \| Pinned revision \| Original SHA-256 \|
	\| --- \| --- \| --- \| --- \|
	\| `1_7b` \| `prism-ml/Bonsai-1.7B-gguf` \| `210a9e99f79cb184909d49595906526eb2b3dd9a` \| `3d7c6c90dd98717a203adb22d5eacd2581850e40aa5327e144b97766cae5f7e3` \|
	\| `4b` \| `prism-ml/Bonsai-4B-gguf` \| `78f2c2bacd0904ffaba24b4873ed975e5818354a` \| `4524b3f997f0f06444e568d1f26e2efd69effa3218c7ad3047432fb171e42168` \|
	\| `8b` \| `prism-ml/Bonsai-8B-gguf` \| `48516770dd04643643e9f9019a2a349cf26c5dbd` \| `284a335aa3fb2ced3b1b01fcb40b08aa783e3b70832767f0dd2e3fdfa134bd54` \|
	\| `27b` \| `prism-ml/Bonsai-27B-gguf` \| `0cf7e3d21581b169b4df1de8bf01316000e2fbb7` \| `17ef842e47450caeb8eaa3ebfbbab5d2f2278b62b79be107985fb69a2f819aa0` \|

	The shard container checksums differ from each original monolithic file because
	GGUF splitting adds split metadata. No quantization or tensor conversion is
	performed.

	The first shard of every tier was loaded through llama.cpp's automatic
	multi-file resolver and compared with its monolithic source under greedy
	sampling. Token IDs and text match exactly: 64/64 tokens for 1.7B and 27B,
	and 16/16 tokens for 4B and 8B. The shorter middle-tier checks are load and
	reassembly guards, not quality evaluations.

	## Loading with wllama

	Pass the ordered URLs from `manifest.json` to `loadModelFromUrl`:

	```ts
	await wllama.loadModelFromUrl(model.files.map((file) => file.url), {
	n_ctx: model.defaultContext,
	n_gpu_layers: 99999,
	flash_attn: false,
	useCache: true,
	});
	```

	The 27B tier is WebGPU-only in the Space. The application must reject it when
	adapter limits, storage quota, or full-graph placement are insufficient; it
	must not silently fall back to CPU.

	## Runtime evidence

	The release gate uses pinned llama.cpp `00fa7cb284cbf133fc426733bd64238a3588a33e`
	and Dawn `18eb229ef5f707c1464cc581252e7603c73a3ef0`. On Apple M2 Max, all required
	Q1_0 and Qwen3.5 hybrid WebGPU operation tests passed. Full-graph 1.7B and 27B
	inference requires the token embedding tensor to remain on WebGPU; Flash
	Attention stays disabled because quantized K/V combinations are not fully
	supported.

	## DFlash

	DFlash remains inspection-only and default-off in the browser. The manifest
	pins the BF16 source and the bundled Q8_0 GGUF reference. Shape, complete
	tokenizer equality, target taps `[2, 17, 32, 47, 62]`, conversion, and a
	32-token temperature-0 native Bonsai-27B parity smoke pass.

	The current custom wllama build contains the DFlash llama.cpp architecture,
	but its TypeScript/worker/glue load path does not yet expose
	`spec_type=draft-dflash`. The Q8_0 `fc.weight` is 139,264,000 bytes, above the
	128 MiB baseline WebGPU storage-buffer binding limit. Browser activation,
	acceptance measurement, and a positive decode-speed result therefore remain
	hard release gates. The reference must not be presented as a browser-ready
	drafter.

	## License

	The Bonsai model files are published under Apache License 2.0. The DFlash
	reference is MIT-licensed. The source license is included as `LICENSE` and
	tier/DFlash attribution is preserved under `notices/`.