Instructions to use TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired

Run Hermes

hermes

OpenClaw new

How to use TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

MLX LM

How to use TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Bonsai 27B MLX BF16 Config-Repaired

This is a mechanically converted MLX BF16 archival baseline for the Bonsai 27B quantization experiment on Apple Silicon. It exists to make later Q4, Q8, oQ4, and oQ8 builds reproducible without treating an opaque local directory as the source of truth.

It is not a recommended inference checkpoint. The 50 GB BF16 footprint cannot be run on the 32 GB Apple Silicon host used to create it, so this upload establishes structure and provenance, not runtime quality.

Provenance

Upstream source: prism-ml/Bonsai-27B-gguf
Source revision: 0cf7e3d21581b169b4df1de8bf01316000e2fbb7
Source file: Bonsai-27B-F16.gguf
Source SHA-256: d4a381a6d07131c34af888607bdbda49fc885c97673a0d22aa3e0f0284bba566
Base architecture: Qwen/Qwen3.6-27B through Bonsai 27B
Conversion output: MLX-compatible safetensors, 11 shards, 851 indexed tensors
License: Apache-2.0. Upstream LICENSE.txt and NOTICE.txt are retained.

The conversion preserved tensor values. One configuration-only repair was made: the source declares one MTP layer but includes no mtp.* tensor. The derived config.json sets text_config.mtp_num_hidden_layers to 0; the untouched configuration is retained as config.pre-mtp-repair.json. See MTP_CONFIGURATION_REPAIR.md.

Integrity

output-sha256.txt records SHA-256 hashes for every model shard and core metadata file. PROVENANCE.json records the source, structural checks, and local artifact hashes. The current derived config.json hash is 4dd1fd34daa8916c5eecc8e6a9ca9ce3564226518704768efe9b76d4deb34b2f.

Intended Use

Use this only as a reproducible conversion baseline for a controlled local quantization study. Follow-on releases should publish their conversion recipe, calibration settings, evaluation results, and exact artifact hashes. A generic 2-bit affine experiment from this study failed deterministic output screens and is intentionally not distributed as a usable model.

Attribution

Created by Technologies Brewster Jennings du Canada for the Bonsai MLX quantization experiment. Bonsai is created by Prism ML and derived from Qwen3.6-27B. Please consult the upstream Bonsai card for original model limitations, benchmarks, and citations.

Downloads last month: 80

Safetensors

Model size

27B params

Tensor type

BF16

MLX

Hardware compatibility

Quantized

Model tree for TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired

Base model

Qwen/Qwen3.6-27B

Quantized

prism-ml/Bonsai-27B-gguf

Finetuned

(4)

this model

Quantizations

1 model