Instructions to use TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired

Run Hermes

hermes

OpenClaw new

How to use TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

MLX LM

How to use TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Bonsai-27B-MLX-BF16-Config-Repaired / docs /ARCHIVE_AND_COMPARISON_PLAN.md

TiGa-RCE

docs: add comparison matrix and archive policy

0d885c2 verified 5 days ago

preview code

Raw

History Blame Contribute Delete

3.64 kB

Bonsai Archive And Comparison Plan

Public Archives

Artifact	Hub repository	Status	Local retention after verification
MLX BF16 config-repaired baseline	`TiGa-RCE/Bonsai-27B-MLX-BF16-Config-Repaired`	Reproducible archival baseline, not locally runnable	Can be removed after Hub hash verification and redownloaded when a conversion run needs it
oQ2e S32 smoke	`TiGa-RCE/Bonsai-27B-oQ2e-S32-Smoke`	Public pipeline/calibration evidence, not a quality release	Can be removed after Hub hash verification; retain only if a local oQe regression test needs it

Do not publish the generic affine Q2 artifact as a model release. It is a negative control: it loaded but failed direct generation and deterministic screens. Keep its small logs and metadata; delete its 7.9 GB weights only after the user explicitly approves a cleanup list.

Upstream Models: Link, Do Not Mirror

Upstream artifact	Role	Necessary now?
`prism-ml/Bonsai-27B-gguf` Q1_0	Native 1-bit reference	Yes, already cached and used through Prism's runner
`prism-ml/Bonsai-27B-mlx-1bit`	Native MLX Q1 reference	Keep metadata; runtime integration remains a separate branch
`peppi314/Bonsai-27B-dSpark-MLX-4bit`	Speculative-decoding sidecar, not a standalone model	No additional copy or mirror needed
`prism-ml/Ternary-Bonsai-27B-gguf` Q2_0	Official retrained ternary quality-oriented comparison	Download later, only when the native Q1 versus ternary screen is scheduled
`prism-ml/Ternary-Bonsai-27B-mlx-2bit`	Official MLX ternary comparison	Download later, only after the compatible Prism MLX runtime is established

Controlled MLX Comparison Matrix

The next local derivatives should use the public BF16 archive as the sole source and must have the same tokenizer, prompts, fixed decoding settings, and evaluation scripts:

Variant	Generation route	Comparison purpose	Promotion gate
MLX Q4	Uniform affine MLX	Standard low-bit baseline	Structural load, direct generation, 100 HellaSwag + 100 ARC-Challenge
oQ4	oQe MLX	Tests calibration benefit over Q4	Same screens and a predeclared memory/latency measurement
MLX Q8	Uniform affine MLX	High-fidelity local reference when memory permits	Same screens; no claim that it is FP16
oQ8	oQe MLX	Tests whether oQe retains any benefit at a high bit-width	Same screens and runtime measurement
Bonsai Q1 native	Prism llama.cpp runtime	Binary operating point	Same semantic screens, throughput recorded separately by runtime
Ternary Bonsai Q2 native	Prism runner	Official retrained ternary operating point	Same semantic screens, clearly labeled as a different trained checkpoint

Important Fairness Boundaries

Q4 versus oQ4 and Q8 versus oQ8 are direct quantization-method comparisons.
Q1 versus official ternary is an operating-point comparison, not proof that one quantizer is better: the official ternary checkpoint has different trained ternary weights.
Accuracy and formatting screens can be reused across all compatible models. Throughput figures cannot be compared across oMLX and Prism llama.cpp without recording each runtime's context size, cache policy, decoding parameters, and hardware path.

Prerequisite Before New oQ Runs

Repair the imatrix collector so it checkpoints and resumes. The current full run stopped after 44 samples without preserving a reusable partial cache. Do not start oQ4 or oQ8 quality calibration until that collector has an atomic checkpoint and a resume test.