# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModel
processor = AutoProcessor.from_pretrained("BoldingBuilds/Wildcat-Mini", trust_remote_code=True)
model = AutoModel.from_pretrained("BoldingBuilds/Wildcat-Mini", trust_remote_code=True, device_map="auto")Wildcat Mini
Wildcat Mini is Ternary Bonsai 2 27B taught to answer decision questions. Ask it to pick one option, answer yes / no, or rate something on a scale, and it returns a probability for every answer, in the same format as d3-ultra and Wildcat. Its language weights stay in Bonsai's own ternary form, about 2 bits per weight, so the model loads in 7.4 GiB of GPU memory.
| Language model | Ternary Bonsai 2 27B (PrismML, Apache-2.0; Qwen3.8-27B family), PQ2_0 weights kept packed |
| Decision head | d3's readout, tokenizer and runtime (Apache-2.0) |
| Training | One LoRA (rank 16) taught by d3, then d3-ultra and Fastino GLiDE |
| GPU memory | 7.4 GiB loaded |
| Download | about 7.5 GiB |
| License | Fastino Community License (see below). Built with Fastino GLiDE. |
How it was made
- Swap the language model. d3's package (code, tokenizer, the 255-code answer readout) with every language-model weight replaced by Ternary Bonsai 2 27B's. Both are Qwen3.8-27B shapes. On its own this scores 41.9 on our sample: the readout does not fit Bonsai's weights yet.
- Teach it. One LoRA is trained in rounds to match a teacher's answer probabilities: first vllm-sr/d3 on about 33,000 questions and then on the 36,559-question mix below, and last on 12,000 of those with d3-ultra's and Fastino GLiDE's answers combined. The questions come only from public training splits, checked against the Decision Index for leaks. No Decision Index data was used for training.
- Serve it ternary. At run time the 2-bit codes stay packed on the GPU; a small Triton kernel unpacks each layer just before its matmul. It scores the same as with the full 49 GB bf16 weights (60.23 vs 60.25 on our sample).
Results
Wildcat Mini has not been run on the full Decision Index or submitted to its board. All numbers below are from our own 2,000-row sample of the Decision Index public set, the same rows for every model. This sample reads about 2.7 points below the board's public score for d3-ultra, so compare rows with each other, not with the board.
| Model (same 2,000 public rows) | GPU memory | Index |
|---|---|---|
| d3-ultra (bf16) | about 54 GB | 68.84 |
| Wildcat (3-bit d3-ultra) | 13.2 GiB | 69.39 |
| Wildcat Mini (this model) | 7.4 GiB | 60.23 |
| Wildcat Mini, bf16 weights | about 49 GB | 60.25 |
| Ternary Bonsai 2 27B + d3 readout, not trained | 7.4 GiB | 41.91 |
In short: about 9 points below d3-ultra on our sample, at a seventh of its memory. If you have the memory for Wildcat, use Wildcat.
Speed. Mini is slower than d3-ultra, because every layer is unpacked from ternary on the fly. On 750 sampled public requests, one at a time:
| ms per request | median | p80 | mean |
|---|---|---|---|
| d3-ultra, RTX PRO 6000 | 168 | 1,067 | 932 |
| Wildcat Mini, RTX PRO 6000 | 173 | 1,878 | 1,393 |
| Wildcat Mini, RTX 3090 (2,000 sample rows) | 1,097 | 9,650 | 7,042 |
What was not tested
- Cards smaller than 24 GB. The weights take 7.4 GiB, but long requests need more memory on top. It was run on an RTX 3090 (24 GB) and an RTX PRO 6000 (96 GB) only.
- Images and video, Box and Track questions. d3's runtime supports them; Mini was only measured on choice, yes / no and score questions.
- AMD GPUs. The unpack kernel is Triton and was only run on NVIDIA (CUDA).
Run it
Needs an NVIDIA GPU, the packages in requirements.txt, plus triton and accelerate.
hf download BoldingBuilds/Wildcat-Mini --local-dir wildcat-mini
With the Decision Index kit:
PYTHONPATH=wildcat-mini python -m decision_index run --engine d3_tern:D3TernEngine --option model=wildcat-mini \
--rows <rows.jsonl.gz> --out <run dir>
The packed weights and the LoRA in adapter/ load on their own. Nothing else needs passing.
Files
| File | What it is |
|---|---|
mini-base.safetensors |
the packed ternary language weights (2-bit codes + one fp16 scale per 128 weights) plus the other backbone tensors |
adapter/ |
the LoRA (adapter.safetensors, adapter.json) |
readout.safetensors, tokenizer, configs, d3_*.py |
from d3, unchanged |
d3_tern.py |
the ternary engine and loader |
d3_q3.py |
shared helpers from Wildcat's engine (memory limits for long requests) |
Training data
The same public-train-split mix as Wildcat: intent and topic (CLINC150, BANKING77, DBpedia-14, LEDGAR); reading, inference and paraphrase (SQuAD v2, BoolQ, SNLI, WANLI, VitaminC, PAWS, STS-B, PubMedQA, claim_stance + Perspectrum); commonsense and science (HellaSwag, WinoGrande, PIQA, Social IQa, CommonsenseQA, Cosmos QA, ARC, OpenBookQA, QASC, RuleTaker); maths and knowledge (GSM8K, AQuA-RAT, MATH, MMLU train, MedMCQA); ratings and taste (Feedback Collection, HelpSteer2, WMT DA, GoEmotions, Civil Comments, New Yorker Caption Contest, iSarcasmEval); tools and search (When2Call, Amazon ESCI); and questions generated here (MIT). Several are train splits of benchmarks the Decision Index tests, so expect Mini to do relatively better there. Each source keeps its own licence; no non-commercial data.
License
Wildcat Mini learned from Fastino GLiDE's answers, so it is a Derivative Work under the Fastino Community License and ships under it. In short:
- Free to use, change and share. Keep this license with any copy.
- Commercial use: show "Built with Fastino GLiDE" in the product's docs, about page or UI.
- Companies with more than $10M yearly revenue using it in production need a separate agreement with Fastino (hello@fastino.ai). Evaluation, testing, benchmarking and research are not production.
Ternary Bonsai 2's weights, and d3's readout and runtime, are Apache-2.0 (LICENSE, NOTICE).
Built with Fastino GLiDE.
Credits
- Ternary Bonsai 2 27B: PrismML (prism-ml/Ternary-Bonsai-2-27B-gguf). Mini runs on their ternary weights.
- d3 and d3-ultra: the vLLM Semantic Router team (vllm-sr/d3, vllm-sr/d3-ultra). d3 supplies the readout, tokenizer and runtime; both were teachers.
- Fastino GLiDE: teacher (fastino/GLiDE).
- Qwen (Alibaba Cloud): the base model family.
- The Decision Index (multimodalart) for the benchmark and kit.
- The authors of every public dataset in the training mix (above).
- Downloads last month
- 20
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("zero-shot-classification", model="BoldingBuilds/Wildcat-Mini", trust_remote_code=True)