README.md

---
license: other
license_name: fastino-community-license
license_link: https://huggingface.co/BoldingBuilds/Wildcat/blob/main/LICENSES/FASTINO-LICENSE.txt
base_model: vllm-sr/d3-ultra
pipeline_tag: zero-shot-classification
library_name: transformers
tags:
- decision-model
- system-one
- classification
- 3-bit
- hqq
- lora
---

# Wildcat

**Wildcat is d3-ultra squeezed to 3 bits, then repaired.** It answers the same decision questions as
[vllm-sr/d3-ultra](https://huggingface.co/vllm-sr/d3-ultra) (pick one option, yes / no, rate on a scale) and
returns a probability for every answer. It loads in **about 13 GB of GPU memory** and runs on a single 24 GB card
(tested on an RTX 3090). d3-ultra in bf16 needs about 54 GB.

| | |
| --- | --- |
| **Base** | d3-ultra (27B, Qwen3.5 family), Apache-2.0 |
| **Weights** | 400 text Linears in 3-bit HQQ (group 64), packed; everything else bf16 as in d3-ultra |
| **Repair** | Two LoRA adapters (rank 32, two training seeds) averaged, applied on top of the 3-bit weights |
| **GPU memory** | 13.19 GiB loaded; runs on 24 GB cards |
| **Download** | 13.6 GiB |
| **License** | Fastino Community License (see below). Built with Fastino GLiDE. |

## How it was made

1. **Squeeze.** d3-ultra's text-layer weights are quantized to 3 bits with HQQ (group size 64). The vision tower,
   embeddings, norms and the small `in_proj_a` / `in_proj_b` projections stay in bf16. The 3-bit codes are stored
   packed (a 2-bit plane plus a 1-bit plane); a small Triton kernel unpacks each layer just before its matmul, bit
   for bit equal to HQQ's own dequantize.
2. **Repair.** A LoRA is trained on top of the frozen 3-bit model to match the answer probabilities of three
   teachers: d3-ultra, [Jeffan](https://huggingface.co/tattrongvu) and [Fastino GLiDE](https://huggingface.co/fastino/GLiDE).
   The questions come only from public training splits of permissively licensed datasets, checked against the
   Decision Index for leaks. No Decision Index data was used for training.
3. **Blend.** Two runs with different seeds are averaged (each adapter at half strength).

## Results

**Decision Index 0.3 public index: 70.66** (complete run, all 140,620 requests, one RTX PRO 6000). d3-ultra's public
index on the board is 71.57, so Wildcat is **0.9 points below the model it was made from**, at a quarter of the memory.
The board's own ranking uses its private tests, which we have not seen.

| Area (public index, skill) | Wildcat | d3-ultra (board) |
| --- | --- | --- |
| Knowledge & reasoning | 58.9 | 62.3 |
| Language understanding | 73.7 | 72.4 |
| Retrieval & classification | 71.9 | 71.8 |
| Tools & automation | 87.2 | 87.8 |
| Arts & human taste | 60.7 | 63.5 |
| **Index** | **70.66** | **71.57** |

Where the 3-bit squeeze shows: multi-step reasoning and hard knowledge (MuSR, GPQA Diamond, CRUXEval, MMLU-Pro,
GSM8K are 3 to 10 points lower). The repair LoRA gains on language tasks (iSarcasmEval +8.7, FinEntity +2.7).

Speed over the full public run on one RTX PRO 6000 (in-process, one request at a time): median 157 ms, p80 351 ms
per request. The mean is 1,258 ms because about 1,000 very long ToolRet requests take minutes each; without ToolRet
the mean is 438 ms. On 750 sampled rows timed the same way, Wildcat and d3-ultra were level (mean 818 vs 932 ms).

On cards under 40 GB the engine answers one question at a time (batch size 1) so that requests with many long
questions fit; on cards over 72 GB it unpacks to bf16 and merges the LoRA for speed. Very long requests can fall
back to a slower path that still answers them.

## What was not tested

- **Box and Track** (image and video locations). d3-ultra supports them; Wildcat was only measured on choice,
  yes / no and score questions. Expect them to be weaker than d3-ultra's.
- AMD GPUs. The packed kernel is Triton and was only run on NVIDIA (CUDA).

## Run it

Needs an NVIDIA GPU with 24 GB or more, and the packages in `requirements.txt` plus `triton` and `accelerate`.

```bash
hf download BoldingBuilds/Wildcat --local-dir wildcat
```

With the Decision Index kit (this is how the board runs it):

```bash
PYTHONPATH=wildcat python -m decision_index run --engine d3_q3:D3Q3Engine --option model=wildcat \
  --rows <rows.jsonl.gz> --out <run dir>
```

The LoRA in `lora.safetensors` attaches on load. Nothing else needs passing.

## Files

| File | What it is |
| --- | --- |
| `q3.safetensors` | the packed 3-bit Linears (`<module>.lo / .hi / .scale / .zero`) |
| `model-rest.safetensors` | every other backbone tensor, bf16 |
| `lora.safetensors` | the repair LoRA, both seeds stacked and scaled (`<module>.A / .B`) |
| `readout.safetensors`, tokenizer, configs, `d3_*.py` | from d3-ultra, unchanged |
| `d3_q3.py` | the 3-bit engine and loader |
| `MODEL_MANIFEST.json` | SHA-256 of every file; checked on load |

## License

Wildcat learned from Fastino GLiDE's answers, so it is a Derivative Work under the
[Fastino Community License](LICENSES/FASTINO-LICENSE.txt) and ships under it. In short:

- Free to use, change and share. Keep this license with any copy.
- **Commercial use:** show "Built with Fastino GLiDE" in the product's docs, about page or UI.
- **Companies with more than $10M yearly revenue** using it in production need a separate agreement with Fastino
  (hello@fastino.ai). Evaluation, testing, benchmarking and research are not production.

d3-ultra's weights, readout and runtime are Apache-2.0 (`LICENSE`, `NOTICE`). Qwen's Apache-2.0 license is in
`LICENSES/Qwen3.8-27B-LICENSE`.

**Built with Fastino GLiDE.**

## Credits

- **d3-ultra** and its runtime: the vLLM Semantic Router team ([vllm-sr/d3-ultra](https://huggingface.co/vllm-sr/d3-ultra)). Wildcat is their model, made smaller.
- **Fastino GLiDE**: teacher ([fastino/GLiDE](https://huggingface.co/fastino/GLiDE)).
- **Jeffan** (tattrongvu): teacher.
- **Qwen** (Alibaba Cloud): the base model family.
- **HQQ** (Mobius Labs): the 3-bit quantizer.
- The **Decision Index** (multimodalart) for the benchmark and kit.
- The authors of every public dataset in the repair mix (below).

## Repair data

36,559 questions, all from **train splits** of public datasets (or generated here), each checked against the
Decision Index public rows for overlapping text; overlapping rows were dropped. Several sources are the train
splits of benchmarks the Decision Index tests (for example BANKING77, CLINC150, GSM8K, WinoGrande, HellaSwag,
When2Call, Amazon ESCI, iSarcasmEval, New Yorker captions), so expect Wildcat to do relatively better there.

- Intent and topic: CLINC150, BANKING77, DBpedia-14, LEDGAR
- Reading, inference and paraphrase: SQuAD v2, BoolQ, SNLI, WANLI, VitaminC, PAWS, STS-B, PubMedQA, claim_stance + Perspectrum
- Commonsense and science: HellaSwag, WinoGrande, PIQA, Social IQa, CommonsenseQA, Cosmos QA, ARC, OpenBookQA, QASC, RuleTaker
- Maths and knowledge: GSM8K, AQuA-RAT, MATH (hendrycks), MMLU (train), MedMCQA
- Ratings and taste: Feedback Collection, HelpSteer2, WMT DA, GoEmotions, Civil Comments, New Yorker Caption Contest, iSarcasmEval
- Tools and search: When2Call, Amazon ESCI
- Generated here (MIT): chord, CRUX-style code tracing, BBH-style, CLadder-style and SQuAD-based hallucination questions

Each source keeps its own licence (CC-BY, CC-BY-SA, Apache-2.0, MIT, CC0, AFL-3.0); no non-commercial data.