Wildcat (bf16)

The same model as Wildcat-d3-ultra-3bit, stored as plain bf16 weights in d3-ultra's exact layout. It loads with d3-ultra's own runtime, unchanged: no extra code, no extra packages.

Use this version on a card with room for the bf16 weights (about 54 GB). For a 24 GB card, use the 3-bit repository.

Run it

hf download BoldingBuilds/Wildcat-d3-ultra-bf16 --local-dir wildcat-bf16
PYTHONPATH=wildcat-bf16 python -m decision_index run --engine d3_engine:D3Engine --option model=wildcat-bf16

It is the same command as for vllm-sr/d3-ultra. Every runtime file here (d3_*.py, modeling_d3.py, pipeline_d3.py, requirements.txt) is byte-identical to d3-ultra's.

How it was made

The 3-bit package's engine (d3_q3.py), on any card over 72 GB, unpacks each 3-bit layer to bf16 and merges the LoRA into it when it loads. This repository is that step done once and saved:

  1. Each of the 400 packed Linears is unpacked with d3_q3.py's own kernel.
  2. The LoRA is merged with the same expression the engine uses, (w.float() + B.float() @ A.float()).to(bfloat16).
  3. Everything else (embeddings, norms, vision tower, readout) is copied unchanged and is d3-ultra's.
  4. The shards use d3-ultra's model.safetensors.index.json: same file names, tensor names, shapes and dtypes.

Source: BoldingBuilds/Wildcat-d3-ultra-3bit at revision 11ae6bf2aa45989826450632ac83385a8739034d.

Checks

  • Layout: all 11 shards have the same byte size as d3-ultra's; all 1,183 tensor names, shapes and dtypes match.
  • Unquantized tensors: 25 sampled tensors are byte-identical to d3-ultra's (revision 37fee019).
  • Quantized layers: 14 layers rebuilt independently in plain PyTorch (no Triton) with the merge in fp64: 157 of 975 million weights differ, each by one bf16 rounding step (max 0.00012). That is fp32-versus-fp64 rounding in the LoRA merge; across the whole model it is 4,124 of 24.3 billion weights.
  • Answers: 23 Decision Index 0.3 requests from 23 benchmarks (30 questions), run through d3_runtime on CPU, give the same answer as the published Wildcat run on all 30; the largest probability difference is 0.013.
  • d3_runtime.verify_package(..., "release") passes on all files.

Results

Decision Index 0.3 public index 70.66 (140,178 requests, complete), from the run of the 3-bit package on an RTX PRO 6000, which used this bf16 path: scores.json.

Training data, teachers and limitations are on the 3-bit model card.

Licence

d3-ultra's weights, readout and runtime are Apache-2.0. The merged repair LoRA was trained on labels from Fastino GLiDE, so the model is under the Fastino Community License (LICENSES/). Built with Fastino GLiDE.

Downloads last month
42
Safetensors
Model size
26B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for BoldingBuilds/Wildcat-d3-ultra-bf16

Base model

vllm-sr/d3-ultra
Finetuned
(2)
this model

Space using BoldingBuilds/Wildcat-d3-ultra-bf16 1