Instructions to use BoldingBuilds/Wildcat-d3-ultra-bf16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use BoldingBuilds/Wildcat-d3-ultra-bf16 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("zero-shot-classification", model="BoldingBuilds/Wildcat-d3-ultra-bf16", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModel processor = AutoProcessor.from_pretrained("BoldingBuilds/Wildcat-d3-ultra-bf16", trust_remote_code=True) model = AutoModel.from_pretrained("BoldingBuilds/Wildcat-d3-ultra-bf16", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Wildcat (bf16)
The same model as Wildcat-d3-ultra-3bit, stored as plain bf16 weights in d3-ultra's exact layout. It loads with d3-ultra's own runtime, unchanged: no extra code, no extra packages.
Use this version on a card with room for the bf16 weights (about 54 GB). For a 24 GB card, use the 3-bit repository.
Run it
hf download BoldingBuilds/Wildcat-d3-ultra-bf16 --local-dir wildcat-bf16
PYTHONPATH=wildcat-bf16 python -m decision_index run --engine d3_engine:D3Engine --option model=wildcat-bf16
It is the same command as for vllm-sr/d3-ultra. Every runtime file here
(d3_*.py, modeling_d3.py, pipeline_d3.py, requirements.txt) is byte-identical to d3-ultra's.
How it was made
The 3-bit package's engine (d3_q3.py), on any card over 72 GB, unpacks each 3-bit layer to bf16 and merges the LoRA
into it when it loads. This repository is that step done once and saved:
- Each of the 400 packed Linears is unpacked with
d3_q3.py's own kernel. - The LoRA is merged with the same expression the engine uses,
(w.float() + B.float() @ A.float()).to(bfloat16). - Everything else (embeddings, norms, vision tower, readout) is copied unchanged and is d3-ultra's.
- The shards use d3-ultra's
model.safetensors.index.json: same file names, tensor names, shapes and dtypes.
Source: BoldingBuilds/Wildcat-d3-ultra-3bit at revision 11ae6bf2aa45989826450632ac83385a8739034d.
Checks
- Layout: all 11 shards have the same byte size as d3-ultra's; all 1,183 tensor names, shapes and dtypes match.
- Unquantized tensors: 25 sampled tensors are byte-identical to d3-ultra's (revision
37fee019). - Quantized layers: 14 layers rebuilt independently in plain PyTorch (no Triton) with the merge in fp64: 157 of 975 million weights differ, each by one bf16 rounding step (max 0.00012). That is fp32-versus-fp64 rounding in the LoRA merge; across the whole model it is 4,124 of 24.3 billion weights.
- Answers: 23 Decision Index 0.3 requests from 23 benchmarks (30 questions), run through
d3_runtimeon CPU, give the same answer as the published Wildcat run on all 30; the largest probability difference is 0.013. d3_runtime.verify_package(..., "release")passes on all files.
Results
Decision Index 0.3 public index 70.66 (140,178 requests, complete), from the run of the 3-bit package on an RTX PRO 6000, which used this bf16 path: scores.json.
Training data, teachers and limitations are on the 3-bit model card.
Licence
d3-ultra's weights, readout and runtime are Apache-2.0. The merged repair LoRA was trained on labels from Fastino
GLiDE, so the model is under the Fastino Community License (LICENSES/). Built with Fastino GLiDE.
- Downloads last month
- 42
Model tree for BoldingBuilds/Wildcat-d3-ultra-bf16
Base model
vllm-sr/d3-ultra