Instructions to use hf-internal-testing/tiny-random-Mistral4-fp8-nvfp4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use hf-internal-testing/tiny-random-Mistral4-fp8-nvfp4 with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("hf-internal-testing/tiny-random-Mistral4-fp8-nvfp4", device_map="auto") - Notebooks
- Google Colab
- Kaggle
File size: 259 Bytes
ac53c7a | 1 2 3 4 5 6 | ---
library_name: transformers
---
Tiny random Mistral4, pre-quantized with a per-layer plan: fp8 (block 16x16) on attention and routed experts, nvfp4 on shared experts. `dense_reference.pt` holds the bf16 weights before quantization; `make.py` rebuilds it.
|