Tiny random Mistral4, pre-quantized with a per-layer plan: fp8 (block 16x16) on attention and routed experts, nvfp4 on shared experts. dense_reference.pt holds the bf16 weights before quantization; make.py rebuilds it.
- Downloads last month
- 37
Inference Providers NEW
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("hf-internal-testing/tiny-random-Mistral4-fp8-nvfp4", device_map="auto")