Instructions to use hf-internal-testing/tiny-random-Mistral4-fp8-nvfp4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use hf-internal-testing/tiny-random-Mistral4-fp8-nvfp4 with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("hf-internal-testing/tiny-random-Mistral4-fp8-nvfp4", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
2 days ago
-
make.py: FP4 table now lives in mxfp4
Verified signature1236014 -
experts fp8 with 128x128 blocks (grouped kernel), attention 32x32
Verified signature3a23a8f -
every fp8 dim spans at least 2 blocks (grouped kernel)
Verified signaturebc77947 -
fp8 blocks 32x32 (the triton kernel needs K >= 32)
Verified signature4ec61d8 -
tiny Mistral4 with fp8 + nvfp4 layers
Verified signatureac53c7a -
initial commit
Verified signature48fba84