How to use hf-internal-testing/tiny-random-Mistral4-fp8-nvfp4 with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("hf-internal-testing/tiny-random-Mistral4-fp8-nvfp4", device_map="auto")
ac53c7a 3a23a8f
1
2
3
4
version https://git-lfs.github.com/spec/v1 oid sha256:8ed4ddd3ffd4e13f7f80c8719657e752e695a9a71bc73201a4ba45ce075d33a8 size 4376696
