Hugging Face's logo

MiniMaxAI
/
MiniMax-Text-01

Text Generation
Safetensors
minimax_text_01
conversational
custom_code
Model card Files
xet
 Community
40

Can you provide a FP8 version?

#11
by xjpang85 - opened
Discussion
xjpang85

Can you provide a FP8 version for less GPUs.
MiniMax-AI
 MiniMax org

Can the int8 or int4 version meet the requirements?
21world

int4 ok
zzzsssqqq
 MiniMax org

We have submitted a 13454 to vllm, and the inference performance improvement compared to the Hugging Face's implementation is very significant. You can give it a try.
sriting
 MiniMax org

Hi @xjpang85 , our vllm support has been merged today. Feel free to use it with our Text-01 model
https://github.com/vllm-project/vllm/pull/13454

MiniMax-AI changed discussion status to closed

Sign up or log in to comment