Are params.json and tokenizer.model.v7m1 missing?
#1
by
ticoneva
- opened
When loading the model with vLLM as instructed, the model loads but cannot serve, failing with the following error:
Error code: 400 - {'object': 'error', 'message': '', 'type': 'BadRequestError', 'param': None, 'code': 400}
If I add back the Mistral-specific options:
--config-format mistral \
--load-format mistral \
--tokenizer_mode mistral \
vLLm complains that params.json and the tokenizer are missing.
I tried downloading those two files from the original Pixtral Large repository and manually adding them into the HF snapshot directory. The model fails to load due to CUDA out of memory error, even with H100 NVL x 4.