How is the consolidated.safetensors model file generated?
#44
by deanD - opened
May I ask how to convert the model files that have been fine-tuned using PEFT into the consolidated.safetensors required for VLLM deployment?
this does not exist publicly but the idea is to make the revert the operation did here
https://github.com/huggingface/transformers/blob/main/src/transformers/models/voxtral/convert_voxtral_weights_to_hf.py
we plan in the future to make it easier to switch between the two formats.
juliendenize changed discussion status to closed