How to load the model?
#6
by oubahe - opened
Based on the vllm framework, how to load the model through code?
It loads weights automatically. Just run 'vllm serve xxx'.
Based on the vllm framework, how to load the model through code?
It loads weights automatically. Just run 'vllm serve xxx'.