need vllm support and float16 or bf16
#2
by ZhuJunling - opened
vllm support?
float16 or bf16 data type?
We uploaded the bfloat16 version of the model in the bfloat16 branch. We are currently working hard on this and expect to add vLLM support in the next 1-2 weeks.
springxchen changed discussion status to closed