Transformers
#1
by EyRaG - opened
Is this model usable with the Transformers library ?
It may run on transformers but it will not leverage quantization for speedup. You should prioritize using it with vLLM.
alexmarques changed discussion status to closed