Is this the standard version or the tiny version?
Hi Krish, thanks for quantizing and sharing the model publicly. This is something that I was also searching from a long time. I am just curious to know if this is the deepseek-vl2 base model or the deepseek-vl2-tiny variant?
Hi, this is tiny. I will release the base and small models soon. Are you able to run this model from your side in HF or vLLM?
Hi, this is tiny. I will release the base and small models soon. Are you able to run this model from your side in HF or vLLM?
Sure, please release them sooner. I haven't ran this one yet as I wasn't sure about the model version. I was looking for a 8-bit quantized version of the base model. Will give this one a try too!
I ran it, but I got an error in vLLM. Could you provide a code sample to run it?