How to use this model?
#1
by bingw5 - opened
I can see there is example code to run the model. But that's for original model. Do I need to modify any parameters or lines to run quantized model?
Ah, that example code is from the original model card.
The example code of how to run this model is very similar but pointed to this repo. See here: https://huggingface.co/failspy/InternVL-Chat-V1-5-8bit/blob/main/example_inference.py
Running that will be running the quantized model.
I ran into error:
TypeError(\"internvl_chat isn't supported yet.\")
Do you know what's the root cause?