Unsloth implementation
Is it faster than original model? What was unsloth optimization here
We had to change the modeling files to make it compatible with the latest transformers!
Notebook: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Deepseek_OCR_2_(3B).ipynb
installing the latest vllm nightly as per the tutorial (Version: 0.15.0rc2.dev54+g6bf3b46d7) throws an error because the model arch isn't registered. Value error, Model architectures ['DeepseekOCR2ForCausalLM'] are not supported for now. Is there a way around this?
installing the latest vllm nightly as per the tutorial (Version: 0.15.0rc2.dev54+g6bf3b46d7) throws an error because the model arch isn't registered. Value error, Model architectures ['DeepseekOCR2ForCausalLM'] are not supported for now. Is there a way around this?
vLLM doesn't support it as of late apologies