我想把这个模型转换成 gguf 格式，然后导入到 ollama 中，测试下效果，转换时候出错，请求帮助。
使用 llama.cpp 的 convert.py 量化模型出错，什么原因啊？请求帮助。在线等，急急急。
MiniCPM-Llama3-V 2.5 can run with llama.cpp now! See our fork of llama.cpp for more detail.
and here is the MiniCPM-Llama3-V-2_5-gguf
https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5-gguf
@changingshow
Try to import GGUF format model directly into ollama, it can not recognize the image, I don't know what is the problem
Hi! Thanks for the hardwork! I tried to create a Modelfile in ollama as followed:
FROM ./ggml-model-Q4_K_M.gguf
FROM ./mmproj-model-f16.gguf
TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>
{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>
{{ .Response }}<|eot_id|>"""
TEMPLATE "{{ .System }} USER: {{ .Prompt }} ASSISTANT:"
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"
PARAMETER stop "<|reserved_special_token"
If I remove the vision projector, the q4 gguf model works! Ollama core dump though if I combine the projector model. I see that you have forked llama.cpp. It seems that you're providing your own version of CLIP implementation/format for the visual encoder?
https://github.com/OpenBMB/ollama
@y22ma
Here is ollama fork for minicpm-v 2.5
For CLIP, we do make our own adaptations because we have dynamic input for the image