Proposed colab doesn't give reliable results.
I have runned every code block in colab but I didn't get reliable results from the model. The model I'm using in this case 1.6B with full precision. Is the answer expected ? Can you guys help me with that ?
Hello! I encounter the same issue. To elaborate more, it concerns all LFM2-VL models (450M, 1.6B, 3B) executed as GGUF (Q4_0, Q8_0, F16) via llama-cpp-python.
Example Colab notebooks provided by LiquidAI generate poor results.
As a reference to reproduce, please directly start the notebook on LFM2-VL-1.6B-GGUF F16:
https://colab.research.google.com/#fileId=https://huggingface.co/LiquidAI/LFM2-VL-1.6B-GGUF.ipynb
This calls inference on the image with a liberty island in New York:
https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg
Output produced by the model seems random:
{'id': 'chatcmpl-2f80c740-fa34-44a7-8824-e9239e23a76d',
'object': 'chat.completion',
'created': 1765829307,
'model': '/root/.cache/huggingface/hub/models--LiquidAI--LFM2-VL-1.6B-GGUF/snapshots/9b0f98f2dfbb18e850e45cee5d1a022a1896476d/./LFM2-VL-1.6B-F16.gguf',
'choices': [{'index': 0,
'message': {'role': 'assistant',
'content': 'The image depicts a serene landscape with a winding river flowing through a lush green valley, surrounded by tall trees and a clear blue sky.'},
'logprobs': None,
'finish_reason': 'stop'}],
'usage': {'prompt_tokens': 17, 'completion_tokens': 29, 'total_tokens': 46}}
@tarek-liquid, any assistance on this would be greatly appreciated! Thanks a lot in advance.
The problem is most likely that llama-cpp-python wrapper does not load the mmproj (the vision projector):
https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama.__init__
Here are the only supported multi-modal models in the llama-cpp-python:
https://llama-cpp-python.readthedocs.io/en/latest/#multi-modal-models
They require a custom ChatHandler to load the mmproj file and the LFM-2 models are unfortunately not supported.
I created a ticket in github of llama-cpp-python for this:
https://github.com/abetlen/llama-cpp-python/issues/2105
It would be good to fix or remove the example LiquidAI notebooks for VL models because they are misleading:
https://colab.research.google.com/#fileId=https://huggingface.co/LiquidAI/LFM2-VL-1.6B-GGUF.ipynb