LiquidAI

/

LFM2-VL-1.6B-GGUF

like 56

Follow

Liquid AI 2.43k

Hello! I encounter the same issue. To elaborate more, it concerns all LFM2-VL models (450M, 1.6B, 3B) executed as GGUF (Q4_0, Q8_0, F16) via llama-cpp-python.

Example Colab notebooks provided by LiquidAI generate poor results.

As a reference to reproduce, please directly start the notebook on LFM2-VL-1.6B-GGUF F16:
https://colab.research.google.com/#fileId=https://huggingface.co/LiquidAI/LFM2-VL-1.6B-GGUF.ipynb

This calls inference on the image with a liberty island in New York:
https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg

Output produced by the model seems random:

{'id': 'chatcmpl-2f80c740-fa34-44a7-8824-e9239e23a76d',
 'object': 'chat.completion',
 'created': 1765829307,
 'model': '/root/.cache/huggingface/hub/models--LiquidAI--LFM2-VL-1.6B-GGUF/snapshots/9b0f98f2dfbb18e850e45cee5d1a022a1896476d/./LFM2-VL-1.6B-F16.gguf',
 'choices': [{'index': 0,
   'message': {'role': 'assistant',
    'content': 'The image depicts a serene landscape with a winding river flowing through a lush green valley, surrounded by tall trees and a clear blue sky.'},
   'logprobs': None,
   'finish_reason': 'stop'}],
 'usage': {'prompt_tokens': 17, 'completion_tokens': 29, 'total_tokens': 46}} 

@tarek-liquid , any assistance on this would be greatly appreciated! Thanks a lot in advance.

The problem is most likely that llama-cpp-python wrapper does not load the mmproj (the vision projector):
https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama.__init__

Here are the only supported multi-modal models in the llama-cpp-python:
https://llama-cpp-python.readthedocs.io/en/latest/#multi-modal-models

They require a custom ChatHandler to load the mmproj file and the LFM-2 models are unfortunately not supported.

I created a ticket in github of llama-cpp-python for this:
https://github.com/abetlen/llama-cpp-python/issues/2105

It would be good to fix or remove the example LiquidAI notebooks for VL models because they are misleading:
https://colab.research.google.com/#fileId=https://huggingface.co/LiquidAI/LFM2-VL-1.6B-GGUF.ipynb