API Try Docs Blog

LFM3-VL-2B-GGUF

GGUF version of LFM3-VL-2B for local inference on CPUs and GPUs.

Find architecture and usage details in the original model card.

Use with llama.cpp

---

Support for LFM3-VL in llama.cpp is in review in ggml-org/llama.cpp#31252, without video input for now. We will update this card when a llama.cpp release supports it. Add a quantization tag to pick a file. With -hf, llama.cpp also downloads the vision projector.

llama-cli -hf LiquidAI/LFM3-VL-2B-GGUF:Q4_0 \
  --temp 0.1 --top-p 1.0 --top-k 50 --min-p 0.0 --repeat-penalty 1.1 \
  --no-reasoning-preserve

Type /image <IMAGE PATH> in the chat to add an image. More options in the llama.cpp guide.

QAD Q4_0 GGUF

---

The Q4_0 build in this repository is the Quantization-Aware Distillation (QAD) checkpoint, LFM3-VL-2B-QAD-Q4_0.gguf. It was trained to recover the accuracy lost to Q4_0 quantization.

llama-cli -hf LiquidAI/LFM3-VL-2B-GGUF \
  --hf-file LFM3-VL-2B-QAD-Q4_0.gguf \
  --temp 0.1 --top-p 1.0 --top-k 50 --min-p 0.0 --repeat-penalty 1.1 \
  --no-reasoning-preserve