API Try Docs Blog

LFM3-1.7B-GGUF

GGUF version of LFM3-1.7B for local inference on CPUs and GPUs.

Find architecture and usage details in the original model card.

Use with llama.cpp

---

Requires an LFM3-compatible llama.cpp build. Add a quantization tag to pick a file.

llama-cli -hf LiquidAI/LFM3-1.7B-GGUF:Q4_K_M \
  --temp 0.1 --top-k 50 --repeat-penalty 1.1

For math and science, use --temp 1.0 --top-p 0.95 --top-k 20 instead. More options in the llama.cpp guide.

QAD Q4_0 GGUF

---

The Q4_0 build in this repository is the Quantization-Aware Distillation (QAD) checkpoint, LFM3-1.7B-QAD-Q4_0.gguf. It was trained to recover the accuracy lost to Q4_0 quantization.

Example usage with llama.cpp:

llama-cli -hf LiquidAI/LFM3-1.7B-GGUF \
  --hf-file LFM3-1.7B-QAD-Q4_0.gguf \
  --temp 0.1 --top-k 50 --repeat-penalty 1.1