Instructions to use SandLogicTechnologies/LFM2.5-2.6B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use SandLogicTechnologies/LFM2.5-2.6B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M
# Run inference directly in the terminal:
llama cli -hf SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M
# Run inference directly in the terminal:
llama cli -hf SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M
# Run inference directly in the terminal:
./llama-cli -hf SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M

Use Docker

docker model run hf.co/SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M

LM Studio
Jan

vLLM

How to use SandLogicTechnologies/LFM2.5-2.6B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SandLogicTechnologies/LFM2.5-2.6B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SandLogicTechnologies/LFM2.5-2.6B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M

Ollama
How to use SandLogicTechnologies/LFM2.5-2.6B-GGUF with Ollama:
```
ollama run hf.co/SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M
```

Unsloth Studio

How to use SandLogicTechnologies/LFM2.5-2.6B-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for SandLogicTechnologies/LFM2.5-2.6B-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for SandLogicTechnologies/LFM2.5-2.6B-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for SandLogicTechnologies/LFM2.5-2.6B-GGUF to start chatting

How to use SandLogicTechnologies/LFM2.5-2.6B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use SandLogicTechnologies/LFM2.5-2.6B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use SandLogicTechnologies/LFM2.5-2.6B-GGUF with Docker Model Runner:
```
docker model run hf.co/SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M
```

Lemonade

How to use SandLogicTechnologies/LFM2.5-2.6B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M

Run and chat with the model

lemonade run user.LFM2.5-2.6B-GGUF-IQ3_M

List all available models

lemonade list

Hermes Agent

How to use SandLogicTechnologies/LFM2.5-2.6B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default SandLogicTechnologies/LFM2.5-2.6B-GGUF:IQ3_M

Run Hermes

hermes

Atomic Chat

LFM2.5-2.6B

LFM2.5-2.6B is a compact general-purpose text model developed by Liquid AI as part of the LFM2.5 family of hybrid models designed for efficient on-device deployment. The model combines a lightweight architecture with agentic post-training, a 128K context window, and capabilities for tool use, instruction following, multi-step agentic workflows, data extraction, and retrieval-augmented generation (RAG).

This repository contains GGUF quantized variants of LFM2.5-2.6B optimized for efficient local inference using llama.cpp. The quantized formats substantially reduce the model's memory footprint while retaining its core language understanding and agentic capabilities.

LFM2.5-2.6B is a text-only model, not a vision-language or multimodal model. Its primary focus is efficient agentic language processing, particularly for applications where low latency, long context, tool interaction, and local execution are important.

Model Overview

Model Name: LFM2.5-2.6B
Base Model: LiquidAI/LFM2.5-2.6B
Architecture: Hybrid LFM2 architecture with short-convolution blocks and grouped-query attention
Parameter Count: 2.69 Billion Parameters
Modalities: Text
Context Length: 131,072 tokens
Vocabulary Size: 128,000
Languages: Multilingual
Developer: Liquid AI
License: LFM1.0

Quantization Formats

This repository provides GGUF quantized versions of LFM2.5-2.6B optimized for efficient local inference using llama.cpp.

IQ3_M

Size reduction of approx 76.14% (1.20 GB) compared to 16-bit (5.03 GB)
Aggressive 3-bit quantization designed for highly memory-efficient deployment of the 2.6B model
Suitable for edge AI, local agent workloads, and systems where available memory is limited
Enables practical execution of long-context language workloads with a substantially smaller model footprint
More demanding reasoning, tool-use sequences, and complex multi-step interactions may experience greater quality degradation than higher-precision variants

IQ4_NL

Size reduction of approx 70.38% (1.49 GB) compared to 16-bit (5.03 GB)
Advanced 4-bit non-linear quantization providing a stronger balance between model fidelity and memory reduction
Well suited for agentic workflows, tool interaction, RAG, structured information extraction, and long-context processing
Provides improved retention of language and reasoning behavior compared with more aggressive 3-bit compression
May require additional computational resources compared with the smaller IQ3_M variant

IQ4_XS

Size reduction of approx 71.77% (1.42 GB) compared to 16-bit (5.03 GB)
Compact 4-bit quantization offering a balanced deployment profile for local and edge inference
Suitable for conversational workloads, multi-step agent tasks, retrieval-based applications, and structured data extraction
Provides a practical compromise between model footprint and generation quality
Appropriate for deployments where memory efficiency and stable agentic behavior are both important

Training Background (Original Model)

LFM2.5-2.6B is built on the LFM2 hybrid architecture and was pretrained on approximately 34 trillion tokens. Its mid-training process extended the model's context capability to 128K tokens. The post-training pipeline subsequently transformed the base model into an agent-oriented model.

Pretraining

Approximately 34T training tokens used during pretraining
Hybrid architecture combining double-gated short-convolution blocks with grouped-query attention
Mid-training stage used to extend context capability to 128K tokens
Designed for efficient language processing and deployment across constrained hardware environments

Agentic Post-Training

The post-training process consists of multiple stages:

Supervised fine-tuning performed in two rounds
Per-domain teacher specialization
Multi-domain on-policy distillation
Agentic reinforcement learning using popular agentic harnesses

The agentic reinforcement-learning stage exposes the model to tools, system prompts, and interaction patterns used by agent environments, improving its ability to operate through multi-step tool-based workflows.

Key Capabilities

Agentic Task Execution Designed to work through multi-step tasks rather than limiting interaction to single-turn text generation.
Tool Use & Function Calling Supports tool definitions, function calls, tool-result interpretation, and final response generation within an agent workflow.
Long-Context Processing Supports a context length of up to 131,072 tokens, making it suitable for applications involving large inputs and extended interactions.
Data Extraction Suitable for extracting structured information from unstructured textual inputs.
Retrieval-Augmented Generation Recommended by Liquid AI for RAG-oriented applications where retrieved information must be incorporated into multi-step language workflows.
Instruction Following Post-training emphasizes reliable adherence to instructions across conversational and agentic workflows.
Multilingual Processing Supports 16 languages spanning European, Asian, and other language groups.
Efficient Local Inference The LFM2 architecture is specifically designed for efficient deployment across CPU, GPU, and other constrained hardware environments. Liquid AI reports 220 tokens/s on an Apple M5 Max and 113 tokens/s on an AMD Ryzen AI Max+ 395 for the original model configuration.

Recommended Usecases

Agentic AI Systems Build autonomous or semi-autonomous systems capable of performing multi-step tasks.
Tool-Calling Applications Integrate the model with external functions, APIs, databases, and application tools.
RAG Systems Build retrieval-augmented assistants capable of processing retrieved knowledge within long contexts.
Structured Data Extraction Extract organized information from large volumes of unstructured text.
Long-Context Applications Process large documents and extended conversational contexts within a 128K-token context window.
On-Device AI Deploy language intelligence locally on phones, laptops, edge systems, and other constrained devices.
Conversational Assistants Build lightweight assistants requiring low-latency local inference.

Limitations

LFM2.5-2.6B is primarily optimized for agentic workloads, tool use, data extraction, RAG, and long-context applications. The official model card specifically states that it is not recommended for agentic coding or knowledge-heavy tasks.

Usage Example

Using llama.cpp

./llama-cli \
  -m SandLogicTechnologies/LFM2.5-2.6B-IQ4_NL.gguf \
  -p "Analyze the provided information, identify the important entities, and return the results as structured JSON."

Acknowledgments

These quantized models are based on the original work by the Liquid AI development team.

Special thanks to:

The Liquid AI team for developing and releasing LFM2.5-2.6B.
The developers and open-source community behind llama.cpp for enabling efficient GGUF inference across CPU, GPU, and edge hardware.

Contact

For questions, feedback, or support, please reach out at support@sandlogic.com or visit https://www.sandlogic.com/

Downloads last month: -

GGUF

Model size

3B params

Architecture

lfm2

Hardware compatibility

3-bit

4-bit

Model tree for SandLogicTechnologies/LFM2.5-2.6B-GGUF

Base model

LiquidAI/LFM2.5-2.6B-Base

Finetuned

LiquidAI/LFM2.5-2.6B

Quantized

(54)

this model