Instructions to use LiquidAI/LFM2-2.6B-Transcript-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use LiquidAI/LFM2-2.6B-Transcript-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="LiquidAI/LFM2-2.6B-Transcript-GGUF")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("LiquidAI/LFM2-2.6B-Transcript-GGUF", dtype="auto")

llama-cpp-python

How to use LiquidAI/LFM2-2.6B-Transcript-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="LiquidAI/LFM2-2.6B-Transcript-GGUF",
	filename="LFM2-2.6B-Transcript-BF16.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use LiquidAI/LFM2-2.6B-Transcript-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M

Use Docker

docker model run hf.co/LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use LiquidAI/LFM2-2.6B-Transcript-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "LiquidAI/LFM2-2.6B-Transcript-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM2-2.6B-Transcript-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M

SGLang

How to use LiquidAI/LFM2-2.6B-Transcript-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "LiquidAI/LFM2-2.6B-Transcript-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM2-2.6B-Transcript-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "LiquidAI/LFM2-2.6B-Transcript-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "LiquidAI/LFM2-2.6B-Transcript-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use LiquidAI/LFM2-2.6B-Transcript-GGUF with Ollama:
```
ollama run hf.co/LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M
```

Unsloth Studio new

How to use LiquidAI/LFM2-2.6B-Transcript-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for LiquidAI/LFM2-2.6B-Transcript-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for LiquidAI/LFM2-2.6B-Transcript-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for LiquidAI/LFM2-2.6B-Transcript-GGUF to start chatting

Pi new

How to use LiquidAI/LFM2-2.6B-Transcript-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use LiquidAI/LFM2-2.6B-Transcript-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M

Run Hermes

hermes

Docker Model Runner
How to use LiquidAI/LFM2-2.6B-Transcript-GGUF with Docker Model Runner:
```
docker model run hf.co/LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M
```

Lemonade

How to use LiquidAI/LFM2-2.6B-Transcript-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull LiquidAI/LFM2-2.6B-Transcript-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.LFM2-2.6B-Transcript-GGUF-Q4_K_M

List all available models

lemonade list

yungisimon commited on Jan 7

Commit

3bbeb52

verified ·

1 Parent(s): e7cce64

Update README.md

Browse files

Files changed (1) hide show

README.md +1 -84

README.md CHANGED Viewed

@@ -36,67 +36,7 @@ Based on [LFM2-2.6B](https://huggingface.co/LiquidAI/LFM2-2.6B), LFM2-2.6B-Trans
 - Fast summaries in seconds, not minutes
 - Runs fully locally across CPU, GPU, and NPU
-You can find more information about other task-specific models in this [blog post](https://www.liquid.ai/blog/introducing-liquid-nanos-frontier-grade-performance-on-everyday-devices).
-## 📄 Model details
-**Capabilities**: The model is trained for long-form transcript summarization (30-60 minute meetings), producing clear, structured outputs including key points, decisions, and action items with consistent tone and formatting.
-**Use cases**:
-- Internal team meetings
-- Sales calls and customer conversations
-- Board meetings and executive briefings
-- Regulated or sensitive environments where data can't leave the device
-- Offline or low-connectivity workflows
-**Generation parameters**: We strongly recommend using a lower temperature with a `temperature=0.3`.
-**Supported language**: English
-> [!WARNING]
-> ⚠️ The model is intended for single-turn conversations with a specific format, described in the following.
-**Input format**: We recommend using the following system prompt:
-> You are an expert meeting analyst. Analyze the transcript carefully and provide clear, accurate information based on the content.
-We use a specific formatting for the input meeting transcripts to summarize as follows:
-```
-<user_prompt>
-Title (example: Claims Processing training module)
-Date (example: July 2, 2021)
-Time (example: 1:00 PM)
-Duration (example: 45 minutes)
-Participants (example: Julie Franco (Training Facilitator), Amanda Newman (Subject Matter Expert))
-----------
-**Speaker 1**: Message 1 (example: **Julie Franco**: Good morning, everyone. Thanks for joining me today.)
-**Speaker 2**: Message 2 (example: **Amanda Newman**: Good morning, Julie. Happy to be here.)
-etc.
-```
-You can replace `<user_prompt>` with the following, depending on the desired summary type:
-| Summary type | User prompt |
-|--------------|-------------|
-| Executive summary | Provide a brief executive summary (2-3 sentences) of the key outcomes and decisions from this transcript. |
-| Detailed summary | Provide a detailed summary of the transcript, covering all major topics, discussions, and outcomes in paragraph form. |
-| Action items | List the specific action items that were assigned during this meeting. Include who is responsible for each item when mentioned. |
-| Key decisions | List the key decisions that were made during this meeting. Focus on concrete decisions and outcomes. |
-| Participants | List the participants mentioned in this transcript. Include their roles or titles when available. |
-| Topics discussed | List the main topics and subjects that were discussed in this meeting. |
-This is freeform, and you can add several prompts or combine them into a single one, like in the following examples:
-| Title | Input meeting | Model output |
-|-------|---------------|--------------|
-| Budget planning | [Link](https://huggingface.co/LiquidAI/LFM2-2.6B-Transcript/resolve/main/examples/meeting1.txt) | [Link](https://huggingface.co/LiquidAI/LFM2-2.6B-Transcript/resolve/main/examples/output1.txt) |
-| Design review | [Link](https://huggingface.co/LiquidAI/LFM2-2.6B-Transcript/resolve/main/examples/meeting2.txt) | [Link](https://huggingface.co/LiquidAI/LFM2-2.6B-Transcript/resolve/main/examples/output2.txt) |
-| Coffee chat / social hour | [Link](https://huggingface.co/LiquidAI/LFM2-2.6B-Transcript/resolve/main/examples/meeting3.txt) | [Link](https://huggingface.co/LiquidAI/LFM2-2.6B-Transcript/resolve/main/examples/output3.txt) |
-| Procurement / vendor review | [Link](https://huggingface.co/LiquidAI/LFM2-2.6B-Transcript/resolve/main/examples/meeting4.txt) | [Link](https://huggingface.co/LiquidAI/LFM2-2.6B-Transcript/resolve/main/examples/output4.txt) |
-| Task force meeting | [Link](https://huggingface.co/LiquidAI/LFM2-2.6B-Transcript/resolve/main/examples/meeting5.txt) | [Link](https://huggingface.co/LiquidAI/LFM2-2.6B-Transcript/resolve/main/examples/output5.txt) |
 ## 🏃 How to run
@@ -106,29 +46,6 @@ Example usage with [llama.cpp](https://github.com/ggml-org/llama.cpp):
 llama-cli -hf LiquidAI/LFM2-2.6B-Transcript-GGUF
 ```
-## 📈 Performance
-### Quality
-LFM2-2.6B-Transcript was benchmarked using the [GAIA Eval-Judge](https://github.com/amd/gaia/blob/main/docs/eval.md) framework on synthetic meeting transcripts across 8 meeting types.
-![2.6B-AMD Summarization Judge Score](https://cdn-uploads.huggingface.co/production/uploads/646fdf0a850a938d6c555b2a/e1nbAtmUWIg10Zb3tGMF-.png)
-*Accuracy ratings from [GAIA LLM Judge](https://github.com/amd/gaia). Evaluated on 24 synthetic 1K transcripts and 32 synthetic 10K transcripts. Claude Sonnet 4 used for content generation and judging.*
-### Inference Speed
-![2.6B-Transcript - Time](https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/L4Mh5cP3kX6BVepz-S1nu.png)
-*Generated using [llama-bench.exe](https://github.com/ggml-org/llama.cpp) b7250 on an HP Z2 Mini G1a Next Gen AI Desktop Workstation on respective AMD Ryzen devices.  We compute peak memory used during CPU inference by measuring peak memory usage of the llama-bench.exe process executing the command: `llama-bench -m <MODEL> -p 10000 -n 1000 -t 8 -r 3 -ngl 0` The llama-bench executable outputs the average inference times for preprocessing and token generation. The reported inference times are for the iGPU, enabled using the `-ngl 99` flag.*
-### Memory Usage
-![2.6B-Transcript- RAM](https://cdn-uploads.huggingface.co/production/uploads/61b8e2ba285851687028d395/XksTBvOkZ0Xx9bBD60LyQ.png)
-*Generated using [llama-bench.exe](https://github.com/ggml-org/llama.cpp) b7250 on an HP Z2 Mini G1a Next Gen AI Desktop Workstation with an AMD Ryzen AI Max+ PRO 395 processor.  We compute peak memory used during CPU inference by measuring peak memory usage of the llama-bench.exe process executing the command: `llama-bench -m <MODEL> -p 10000 -n 1000 -t 8 -r 3 -ngl 0` The llama-bench executable outputs the average inference times for preprocessing and token generation. The reported inference times are for the iGPU, enabled using the `-ngl 99` flag*
 ## 📬 Contact
 If you are interested in custom solutions with edge deployment, please contact [our sales team](https://www.liquid.ai/contact).

 - Fast summaries in seconds, not minutes
 - Runs fully locally across CPU, GPU, and NPU
+You can find more information about this model [here](https://huggingface.co/LiquidAI/LFM2-2.6B-Transcript).
 ## 🏃 How to run
 llama-cli -hf LiquidAI/LFM2-2.6B-Transcript-GGUF
 ```
 ## 📬 Contact
 If you are interested in custom solutions with edge deployment, please contact [our sales team](https://www.liquid.ai/contact).