🐨 KoarAI / LFM2.5-350M-Thinking-0002-GGUF

📌 Release Note: Model Code 0002 (Dedicated GGUF Release)

Model Code: 0002
This repository contains official GGUF quantizations of KoarAI/LFM2.5-350M-Thinking (Revision 0002).
Model 0002 was trained with 100% Full Parameter Fine-Tuning across 9 full epochs using a multi-teacher reasoning blend (Qwen 3.8 Max, GLM 5.2, Kimi K3, DeepSeek-V4-Pro Agentic, MMLU-Pro, AIME 2026).

🚀 Versioning Policy: Starting with this release, distinct repositories are created for each revision (...-0002-GGUF, etc.) to maintain reproducible checkpoints.

---

📦 Quantized Files

File	Precision	File Size	Description
LFM2.5-350M-Thinking-0002-f16.gguf	FP16	~678 MB	Full precision unquantized weights. Maximum fidelity.
LFM2.5-350M-Thinking-0002-Q8_0.gguf	Q8_0	~347 MB	8-bit quantization. Extremely fast with near-identical perplexity to FP16.

---

🌟 Overview

KoarAI/LFM2.5-350M-Thinking-0002-GGUF brings native Chain-of-Thought (CoT) reasoning to edge devices, CPUs, and local runtimes like llama.cpp, Ollama, and LM Studio.

• Ultra-Lightweight: Runs smoothly even on low-spec laptops, Raspberry Pi, and mobile devices.
• Native <think> CoT: Performs step-by-step reasoning before outputting final answers.
• Base Architecture: Liquid Foundation Model (LFM2.5).

---

💡 Native Thinking Example

<|im_start|>user
Solve: 32 + 32 - 42<|im_end|>
<|im_start|>assistant
<think>
1. Calculate 32 + 32 = 64.
2. Calculate 64 - 42 = 22.
</think>
\boxed{22}<|im_end|>

---

⚡ Quickstart

1. Run with llama.cpp

llama-cli -m LFM2.5-350M-Thinking-0002-Q8_0.gguf \
  -p "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nSolve: 32 + 32 - 42<|im_end|>\n<|im_start|>assistant\n<think>\n" \
  -n 512 --temp 0.6

2. Run with Ollama

Create a Modelfile:

FROM ./LFM2.5-350M-Thinking-0002-Q8_0.gguf
TEMPLATE """{{- range .Messages }}<|im_start|>{{ .Role }}
{{ .Content }}<|im_end|>
{{- end }}<|im_start|>assistant
<think>
"""
PARAMETER temperature 0.6
PARAMETER stop <|im_end|>

Then create and run:

ollama create lfm2-350m-thinking-0002 -f Modelfile
ollama run lfm2-350m-thinking-0002

---

🔗 Related Models

• Base FP16 Weights (HF): KoarAI/LFM2.5-350M-Thinking
• Revision 0001 GGUF Repo: KoarAI/LFM2.5-350M-Thinking-GGUF

---

🐨 Maintained by KoarAI Lab

Released for the open-source AI community by KoarAI.