Instructions to use lewisdog/lfm2.5-230m-cogs-ingest with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lewisdog/lfm2.5-230m-cogs-ingest with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lewisdog/lfm2.5-230m-cogs-ingest")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lewisdog/lfm2.5-230m-cogs-ingest")
model = AutoModelForCausalLM.from_pretrained("lewisdog/lfm2.5-230m-cogs-ingest")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lewisdog/lfm2.5-230m-cogs-ingest with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lewisdog/lfm2.5-230m-cogs-ingest"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lewisdog/lfm2.5-230m-cogs-ingest",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/lewisdog/lfm2.5-230m-cogs-ingest

SGLang

How to use lewisdog/lfm2.5-230m-cogs-ingest with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lewisdog/lfm2.5-230m-cogs-ingest" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lewisdog/lfm2.5-230m-cogs-ingest",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lewisdog/lfm2.5-230m-cogs-ingest" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lewisdog/lfm2.5-230m-cogs-ingest",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use lewisdog/lfm2.5-230m-cogs-ingest with Docker Model Runner:
```
docker model run hf.co/lewisdog/lfm2.5-230m-cogs-ingest
```

lfm2.5-230m-cogs-ingest

A 230M ingest student for the Cogitarium wiki pipeline: distils one raw captured document into structured JSON (extract / suggest_links / contradiction / page_update).

Serving pins (important)

temperature 0, repeat_penalty 1.0 (NO penalty). A repeat penalty produces schema-valid but empty/degenerate JSON on extraction at this size — the extract task copies input tokens and a penalty starves it.
Cap max_new_tokens and/or constrain with a JSON grammar: the extract task can run away under pure greedy (well-formed but unterminated JSON). This is a decoding artifact, not a format defect — do NOT reach for a repeat penalty to fix it.

Sanity eval (5 samples / 4 task types, temp 0, rep 1.0)

quant	strict JSON / keys	note
F16 / Q8_0	4/5	`suggest_links`, `page_update`, `contradiction` perfect; `extract` may not terminate under greedy
Q4_K_M	3/5	quant cliff — `page_update` loses the `section_md` key

Recommended quant: Q8_0 (233 MB, 723 tok/s on GB10). Q4_K_M only where size dominates and page_update is not used. Training matched the Qwen3-1.7B token accuracy (0.752 vs 0.756) at 7x fewer params.

Base model & license

Fine-tuned from LiquidAI/LFM2.5-230M. Use is governed by the LFM Open License v1.0 (lfm1.0) — see the LICENSE in the base repo. This derivative complies with and inherits those terms; attribution to LiquidAI is retained above.

Provenance

LoRA SFT (TRL) on the Cogitarium distillation datasets, DGX Spark (GB10). Full methodology, loss curves, eval harnesses and per-quant results: see the project RESULTS.md. This is the "fast/small tier" of the Cogitarium model picker; the Qwen3-1.7B students remain the quality tier.

Downloads last month: 452

Safetensors

Model size

0.2B params

Tensor type

BF16

Model tree for lewisdog/lfm2.5-230m-cogs-ingest

Base model

LiquidAI/LFM2.5-230M-Base

Finetuned

LiquidAI/LFM2.5-230M

Finetuned

(21)

this model