Instructions to use Nichonauta/LFM2.5-1.2B-Thinking-ToMoE with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Nichonauta/LFM2.5-1.2B-Thinking-ToMoE with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Nichonauta/LFM2.5-1.2B-Thinking-ToMoE", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Nichonauta/LFM2.5-1.2B-Thinking-ToMoE", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("Nichonauta/LFM2.5-1.2B-Thinking-ToMoE", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Nichonauta/LFM2.5-1.2B-Thinking-ToMoE with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Nichonauta/LFM2.5-1.2B-Thinking-ToMoE"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nichonauta/LFM2.5-1.2B-Thinking-ToMoE",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Nichonauta/LFM2.5-1.2B-Thinking-ToMoE

SGLang

How to use Nichonauta/LFM2.5-1.2B-Thinking-ToMoE with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Nichonauta/LFM2.5-1.2B-Thinking-ToMoE" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nichonauta/LFM2.5-1.2B-Thinking-ToMoE",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Nichonauta/LFM2.5-1.2B-Thinking-ToMoE" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Nichonauta/LFM2.5-1.2B-Thinking-ToMoE",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Nichonauta/LFM2.5-1.2B-Thinking-ToMoE with Docker Model Runner:
```
docker model run hf.co/Nichonauta/LFM2.5-1.2B-Thinking-ToMoE
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

LFM2.5-1.2B-Thinking-ToMoE

A Mixture-of-Experts (MoE) conversion of LiquidAI/LFM2.5-1.2B-Thinking, obtained with the ToMoE method (Dynamic Structural Pruning + Hypernetwork).

The dense 1.17B reasoning model is converted into a channel-MoE where:

feed_forward / MLP (16 layers): per-token expert routing over 8 channel-experts (top-1 via Gumbel-softmax), the FFN is cut to the union of the expert channel masks.
Full attention (6 layers): Q/K kept at full width; the V channels are dynamically masked per token (deterministic masks).
ShortConv (10 layers): statically pruned (a single channel index shared by the B/C/x projections, the conv and the output projection).

Metrics

Metric	Dense base	ToMoE MoE
Total params	1170.3M	1031.9M stored
Active params per token	1170.3M	~700M (est.)
PPL wikitext-2 (raw, 1810 tok)	16.9	504.3
PPL chat-formatted (prompt + answer)	71.1	711.7
Embeddings (always active)	134.2M	134.2M

The pruned budget is p = 0.05 with the ToMoE regularization (lam 64→128, Gumbel base annealed 1.0→−0.25, 8 experts), trained 2,400 steps at seq 2048 with online KD against the dense teacher on wikitext-103 (4M tokens, raw text). The attention QK is excluded from the budget (kept full) because the small head-dim (64) collapses under aggressive QK pruning.

Notes:

LFM2.5-1.2B-Thinking is a reasoning (RL) model: its native output format includes a <think> block before the answer. Unlike the 350M instruct model, its raw-text distribution is NOT degenerate (raw PPL 16.9 — the RL training preserved base-LM quality), so raw-text KD works well for this conversion.

The MoE keeps the whole base distribution (it can still emit <think> blocks) but is strongly degraded: raw PPL ~30x and chat PPL ~10x worse than the dense base. Generation repeats — this is the known structural limit of the ToMoE conversion on LFM2.

Usage (transformers, trust_remote_code)

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "Nichonauta/LFM2.5-1.2B-Thinking-ToMoE",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="cuda:0",
    attn_implementation="eager",
)
tokenizer = AutoTokenizer.from_pretrained("Nichonauta/LFM2.5-1.2B-Thinking-ToMoE")

messages = [{"role": "user", "content": "What is the capital of France?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_tensors="pt")
out = model.generate(**prompt, max_new_tokens=64)
print(tokenizer.decode(out[0], skip_special_tokens=True))

Note: this model requires trust_remote_code=True — the custom modeling_lfm2_moe_final.py defines the Lfm2MoEForCausalLM class.

Conversion details

Base: LiquidAI/LFM2.5-1.2B-Thinking (LFM Open License v1.0, lfm1.0)
Method: ToMoE — a bidirectional-GRU hypernetwork generates differentiable channel masks per sublayer; the model weights are frozen and only the hypernetwork is trained (online KD against the dense teacher). The final pruned weights are extracted from the trained masks.
Training: 2,400 steps, seq 2048, online knowledge distillation (2× forward KL), gradient checkpointing, wikitext-103 (4M tokens).
Architecture deltas: 16 layers (10 ShortConv + 6 full attention), FFN 8192-wide per layer (auto-adjusted from 12288), hidden 2048, head_dim 64, tied embeddings.

Limitations

The model is significantly degraded versus the dense base (chat PPL ~712 vs ~71; raw ~504 vs ~17) and generation tends to repeat — the conv layers are cut to ~5–40% width and each token activates ~50–70% of the MLP channels through a single expert.
This is a research artifact demonstrating the ToMoE pipeline on a mid-size LFM2 architecture; it is NOT a usable chat assistant.

License

This model is a derivative of LiquidAI/LFM2.5-1.2B-Thinking and is released under the LFM Open License v1.0 (see LICENSE).

Files

model.safetensors — the pruned MoE weights (fp32)
modeling_lfm2_moe_final.py — the custom model definition (trust_remote_code)
config.json — model configuration (with auto_map)
tokenizer.json, tokenizer_config.json, chat_template.jinja, generation_config.json