Hskyto/lfm2.5-2.6b-toolcall-mlx

LFM2.5-2.6B Tool-Call MLX (16-Bit Master)

This repository contains the unquantized 16-bit (Bfloat16) MLX version of Liquid AI's LFM2.5-2.6B, merged with a custom QLoRA adapter trained specifically for on-device tool and function calling on Apple Silicon and native iOS integration.

This is the master build — full precision, and the source from which the smaller quantized variants are produced. Use it on Mac / Apple Silicon for maximum fidelity, or as the base for your own quantization.

Model Details

• Base Model: LiquidAI/LFM2.5-2.6B
• Fine-Tuning: Custom QLoRA adapter trained for structured tool routing and argument normalization.
• Training Data: Salesforce/xlam-function-calling-60k
• Precision: 16-bit (Bfloat16 / master weights)
• Framework: Apple MLX (mlx-lm / mlx-swift-lm)
• Quantized siblings (for on-device): 8-bit (2.7 GB) · 4-bit (1.5 GB, fits 6 GB iPhones)

---

Capabilities & Strengths

• Argument Normalization: High accuracy transforming natural, casual user phrasing (e.g. "call Mom at 6pm") into precise programmatic arguments (e.g. time='18:00').
• Multi-Tool Orchestration: Capable of emitting parallel independent function calls in a single response.
• Full Fidelity: Retains the exact precision of the merged QLoRA training run before any quantization step.

---

Reasoning toggle (enable_thinking)

LFM2 forces an opening <think> block on every response. For latency-sensitive on-device tool calling, the 4-bit build ships a modified chat_template.jinja that adds an enable_thinking guard, letting you skip the chain-of-thought and go straight to the tool call. See that repo for details.

---

How to Use (Python)

You can load and test this model locally using the mlx-lm package:

from mlx_lm import load, generate

model, tokenizer = load("Hskyto/lfm2.5-2.6b-toolcall-mlx")

prompt = tokenizer.apply_chat_template(
    [
        {"role": "system", "content": "You are an assistant with local iOS tools."},
        {"role": "user", "content": "Remind me to call Mom at 6pm"},
    ],
    tokenize=False,
    add_generation_prompt=True,
)

response = generate(model, tokenizer, prompt=prompt, verbose=True)
print(response)