LFM2-2.6B — LiteRT-LM

LiquidAI/LFM2-2.6B converted to the LiteRT-LM (.litertlm) format for Google's LiteRT-LM runtime. Tested on litert-lm 0.18.0 on a Mac (checks, evaluations, benchmarks and the commands below); the phone rows below ran the prebuilt Android binary named there. On litert-lm 0.17.1 the int8 file answered "What is the capital of France?" as if no question had been asked (one CPU run; the int4 file was not tried on 0.17.1).

Liquid AI suggests LFM2 for agentic tasks, data extraction, RAG, creative writing and multi-turn conversations, not for knowledge-intensive tasks or programming, and recommends fine-tuning it on narrow use cases. LFM2-2.6B has 2,569,272,320 parameters in 30 layers (22 convolution + 8 attention); Liquid AI lists English, Arabic, Chinese, French, German, Japanese, Korean and Spanish and states a 32,768-token context (4096 in these files). Per Liquid AI, it is the only LFM2 model that uses dynamic hybrid reasoning, a trace between <think> and </think> for complex or multilingual prompts.

File	Recipe	Size	sha256
LFM2-2.6B_int8.litertlm	int8 dynamic on linears + int8 embedding, convs float	2.72 GB	3dda920ce47e5ef2597c1a43732daeecc5d4b9b91fe29599cb9eaafb874b29e8
LFM2-2.6B_int4.litertlm	int4 blockwise-32 + OCTAV on linears, int8 lm_head + embedding	1.53 GB	01c77fbd925f593014636573b024c162c6b910c4c1372cea010d54955e6942f8

On GSM8K (n=100) the three scores are within two questions of each other: bf16 88 of 100 (transformers), int8 89, int4 87 (litert-lm). On IFEval-lite (n=120) they are within two prompts: 85, 85 and 87 of 120. The int4 file is 43.8 % smaller. At the rates in the tables below, a 1024-token prefill plus 256 decoded tokens at context 2048 took less time with int4 on the three GPUs measured (int4/int8: Mac M4 Max 1.98/2.45 s, Galaxy S26 10.70/13.47 s, Pixel 8a 22.11/25.07 s) and less time with int8 on the three CPUs (int8/int4: Mac 9.10/10.75 s, Galaxy S26 18.17/23.54 s, Pixel 8a 50.17/68.48 s). Not measured on iOS. The int8 file's one weights section is 2,722,843,088 bytes (2.54 GiB), over 2 GiB; the LFM2.5-2.6B card reports its int8 file, whose single weight section is 2.87 GB, running on an iPhone 17 Pro only when the app carries the extended-virtual-addressing and increased-memory entitlements.

Correctness

8-question check on the Mac M4 Max (the questions with "Answer briefly." appended, --temperature 0, one litert-lm run process each): int8 7/8 on CPU and GPU, int4 8/8 on CPU and 7/8 on GPU, bf16 (transformers) 7/8.

Both files' 12 signatures were fully delegated on the three GPUs measured. On both phones, both files answered "What is the capital of France? Answer in one sentence." on GPU and CPU: "The capital of France is Paris."

Accuracy

GSM8K: test questions 0–99, prompted for step-by-step reasoning and a final #### <number> line. IFEval-lite: the first 120 google/IFEval rows with mechanically checkable instructions, prompt-level strict. Greedy, at most 2048 output tokens; litert-lm on the Mac GPU, a fresh process per item. Scores are comparable only within this table, not with published IFEval scores or Liquid AI's GSM8K 82.41 (a different setup).

Configuration	GSM8K (n=100)	IFEval-lite (n=120)
bf16 reference (transformers, MPS)	88	85
int8, litert-lm	89	85
int4, litert-lm	87	87

Scores exclude the reasoning trace: no GSM8K reply had one, and the one IFEval-lite prompt that traced per row passed on its reply.

Usage

litert-lm run ./LFM2-2.6B_int8.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17."
litert-lm run ./LFM2-2.6B_int8.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17." --backend gpu --cache no
litert-lm benchmark ./LFM2-2.6B_int8.litertlm -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no --backend cpu

Both run commands, and the int4 file, listed the two dates with no chat marker such as <|im_end|> and no [thought] block. Without --backend, run uses the CPU. Without --cache, it writes a compiled-weights cache next to the model (on CPU 2.58 GB for int8, 1.51 GB for int4); --cache no writes none. For Android, see android-gpu.md.

The bundle's default sampler is temperature 0.1, top-k 50, top-p 1.0; Liquid AI recommends temperature 0.3, min_p 0.15 and repetition_penalty 1.05, and the bundle has no min_p or repetition penalty. run uses the bundle's sampler unless you pass --top-k, --top-p, --temperature or --seed; --top-k 1 gives greedy output.

The chat template is byte-identical to LiteRT-LM's models/lfm2/chat_template.jinja at commit 86a7e6d0, with that configuration's stops <|im_end|> and <|endoftext|> and start token <|startoftext|>. The bundle routes a trace to its thought channel, marked by <think> and </think>, apart from the reply; the CLI prints it between [thought] and [/thought]. Traces appeared for "What is the capital of France?" (int8, CPU and GPU) and 1 of 120 IFEval-lite prompts per row, none for GSM8K questions, checks, phone prompts or the commands above. --thinking-budget 0 and 32 did not suppress the trace on that prompt: on the GPU the int8 output was byte-identical to the run without the flag, and on the CPU each run still printed a [thought] block.

Tool calling is declared (model type lfm2): both files returned a parsed get_weather call (Python API, CPU, greedy) and, with automatic tool calling, used its result. Without tools, the template rendered the same text as Liquid AI's for the four conversations checked (a user turn, system + user, three turns, empty content); with tools it omits Liquid AI's <|tool_list_start|> / <|tool_list_end|> around the tool list and <|tool_response_start|> / <|tool_response_end|> around the tool result (wrapping it in [ … ] instead), and writes call arguments in double quotes where Liquid AI's template uses single quotes.

Measured on the int8 file (Python API, CPU, greedy; 3 runs on a shared Engine and 3 on fresh Engines; the two conversations had different system prompts and the same three questions): after an Engine served one conversation, the next conversation's two factual answers ("Paris"; "391" for 17 × 23) were the same in the 3 shared-Engine runs and the 3 fresh-Engine runs. The third answer, a self-description, varied in wording, as it did between the fresh runs. The open issue LiteRT-LM #3165 reports the runtime reusing the recurrent state across conversations that share prompt tokens; the probe's two conversations started with different system prompts. Open a new Engine for each conversation.

Performance

Mac Studio M4 Max, litert-lm 0.18.0: the benchmark command above with --backend cpu or gpu (1024-token prefill, 256-token decode, context 2048), one warmup, then the mean of 3 iterations; GPU is WebGPU on Metal.

Mac M4 Max	Prefill 1024 (tok/s)	Decode 256 (tok/s)	TTFT (s)
int8 GPU	2466.00	125.63	0.4232
int4 GPU	2452.56	163.77	0.4236
int8 CPU	399.41	39.18	2.5896
int4 CPU	185.00	49.07	5.5556

Engine init (first iteration): GPU int8 2.6537 s, int4 3.1914 s; CPU int8 30.2006 s, int4 8.2622 s.

Phones, Android 16: Galaxy S26 (SM-S942Q, Qualcomm SM8850, Adreno 840, MemTotal 10.9 GiB) and Pixel 8a (Tensor G3, Mali-G715, MemTotal 7.4 GiB), the same workload through the prebuilt litert_lm_advanced_main (sha256 adac974b…, 40,382,152 bytes) published 2026-09-18 in LiteRT-LM's public binary bucket (binaries/latest/android_arm64/litert_lm/), with --benchmark=true --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --max_num_tokens=2048 --num_iterations=3 --report_peak_memory_footprint=true. Rows are the mean of iterations 2 and 3 (GPU: LITERT_CL). Cold init is the first process after the compiled-program caches were removed, warm init the benchmark process, peak the largest of the benchmark process's three Peak system ram usage lines.

Galaxy S26	Prefill 1024 (tok/s)	Decode 256 (tok/s)	TTFT (s)
int8 GPU	1122.5	20.39	0.965
int4 GPU	682.3	27.84	1.54
int8 CPU	322.6	17.07	3.26
int4 CPU	83.0	22.85	12.38

Init cold/warm and peak: GPU int8 9.05/5.28 s, 784.4 MB; GPU int4 9.99/6.45 s, 1030.3 MB; CPU int8 3.10/0.22 s, 4158.3 MB; CPU int4 5.76/4.71 s, 3108.7 MB. The clocks lowered during these runs: in the GPU runs the GPU clock ceiling fell from 1300 MHz to 902 MHz (int8) and 826 MHz (int4) inside the benchmark; in the CPU runs the CPU clock was lowered in 24 of 25 (int8) and 34 of 35 (int4) samples, taken about 2 s apart.

Pixel 8a	Prefill 1024 (tok/s)	Decode 256 (tok/s)	TTFT (s)
int8 GPU	504.4	11.11	2.12
int4 GPU	171.9	15.85	6.02
int8 CPU	87.3	6.66	11.93
int4 CPU	30.3	7.38	33.94

Init cold/warm and peak: GPU int8 116.2/12.55 s, 1654.3 MB; GPU int4 24.53/12.97 s, 1510.7 MB; CPU int8 12.24/0.73 s, 3647.4 MB; CPU int4 13.18/10.02 s, 2602.4 MB. In the CPU runs the CPU clock was lowered in 40 of 48 (int8) and 63 of 66 (int4) samples, taken about 3 s apart; the clock sampler reads only an Adreno GPU's clock, so no Pixel 8a GPU clock was recorded. In the int8 GPU benchmark process the lowest MemAvailable sample was 419,100 kB. Not run on the NPU.

Conversion notes

• Source and stack: LiquidAI/LFM2-2.6B at revision 36ed799f; litert-torch 0.9.4, litert-converter 0.4.0, ai-edge-quantizer 0.9.0, transformers 5.14.1, torch 2.13.0; packed by the litert-lm 0.18.0 CLI. lfm_work/convert_lfm2_26b.sh in hf-to-litertlm rebuilt both files, all four sections byte-identical (only the packer's uuid and timestamp differ).
• Order: unquantized export (prefill signatures 1024, 512, 256, 128, 64, 32, 16, 8, 4, 2 and 1, plus decode, over a 4096-position KV cache) → post-hoc int8 and int4 quantization → int4 zero-scale fix (746,432 zero block scales set to their tensor's smallest nonzero scale, block payload zeroed; every scale normal on re-scan) → metadata repack with LiteRT-LM's lfm2 configuration.
• Template: Liquid AI's chat template fails at render on litert-lm 0.18.0 (map has no method named get), so the bundle carries LiteRT-LM's lfm2 template, which reads message content as a list of parts.

License and changes

Distributed under the LFM Open License v1.0 inherited from the base model (LICENSE); its section 5 conditions commercial use on an annual revenue below 10 million US dollars. Changes from the original work: exported to the LiteRT-LM format and quantized to int8 and int4 as described above; chat template and stop tokens taken from LiteRT-LM's models/lfm2 configuration, with max_num_tokens changed from 32768 to 4096. This conversion was made and is published independently of Liquid AI.