LFM2-700M — LiteRT-LM

LiquidAI/LFM2-700M converted to the LiteRT-LM (.litertlm) format for Google's LiteRT-LM runtime. Tested on litert-lm 0.18.0 on a Mac (checks, evaluations, benchmarks and the commands below); the phone rows below ran the prebuilt Android binary named there. On litert-lm 0.17.1 the plain run of the int8 file answered as if no question had been asked (the int4 file was not tried on 0.17.1).

Liquid AI suggests LFM2 for agentic tasks, data extraction, RAG, creative writing and multi-turn conversations, not for knowledge-intensive tasks or programming, and recommends fine-tuning it on narrow use cases. LFM2-700M has 742,489,344 parameters in 16 layers (10 convolution + 6 attention); Liquid AI lists English, Arabic, Chinese, French, German, Japanese, Korean and Spanish and states a 32,768-token context (4096 in these files).

File	Recipe	Size	sha256
LFM2-700M_int8.litertlm	int8 weights on linears, the 10 convolutions and the embedding (litert-torch's default export recipe dynamic_wi8_afp32), float activations	808.3 MB	ccc95007d868878ac749073cbc28b88097cb7d437fca3f9aec72f535cc7fd99b
LFM2-700M_int4.litertlm	int4 blockwise-32 + OCTAV on linears, int8 lm_head + embedding, convs float	480.8 MB	4ecfc71299a304e97e8f0ef5134085ecc4b5d414755b8c1709f6b552d20250b9

On IFEval-lite (n=120) bf16 passed 77 of 120 prompts, the int8 file 75 and 74 (two runs) and the int4 file 67; the int4 file is 40.5 % smaller. At the rates in the tables below, a 1024-token prefill plus 256 decoded tokens at context 2048 took less time with int4 on the Galaxy S26 GPU (3.92 vs 4.64 s) and the Mac M4 Max GPU (0.87 vs 0.97 s) and the same time with either file on the Pixel 8a GPU (8.07 s); on the CPUs, int8 took less time on the Mac (3.72 vs 4.18 s) and on the Galaxy S26 (6.76 vs 7.43 s; the int4 row is from a run that started on a hot phone, noted below), and int4 on the Pixel 8a (15.13 vs 16.21 s). Not measured on iOS.

Correctness

8-question check on the Mac M4 Max (the questions with "Answer briefly." appended, --temperature 0, one litert-lm run process each): int8 7/8 on CPU and GPU, int4 7/8 on CPU and GPU, bf16 (transformers) 7/8; the miss was the rhyme question in every case.

Both files' 12 signatures were fully delegated on the three GPUs measured. On both phones, both files answered "What is the capital of France? Answer in one sentence." on GPU and CPU: "The capital of France is Paris."

Accuracy

IFEval-lite: the first 120 google/IFEval rows with mechanically checkable instructions, prompt-level strict, with a 1400-token budget per prompt (bf16: at most 1400 new tokens). GSM8K: test questions 0–99, prompted for step-by-step reasoning and a final #### <number> line, in two wordings of that prompt, at most 1024 output tokens. Greedy; litert-lm on the Mac CPU, a fresh process per item. Scores are comparable only within this table, not with published IFEval scores or Liquid AI's IFEval 72.23 (a different setup).

Configuration	IFEval-lite (n=120)	GSM8K, prompt 1 (n=100)	GSM8K, prompt 2 (n=100)
bf16 reference (transformers, MPS)	77	59	59
int8, litert-lm	75 and 74 (two runs)	58	62
int4, litert-lm	67	52	57

Prompt 1 ends Solve this step by step. After your reasoning, write the final answer on its own line in the exact form: followed by #### <number> on the next line; prompt 2 ends Think step by step, then write the final answer on its own line as '#### <number>'. The same int8 file run twice on IFEval-lite flipped 5 prompts (3 passed only in the first run, 2 only in the second): greedy replies of this file differed between processes on the CPU path (24 prompts re-run gave byte-identical replies for 4).

Usage

litert-lm run ./LFM2-700M_int8.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17."
litert-lm run ./LFM2-700M_int8.litertlm --prompt "Extract the dates from: The meeting moved from May 3 to May 17." --backend gpu --cache no
litert-lm benchmark ./LFM2-700M_int8.litertlm -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no --backend cpu

Both run commands, and the int4 file, listed the two dates with no chat marker such as <|im_end|> and no thought block. Without --backend, run uses the CPU. Without --cache, it writes a compiled-weights cache next to the model (on CPU 747 MB for int8, 465 MB for int4); --cache no writes none. For Android, see android-gpu.md.

The bundle's default sampler is temperature 0.1, top-k 50, top-p 1.0; Liquid AI recommends temperature 0.3, min_p 0.15 and repetition_penalty 1.05, and the bundle has no min_p or repetition penalty. run uses the bundle's sampler unless you pass --top-k, --top-p, --temperature or --seed; --top-k 1 gives greedy output.

The chat template is byte-identical to LiteRT-LM's models/lfm2/chat_template.jinja at commit 86a7e6d0, and the stop set comes from the same configuration: stops <|im_end|> and <|endoftext|>, start token <|startoftext|>. The metadata marks it non-thinking (supports_thinking false) and keeps that configuration's thought channel (<think> / </think>); no reply in the checks and evaluations contained those strings.

Tool calling is declared (model type lfm2): both files returned a parsed get_weather call (Python API, CPU, greedy) and, with automatic tool calling, used its result. Without tools, the template rendered the same text as Liquid AI's for the four conversations checked (a user turn, system + user, three turns, empty content); with tools it omits Liquid AI's <|tool_list_start|> / <|tool_list_end|> around the tool list and <|tool_response_start|> / <|tool_response_end|> around the tool result (wrapping it in [ … ] instead), and writes call arguments in double quotes where Liquid AI's template uses single quotes.

Measured on the int8 file (Python API, CPU, greedy; 3 processes with a shared Engine and 3 with fresh Engines; the two conversations had different system prompts): after an Engine served one conversation, the next conversation's three answers ("Paris", "Blue", "Eight") and its token count were the same as on a fresh Engine in 3 of 3 processes. The open issue LiteRT-LM #3165 reports the runtime reusing the recurrent state across conversations that share prompt tokens; the probe's two conversations started with different system prompts. Open a new Engine for each conversation.

Performance

Mac M4 Max, litert-lm 0.18.0: the benchmark command above with --backend cpu or gpu (1024-token prefill, 256-token decode, context 2048), one warmup, then the mean of 3 iterations; GPU is WebGPU on Metal. The int8 GPU row is the one of three benchmark processes with the median time for the 1024-token prefill plus 256 decoded tokens (0.959, 0.969 and 0.966 s); the int4 GPU row is one benchmark process; each CPU row is the mean of two benchmark processes.

Mac M4 Max	Prefill 1024 (tok/s)	Decode 256 (tok/s)	TTFT (s)
int8 GPU	7870.47	306.34	0.1334
int4 GPU	7839.15	348.26	0.1335
int8 CPU	1134.25	90.87	0.9139
int4 CPU	580.71	105.97	1.7826

Engine init (first iteration): GPU int8 1.2774 s, int4 1.4619 s; CPU int8 9.5982 s, int4 2.3391 s.

Phones, Android 16: Galaxy S26 (SM-S942Q, Qualcomm SM8850, Adreno, MemTotal 10.9 GiB) and Pixel 8a (Tensor G3, Mali, MemTotal 7.4 GiB), the same workload through the prebuilt litert_lm_advanced_main (sha256 adac974b…, 40,382,152 bytes) published 2026-09-18 in LiteRT-LM's public binary bucket (binaries/latest/android_arm64/litert_lm/), with --benchmark=true --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --max_num_tokens=2048 --num_iterations=3 --report_peak_memory_footprint=true. Rows are the mean of iterations 2 and 3 (GPU: LITERT_CL). Cold init is the first process after the compiled-program caches were removed, warm init the benchmark process, peak the largest of the benchmark process's three Peak system ram usage lines.

Galaxy S26	Prefill 1024 (tok/s)	Decode 256 (tok/s)	TTFT (s)
int8 GPU	3396.4	58.97	0.32
int4 GPU ¹	2427.3	73.25	0.435
int8 CPU	890.1	45.65	1.20
int4 CPU ²	346.8	57.12	3.06

Init cold/warm and peak: GPU int8 4.30/1.28 s, 517.2 MB; GPU int4 4.97/1.82 s, 534.4 MB; CPU int8 0.93/0.13 s, 1948.0 MB; CPU int4 1.59/1.22 s, 1652.5 MB. Before each process of the first three rows the harness waited for Android thermal status 0 and uncapped clocks; in the int8 CPU run the CPU clock was lowered in 9 of 10 samples, taken 2–3 s apart. ¹ The GPU clock limit fell from 1300 to 1100 MHz in the last 2 of 7 samples; the unflagged first iteration read 2494.6 / 69.22 tok/s. ² An earlier run that started on a hot phone (thermal status 3); the CPU clock was lowered in 10 of 11 samples.

Pixel 8a	Prefill 1024 (tok/s)	Decode 256 (tok/s)	TTFT (s)
int8 GPU	1451.1	34.75	0.735
int4 GPU	619.7	39.91	1.675
int8 CPU	207.0	22.73	4.99
int4 CPU	153.0	30.33	6.78

Init cold/warm and peak: GPU int8 19.42/1.88 s, 1107.6 MB; GPU int4 8.95/3.12 s, 1026.0 MB; CPU int8 2.03/0.46 s, 1534.8 MB; CPU int4 3.05/2.62 s, 1250.7 MB. Every run started at thermal status 1; in the CPU runs the CPU clock was lowered in 13 of 17 samples in each file's run, taken 2–4 s apart, and in the GPU runs in 3 of 11 (int8) and 5 of 11 (int4); the harness could not read the Mali GPU clock. Not run on the NPU.

Conversion notes

• Source and stack: LiquidAI/LFM2-700M at revision 86f49fc9; litert-torch 0.9.4, litert-converter 0.4.0, ai-edge-quantizer 0.9.0, transformers 5.14.1, torch 2.13.0; packed by the litert-lm 0.18.0 CLI. lfm_work/convert_lfm2_700m.sh in hf-to-litertlm rebuilt both files, all four sections byte-identical (only the packer's uuid and timestamp differ).
• Order: two exports (prefill signatures 1024, 512, 256, 128, 64, 32, 16, 8, 4, 2 and 1, plus decode, over a 4096-position KV cache): the int8 file is the export with the default int8 recipe; the int4 file is post-hoc int4 quantization of an unquantized export (no zero or subnormal block scale in its 92 blockwise tensors) → metadata repack of each file with LiteRT-LM's lfm2 configuration.
• Template: Liquid AI's chat template fails at render on litert-lm 0.18.0 (message.get("content")), so the bundle carries LiteRT-LM's lfm2 template, which reads message content as a list of parts.

License and changes

Distributed under the LFM Open License v1.0 inherited from the base model (LICENSE); its section 5 conditions commercial use on an annual revenue below 10 million US dollars. Changes from the original work: exported to the LiteRT-LM format and quantized to int8 and int4 as described above; chat template and stop tokens taken from LiteRT-LM's models/lfm2 configuration, with max_num_tokens changed from 32768 to 4096 and supports_thinking from true to false. This conversion was made and is published independently of Liquid AI.