daveycodez/LFM3-1.7B-Web

LFM3-1.7B ONNX for the web

A 4-bit ONNX build of LFM3-1.7B by Liquid AI, made for in-browser use with Transformers.js on WebGPU, including iPhone Safari.

This is an unofficial community build. It is not made, reviewed or endorsed by Liquid AI.

Changes from the original

This repository contains modified files. Compared with onnx-community/LFM3-1.7B-ONNX (onnx/model_q4f16.onnx):

• Re-quantized weights: all MatMulNBits weights and the tied embedding / LM head were re-quantized from the fp16 ONNX model to 4-bit with block size 32, using GPTQ for the matrices. The gate, up, q, k and v projections are symmetric (no zero points), so onnxruntime's fused WebGPU kernels apply. Block size 32 also lets onnxruntime-web use its fast prompt-processing kernel.
• Smaller vocabulary: the vocabulary was trimmed from 128,000 to 49,012 BPE tokens (plus the same 140 special tokens; 49,152 embedding rows). Every byte token and all special tokens (chat, tool-call and thinking tokens) are kept, and the merges stay consistent, so any text still tokenizes and decodes back exactly. Less common words split into more tokens, and non-English text needs more tokens. The embedding / LM head, tokenizer.json, config.json and generation_config.json (special token ids) changed to match.
• Shorter rotary caches: the cos/sin caches were cut from 262,144 to 16,384 positions. Sequences longer than 16,384 tokens fail with a clear error instead of running.
• No second copy of the embedding at load: in the original graph, a Reshape feeds the shared embedding / LM-head weights to the LM head, which makes onnxruntime-web copy that tensor while creating the session. The graph now gives the LM head the same weights without the Reshape. Outputs are identical.
• Flash attention for the sliding-window layers: the five sliding-window layers (window 256) get their window as an attention bias instead of GroupQueryAttention's local_window_size. onnxruntime-web runs local_window_size layers without flash attention, which keeps a scores buffer of 16 heads × prompt length² (261 MB per layer for a 2,859-token prompt). The mask is the same, so outputs differ only by float rounding: on WebGPU, next-token loss over a 2,000-token prompt went from 3.0031 to 3.0009 and from 3.4849 to 3.4882 on two texts, and the difference is no larger inside the window than before it, where only the attention kernel changed. On a Mac, peak GPU buffers for a 2,859-token prompt went from 1,575 MB to 1,342 MB, and prompts that long process about 5% faster.
• Calibration data: about 36k tokens from the WikiText-2 training split plus chat replies generated by the fp16 model. No calibration text is included in the files.

The chat template, tokenizer_config.json and special_tokens_map.json are unchanged.

	Original q4f16	This build
model_q4f16.onnx_data	1,087,789,056 bytes	900,061,184 bytes (−188 MB)
WikiText-2 test perplexity, per original token (fp16: 24.87)	27.10	29.17
Top-1 agreement with fp16 on chat prompts	81.0%	89.1%
KL divergence from fp16 on chat prompts	0.153	0.083
Prompts answered with a direct tool call (fp16: 20/20)	13/20	20/20
Max sequence length	262,144	16,384

Measurements were taken with onnxruntime 1.31. Perplexity is scored per original token so the two vocabularies compare fairly; the higher WikiText perplexity comes mostly from rare words that the smaller vocabulary splits. The WikiText numbers favor this build slightly, since calibration used WikiText training text; the chat-prompt numbers don't share that overlap.

Usage

Use it like the original ONNX model, with dtype: "q4f16" and device: "webgpu". See onnx-community/LFM3-1.7B-ONNX for examples.

License

LFM3 is released under the LFM Open License v1.0, which also covers this build. Commercial use is licensed only for entities with annual revenue under $10 million. Original model © Liquid AI, Inc.