| --- |
| license: other |
| license_name: lfm1.0 |
| license_link: LICENSE |
| base_model: |
| - LiquidAI/LFM2.5-1.2B-Instruct |
| - LiquidAI/LFM2.5-1.2B-Instruct-ONNX |
| library_name: transformers.js |
| pipeline_tag: text-generation |
| language: |
| - en |
| - ja |
| - ko |
| - fr |
| - es |
| - de |
| - it |
| - pt |
| - ar |
| - zh |
| tags: |
| - onnx |
| - webgpu |
| - liquid |
| - lfm2.5 |
| --- |
| |
| # LFM2.5-1.2B-Instruct ONNX for the web |
|
|
| A 4-bit ONNX build of [LFM2.5-1.2B-Instruct](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct) by [Liquid AI](https://www.liquid.ai/), made for in-browser use with [Transformers.js](https://github.com/huggingface/transformers.js) on WebGPU, including iPhone Safari. |
|
|
| This is an unofficial community build. It is not made, reviewed or endorsed by Liquid AI. |
|
|
| ## Changes from the original |
|
|
| This repository contains a modified file. Compared with [LiquidAI/LFM2.5-1.2B-Instruct-ONNX](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-ONNX) at commit [`10f72e7`](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-ONNX/tree/10f72e70abf67ac0fd7ebf15bc5854726891d864) (`onnx/model_q4f16.onnx`): |
|
|
| - **Logits only for the positions you need:** the graph has a new `num_logits_to_keep` input (int64 scalar), the same input [onnx-community/LFM3-1.7B-ONNX](https://huggingface.co/onnx-community/LFM3-1.7B-ONNX) has. The final-norm hidden states are sliced to the last `num_logits_to_keep` positions before the LM head; `0` keeps every position, as before. Transformers.js passes `1` during `generate()`, so reading a prompt computes one row of logits instead of one per prompt token. The `logits` output is `[batch, num_logits_to_keep, 65536]`, or `[batch, sequence_length, 65536]` for `0`. |
| - **Why:** the original graph returns float32 logits for every prompt token. For an 800-token prompt that is a 210 MB tensor (plus a 105 MB float16 one before the cast), made on the GPU and then copied to the page. On iPhone Safari, where a tab gets about 2 GB including GPU memory, long prompts such as ones with a tool definition crashed the tab. |
| - **Nothing else changed:** the edit adds three nodes (`Neg`, `Unsqueeze`, `Slice`), three small int64 constants and the input. The weights are not touched: `onnx/model_q4f16.onnx_data` is byte-identical to the original (SHA-256 `46cfacc12941150620a3f644a5269e9baebd75d681cfa09cadeef71b8ed64ac2`). |
|
|
| `config.json`, `generation_config.json`, `tokenizer.json`, `tokenizer_config.json` and `chat_template.jinja` are unchanged copies. Only the q4f16 model is included. |
|
|
| | | Original q4f16 | This build | |
| |---|---|---| |
| | `logits` for an 800-token prompt | `[1, 800, 65536]`, 209,715,200 bytes | `[1, 1, 65536]`, 262,144 bytes | |
| | Last-token logits on 5 prompts of 18–963 tokens, max abs difference | | ≤ 0.0078 (logits up to ±17), same top token on every prompt | |
| | All positions (`num_logits_to_keep = 0`), max abs difference | | 0 | |
| | KV and conv caches after the prompt | | identical | |
| | 8 greedy decode steps after each prompt | | identical tokens and logits | |
| | 541-token tool prompt + 24 new tokens, Transformers.js 4.3.2 on WebGPU (Node, Apple Silicon) | 250–259 ms | 222–228 ms | |
| | 963-token prompt, onnxruntime CPU | 1,669 ms | 1,497 ms | |
|
|
| Measurements were taken with onnxruntime 1.31 on CPU unless noted. The small last-token difference comes from the LM head's 4-bit matmul running on one row instead of hundreds, which changes the order of the float16 sums. Through Transformers.js 4.3.2, greedy generation gave the same tokens with both builds. |
|
|
| ## Usage |
|
|
| Use it like the original ONNX model, with `dtype: "q4f16"` and `device: "webgpu"`. See [LiquidAI/LFM2.5-1.2B-Instruct-ONNX](https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct-ONNX) for examples. Transformers.js sets `num_logits_to_keep` on its own. If you run the graph with onnxruntime directly, pass it as an int64 scalar: `1` for generation, `0` for all positions. |
|
|
| ## License |
|
|
| LFM2.5 is released under the [LFM Open License v1.0](LICENSE), which also covers this build. Commercial use is licensed only for entities with annual revenue under $10 million. Original model © Liquid AI, Inc. |
|
|
