File size: 4,387 Bytes
768e6d6 d5f69da 768e6d6 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 | ---
license: other
license_name: lfm1.0
license_link: https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M-Prompt-Router/blob/main/LICENSE
base_model: LiquidAI/LFM2.5-Encoder-350M-Prompt-Router
base_model_relation: quantized
library_name: transformers.js
pipeline_tag: zero-shot-classification
tags:
- onnx
- transformers.js
- lfm2
- quantized
language:
- en
- de
- es
- fr
- it
- nl
- pl
- pt
- ar
- hi
- ja
- ru
- tr
- vi
- zh
---
# LFM2.5-Encoder-350M-Prompt-Router-ONNX
ONNX export of [`LiquidAI/LFM2.5-Encoder-350M-Prompt-Router`](https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M-Prompt-Router), quantized to run **fully in the browser** through
[transformers.js](https://github.com/huggingface/transformers.js). No inference server: the weights are
fetched once, cached, and every forward pass happens in the tab.
A zero-shot prompt router. Categories are ordinary prose supplied at call time — nothing is
trained or cached per label set. One bidirectional pass over the category list *and* the text
scores every category at once.
All credit for the model itself goes to [Liquid AI](https://huggingface.co/LiquidAI). This repository
contains only a re-export; the weights are unchanged apart from quantization, and the original
[LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M-Prompt-Router/blob/main/LICENSE) applies.
**[Try it in your browser →](https://kucukkanat.github.io/lfm-encoders/)** — no install, no API key.
Tooling, demo and the export pipeline: <https://github.com/kucukkanat/lfm-encoders>
## Files
| dtype | File | Size |
| --- | --- | --: |
| `q8` | `onnx/model_quantized.onnx` | 357 MB |
| `q4` | `onnx/model_q4.onnx` | 449 MB |
The graph takes `input_ids` + `attention_mask`, is dynamic in batch and sequence, and returns
`token_proj` and `rule_proj`.
## Usage
```js
import { AutoTokenizer, PreTrainedModel, Tensor } from "@huggingface/transformers";
const id = "kucukkanat/LFM2.5-Encoder-350M-Prompt-Router-ONNX";
const tokenizer = await AutoTokenizer.from_pretrained(id);
const model = await PreTrainedModel.from_pretrained(id, { dtype: "q8" });
const { input_ids } = tokenizer("some text");
const out = await model({
input_ids,
attention_mask: new Tensor("int64", new BigInt64Array(input_ids.dims[1]).fill(1n), input_ids.dims),
});
```
`PreTrainedModel` rather than `AutoModel` is deliberate: this is a plain "feed the named inputs, read the
named outputs" session, not one of transformers.js's built-in architectures.
### Prompt format
Both projection towers expect one string laid out exactly like this — the model was trained on it and the
character arithmetic that locates each label depends on it byte for byte:
```
Categories:
- label one
- label two
Text:
<the text>
```
`token_proj` is the query tower and `rule_proj` the key tower, both 256-d and emitted **per token**. Pool
the tokens covering each label to get its vector. Pooling after projecting is exact rather than an
approximation: both towers are affine, and an affine map commutes with a mean — which is what keeps one
static graph usable for any number of labels.
Scoring is `cosine between the L2-normalised pooled towers, scaled by a learned temperature, then a softmax across labels`.
[`@lfm-encoder/tasks`](https://github.com/kucukkanat/lfm-encoders) implements all of this, including the
character-offset reconstruction transformers.js does not provide.
## Accuracy
Measured from JavaScript against the fp32 PyTorch reference. Δ is the largest absolute difference in a
final probability.
| dtype | max Δ | mean Δ | top-1 flips (4 cases) |
| --- | --: | --: | --: |
| `fp32` | 6.4e-5 | 1.6e-5 | 0 |
| `q8` | 0.0910 | 0.0230 | 0 |
| `q4` | 0.1221 | 0.0309 | 0 |
## Notes
- `q8` is smaller on disk but uses **more** browser RAM than fp32 and runs slower: onnxruntime's WASM
kernels compute in float, so quantized weights are unpacked at session load. Quantization here buys
download size, not speed or memory.
- Budget roughly 1.5 GB of RAM per resident model, and expect a tab to hold its high-water mark until
reloaded.
- `fp16` / `q4f16` are deliberately absent: RMSNorm's variance overflows fp16 on this architecture and
every hidden state collapses to zeros.
|