README.md

---
license: other
license_name: lfm1.0
license_link: https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M-Prompt-Router/blob/main/LICENSE
base_model: LiquidAI/LFM2.5-Encoder-350M-Prompt-Router
base_model_relation: quantized
library_name: transformers.js
pipeline_tag: zero-shot-classification
tags:
- onnx
- transformers.js
- lfm2
- quantized
language:
- en
- de
- es
- fr
- it
- nl
- pl
- pt
- ar
- hi
- ja
- ru
- tr
- vi
- zh
---

# LFM2.5-Encoder-350M-Prompt-Router-ONNX

ONNX export of [`LiquidAI/LFM2.5-Encoder-350M-Prompt-Router`](https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M-Prompt-Router), quantized to run **fully in the browser** through
[transformers.js](https://github.com/huggingface/transformers.js). No inference server: the weights are
fetched once, cached, and every forward pass happens in the tab.

A zero-shot prompt router. Categories are ordinary prose supplied at call time — nothing is
trained or cached per label set. One bidirectional pass over the category list *and* the text
scores every category at once.

All credit for the model itself goes to [Liquid AI](https://huggingface.co/LiquidAI). This repository
contains only a re-export; the weights are unchanged apart from quantization, and the original
[LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M-Prompt-Router/blob/main/LICENSE) applies.

**[Try it in your browser →](https://kucukkanat.github.io/lfm-encoders/)** — no install, no API key.

![Zero-shot prompt routing in the browser](https://raw.githubusercontent.com/kucukkanat/lfm-encoders/main/docs/screenshots/prompt-routing.png)

Tooling, demo and the export pipeline: <https://github.com/kucukkanat/lfm-encoders>

## Files

| dtype | File | Size |
| --- | --- | --: |
| `q8` | `onnx/model_quantized.onnx` | 357 MB |
| `q4` | `onnx/model_q4.onnx` | 449 MB |

The graph takes `input_ids` + `attention_mask`, is dynamic in batch and sequence, and returns
`token_proj` and `rule_proj`.

## Usage

```js
import { AutoTokenizer, PreTrainedModel, Tensor } from "@huggingface/transformers";

const id = "kucukkanat/LFM2.5-Encoder-350M-Prompt-Router-ONNX";
const tokenizer = await AutoTokenizer.from_pretrained(id);
const model = await PreTrainedModel.from_pretrained(id, { dtype: "q8" });

const { input_ids } = tokenizer("some text");
const out = await model({
  input_ids,
  attention_mask: new Tensor("int64", new BigInt64Array(input_ids.dims[1]).fill(1n), input_ids.dims),
});
```

`PreTrainedModel` rather than `AutoModel` is deliberate: this is a plain "feed the named inputs, read the
named outputs" session, not one of transformers.js's built-in architectures.

### Prompt format

Both projection towers expect one string laid out exactly like this — the model was trained on it and the
character arithmetic that locates each label depends on it byte for byte:

```
Categories:
- label one
- label two

Text:
<the text>
```

`token_proj` is the query tower and `rule_proj` the key tower, both 256-d and emitted **per token**. Pool
the tokens covering each label to get its vector. Pooling after projecting is exact rather than an
approximation: both towers are affine, and an affine map commutes with a mean — which is what keeps one
static graph usable for any number of labels.

Scoring is `cosine between the L2-normalised pooled towers, scaled by a learned temperature, then a softmax across labels`.

[`@lfm-encoder/tasks`](https://github.com/kucukkanat/lfm-encoders) implements all of this, including the
character-offset reconstruction transformers.js does not provide.


## Accuracy

Measured from JavaScript against the fp32 PyTorch reference. Δ is the largest absolute difference in a
final probability.

| dtype | max Δ | mean Δ | top-1 flips (4 cases) |
| --- | --: | --: | --: |
| `fp32` | 6.4e-5 | 1.6e-5 | 0 |
| `q8` | 0.0910 | 0.0230 | 0 |
| `q4` | 0.1221 | 0.0309 | 0 |

## Notes

- `q8` is smaller on disk but uses **more** browser RAM than fp32 and runs slower: onnxruntime's WASM
  kernels compute in float, so quantized weights are unpacked at session load. Quantization here buys
  download size, not speed or memory.
- Budget roughly 1.5 GB of RAM per resident model, and expect a tab to hold its high-water mark until
  reloaded.
- `fp16` / `q4f16` are deliberately absent: RMSNorm's variance overflows fp16 on this architecture and
  every hidden state collapses to zeros.