---
license: other
license_name: lfm1.0
license_link: LICENSE
base_model: LiquidAI/LFM2.5-Encoder-350M-Prompt-Router
pipeline_tag: zero-shot-classification
library_name: onnx
tags:
- onnx
- lfm2
- zero-shot-classification
- lemonade
language:
- en
- de
- es
- fr
- it
- nl
- pl
- pt
- ar
- hi
- ja
- ru
- tr
- vi
- zh
---

# LFM2.5-Encoder-350M-Prompt-Router-ONNX

An ONNX export of
[LiquidAI/LFM2.5-Encoder-350M-Prompt-Router](https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M-Prompt-Router),
for [Lemonade](https://github.com/lemonade-sdk/lemonade) and its
[ort-server](https://github.com/lemonade-sdk/ort-server) backend.

**All credit for the model goes to [Liquid AI](https://huggingface.co/LiquidAI).**
This repository only changes the file format. The weights are the original
float32 weights, and the LFM Open License v1.0 applies (see `LICENSE`).

The model is a zero-shot prompt router. You give it a text and a list of labels
at request time. It scores the text against every label in one encoder pass, and
the scores sum to 1.

## Changes from the original

| File | Origin |
|---|---|
| `model.onnx` | Converted from `model.safetensors` at revision `35ca4a0469f180f1cf05a630df8842fa17ac18e3`. |
| `config.json`, `tokenizer.json`, `LICENSE` | Unchanged copies. |
| `manifest.json` | New. ort-server settings: `task: zero-shot-classification`, `max_length: 2048`. |

The window is 2,048 tokens. The position table holds 128,000, but this export
uses eager attention, so memory grows with the square of the window: about
1.4 GB extra at 2,048 tokens and 5.3 GB at 4,096. A longer text is cut, and the
response then has `"truncated": true`. To raise the window, pass ort-server's
`--max-length` (in Lemonade: `--onnxruntime-args "--max-length 4096"`).

The graph takes `input_ids`, `attention_mask`, `text_pool` `[b, 1, s]` and
`category_pool` `[b, r, s]`, and returns `logits` `[b, r]`. The label count `r`
and the sequence length `s` are dynamic. The two pooling inputs mark which tokens
belong to the text and to each label. ort-server builds them from the
tokenizer's character offsets.

## Use with Lemonade

Lemonade needs ort-server 0.4.0 or later. The model is built in:

```bash
lemonade pull LFM2.5-Encoder-350M-Prompt-Router-ONNX
```

Classify a text:

```bash
curl -X POST http://localhost:13305/api/v1/classify \
  -H "Content-Type: application/json" \
  -d '{"model": "LFM2.5-Encoder-350M-Prompt-Router-ONNX",
       "input": "Can you help me debug a failing Python unit test?",
       "labels": ["Coding", "Sales", "Creative writing", "General knowledge"]}'
```

```json
{"labels": {"Coding": 0.838, "Creative writing": 0.054, "General knowledge": 0.054, "Sales": 0.054}, "truncated": false}
```

In a router policy, use the model in a `zero_shot` classifier. See Lemonade's
`docs/dev/router-policy.md`.

## How this export was made

The scripts are in ort-server's
[`tools/zero_shot_router/`](https://github.com/lemonade-sdk/ort-server/tree/main/tools/zero_shot_router).
From an ort-server checkout:

```bash
pip install torch transformers onnx onnxruntime huggingface_hub
python tools/zero_shot_router/export.py <out_dir> --revision 35ca4a0469f180f1cf05a630df8842fa17ac18e3
python tools/zero_shot_router/parity.py <out_dir> --revision 35ca4a0469f180f1cf05a630df8842fa17ac18e3 --binary <ort-server>
```

`export.py` loads the checkpoint with `trust_remote_code=True`. This is required,
because `Lfm2BidirForSequenceRouting` and `route()` are in the checkpoint's
`modeling_lfm2_bidirectional.py`, not in `transformers`. The script then:

1. Replaces the short-conv forward, whose padding and groups come from tensor
   shapes that tracing cannot export.
2. Exports `forward()` at opset 17, not `route()`, so that the label count stays dynamic.
3. Compares the graph with `route()` on 16 cases with 2 to 8 labels, and stops
   on any score difference above `1e-4` or any changed top label.

Tested with transformers 4.57, torch 2.8 and onnxruntime 1.24.

## Parity with the original model

`parity.py` runs 95 labelled dummy cases through three paths:

- the original safetensors model, with its own `route()` in PyTorch
- this ONNX graph in ONNX Runtime
- ort-server over HTTP, which builds the pooling inputs in C++

The cases cover routing, PII, prompt injection, support topics, French,
Spanish, German and Japanese, and three long texts of 900 to 1,900 tokens.

| Path | Accuracy on the dummy labels | Top-1 agreement with PyTorch | Max score difference |
|---|---|---|---|
| PyTorch (safetensors) | 76.84% | - | - |
| ONNX Runtime | 76.84% | 100% | 2.3e-6 |
| ort-server | 76.84% | 100% | 2.3e-6 |

The accuracy column describes the model on hand-written labels, not the
conversion: all three paths get the same cases right and wrong.

## License

LFM Open License v1.0. See `LICENSE` and the
[original model](https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M-Prompt-Router).
Commercial use above the license's threshold is not licensed.