README.md

---
license: other
license_name: lfm1.0
license_link: LICENSE
base_model: LiquidAI/LFM2.5-Encoder-350M-Prompt-Router
pipeline_tag: text-classification
library_name: litert
tags:
- litert
- tflite
- on-device
- edge
- encoder
- zero-shot
- routing
- liquid
- lfm2
- lfm2.5
---

# LFM2.5-Encoder-350M-Prompt-Router — LiteRT

[LiquidAI/LFM2.5-Encoder-350M-Prompt-Router](https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M-Prompt-Router) converted to **LiteRT** (`.tflite`) for on-device inference. Zero-shot prompt routing: define your routing lanes as free text and the model scores the whole prompt against every lane in one CPU pass ([demo Space](https://huggingface.co/spaces/LiquidAI/prompt-routing)).

| File | Recipe | Size | |
|---|---|---|---|
| `LFM2.5-Encoder-350M-Prompt-Router_wi8fc.tflite` | int8 dynamic-range (linears + embedding, convs float) | 365 MB | mobile + desktop (iPhone-verified bit-exact, 145 ms) |
| `LFM2.5-Encoder-350M-Prompt-Router_fp16.tflite` | fp16 weights, float compute | 713 MB | desktop — phone memory limits (XNNPACK per-signature fp32 unpacking) |

## Signatures

`route_128` / `route_512` (S = 128 / 512, batch 1, right-padded, up to **8 lane slots**):

| Input | Shape | |
|---|---|---|
| `input_ids` | int32 `[1, S]` | prompt tokens: `Categories:\n- <lane 1>\n- <lane 2>…\n\nText:\n<prompt>` |
| `attention_mask` | int32 `[1, S]` | 1 = token, 0 = pad |
| `text_pool` | float32 `[1, 1, S]` | mean-pool weights over the prompt's text tokens (`1/n` each) |
| `category_pool` | float32 `[1, 8, S]` | row r = mean-pool weights over lane r's tokens; unused lane rows all-zero |

Output: `logits` float32 `[1, 8]`. Softmax over the first N (real) lanes only — all-zero pool rows produce a constant bias logit that must be ignored.

The pool matrices are built host-side from tokenizer character offsets, exactly like the base repo's `route()` helper:

```python
import numpy as np
from tokenizers import Tokenizer

def build_inputs(text, lanes, tok, S=512):
    body = "\n".join(f"- {r}" for r in lanes)
    prefix = f"Categories:\n{body}\n\nText:\n"
    enc = tok.encode(prefix + text)
    ids, offs = enc.ids, enc.offsets
    x = np.zeros((1, S), np.int32); m = np.zeros((1, S), np.int32)
    x[0, :len(ids)] = ids; m[0, :len(ids)] = 1
    tp = np.zeros((1, 1, S), np.float32)
    ti = [i for i, (a, b) in enumerate(offs) if b > len(prefix) and a != b]
    tp[0, 0, ti] = 1 / len(ti)
    cp = np.zeros((1, 8, S), np.float32)
    pos = len("Categories:\n")
    for r, lane in enumerate(lanes):
        a, b = pos + 2, pos + 2 + len(lane); pos = b + 1
        idx = [i for i, (ta, tb) in enumerate(offs) if ta < b and tb > a and ta != tb]
        cp[0, r, idx] = 1 / len(idx)
    return {"input_ids": x, "attention_mask": m, "text_pool": tp, "category_pool": cp}
```

## Verification

Task-level parity vs the PyTorch reference (demo prompt, 4 lanes): fp32, fp16 **and int8 all reproduce the reference lane probabilities to 4 decimal places** ("coding question" 0.838). On an iPhone 17 Pro the int8 file reproduces the desktop outputs **bit-exactly** (cosine 1.000000, max diff 0.0) at 145 ms per `route_512` pass (6 threads, XNNPACK).

## License

LFM Open License v1.0 (see `LICENSE`, unchanged from the base model). Note the license's commercial-use threshold (Section 5). This repository redistributes converted **Derivative Works** of LiquidAI/LFM2.5-Encoder-350M-Prompt-Router with modification notices per Section 4; all credit for the model to [Liquid AI](https://www.liquid.ai/).