File size: 3,531 Bytes
3ee2a4b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | ---
license: other
license_name: lfm1.0
license_link: LICENSE
base_model: LiquidAI/LFM2.5-Encoder-350M-Prompt-Router
pipeline_tag: text-classification
library_name: litert
tags:
- litert
- tflite
- on-device
- edge
- encoder
- zero-shot
- routing
- liquid
- lfm2
- lfm2.5
---
# LFM2.5-Encoder-350M-Prompt-Router — LiteRT
[LiquidAI/LFM2.5-Encoder-350M-Prompt-Router](https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M-Prompt-Router) converted to **LiteRT** (`.tflite`) for on-device inference. Zero-shot prompt routing: define your routing lanes as free text and the model scores the whole prompt against every lane in one CPU pass ([demo Space](https://huggingface.co/spaces/LiquidAI/prompt-routing)).
| File | Recipe | Size | |
|---|---|---|---|
| `LFM2.5-Encoder-350M-Prompt-Router_wi8fc.tflite` | int8 dynamic-range (linears + embedding, convs float) | 365 MB | mobile + desktop (iPhone-verified bit-exact, 145 ms) |
| `LFM2.5-Encoder-350M-Prompt-Router_fp16.tflite` | fp16 weights, float compute | 713 MB | desktop — phone memory limits (XNNPACK per-signature fp32 unpacking) |
## Signatures
`route_128` / `route_512` (S = 128 / 512, batch 1, right-padded, up to **8 lane slots**):
| Input | Shape | |
|---|---|---|
| `input_ids` | int32 `[1, S]` | prompt tokens: `Categories:\n- <lane 1>\n- <lane 2>…\n\nText:\n<prompt>` |
| `attention_mask` | int32 `[1, S]` | 1 = token, 0 = pad |
| `text_pool` | float32 `[1, 1, S]` | mean-pool weights over the prompt's text tokens (`1/n` each) |
| `category_pool` | float32 `[1, 8, S]` | row r = mean-pool weights over lane r's tokens; unused lane rows all-zero |
Output: `logits` float32 `[1, 8]`. Softmax over the first N (real) lanes only — all-zero pool rows produce a constant bias logit that must be ignored.
The pool matrices are built host-side from tokenizer character offsets, exactly like the base repo's `route()` helper:
```python
import numpy as np
from tokenizers import Tokenizer
def build_inputs(text, lanes, tok, S=512):
body = "\n".join(f"- {r}" for r in lanes)
prefix = f"Categories:\n{body}\n\nText:\n"
enc = tok.encode(prefix + text)
ids, offs = enc.ids, enc.offsets
x = np.zeros((1, S), np.int32); m = np.zeros((1, S), np.int32)
x[0, :len(ids)] = ids; m[0, :len(ids)] = 1
tp = np.zeros((1, 1, S), np.float32)
ti = [i for i, (a, b) in enumerate(offs) if b > len(prefix) and a != b]
tp[0, 0, ti] = 1 / len(ti)
cp = np.zeros((1, 8, S), np.float32)
pos = len("Categories:\n")
for r, lane in enumerate(lanes):
a, b = pos + 2, pos + 2 + len(lane); pos = b + 1
idx = [i for i, (ta, tb) in enumerate(offs) if ta < b and tb > a and ta != tb]
cp[0, r, idx] = 1 / len(idx)
return {"input_ids": x, "attention_mask": m, "text_pool": tp, "category_pool": cp}
```
## Verification
Task-level parity vs the PyTorch reference (demo prompt, 4 lanes): fp32, fp16 **and int8 all reproduce the reference lane probabilities to 4 decimal places** ("coding question" 0.838). On an iPhone 17 Pro the int8 file reproduces the desktop outputs **bit-exactly** (cosine 1.000000, max diff 0.0) at 145 ms per `route_512` pass (6 threads, XNNPACK).
## License
LFM Open License v1.0 (see `LICENSE`, unchanged from the base model). Note the license's commercial-use threshold (Section 5). This repository redistributes converted **Derivative Works** of LiquidAI/LFM2.5-Encoder-350M-Prompt-Router with modification notices per Section 4; all credit for the model to [Liquid AI](https://www.liquid.ai/).
|