File size: 4,274 Bytes
35a9846 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 | ---
license: other
license_name: lfm1.0
license_link: LICENSE
base_model: LiquidAI/LFM2.5-Encoder-230M
pipeline_tag: feature-extraction
library_name: litert
language:
- en
- de
- es
- fr
- it
- nl
- pl
- pt
- ar
- hi
- ja
- ru
- tr
- vi
- zh
tags:
- litert
- tflite
- on-device
- edge
- encoder
- bidirectional
- embeddings
- fill-mask
- liquid
- lfm2
- lfm2.5
---
# LFM2.5-Encoder-230M — LiteRT
[LiquidAI/LFM2.5-Encoder-230M](https://huggingface.co/LiquidAI/LFM2.5-Encoder-230M) converted to **LiteRT** (`.tflite`) for on-device inference. A multilingual (15 languages) bidirectional encoder on the LFM2 hybrid backbone (gated short-convolutions + grouped-query attention) — use it for embeddings, retrieval, classification heads, and masked-token prediction, fully offline on CPU. This is the lightweight sibling of [LFM2.5-Encoder-350M](https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M) for tight latency and memory budgets.
| File | Recipe | Size | |
|---|---|---|---|
| `LFM2.5-Encoder-230M_wi8fc.tflite` | int8 dynamic-range (linears + embedding, convs float) | 246 MB | mobile + desktop (iPhone-verified bit-exact) |
| `LFM2.5-Encoder-230M_fp16.tflite` | fp16 weights, float compute | 463 MB | desktop — XNNPACK's per-signature fp32 unpacking is heavy on phone memory limits |
## Signatures
All signatures take batch-1, right-padded static shapes: `input_ids` int32 `[1, S]`, `attention_mask` int32 `[1, S]` (1 = real token, 0 = pad).
| Signature | Output |
|---|---|
| `encode_64` / `encode_128` / `encode_256` / `encode_512` | `last_hidden_state` float32 `[1, S, 1024]`, zeroed at padded positions |
| `mlm_128` | masked-LM logits float32 `[1, 128, 65536]` |
Padded positions are fully masked inside the graph (conv path and attention), so the output at valid positions is independent of padding length — `encode_64/128/256` agree bitwise on the same sentence, and match the unpadded PyTorch reference.
## Quality (parity vs PyTorch fp32 reference)
16 sentences covering all 15 supported languages; mean-pooled sentence embedding cosine vs the original `Lfm2BidirectionalModel`, and top-5 fill-mask agreement on en/fr/de/ja cloze prompts (the fp16/fp32 conversion reproduces the base card's documented `The capital of France is [MASK].` output verbatim):
| Variant | Pooled cos (min / mean) | Per-token corr (min) | Fill-mask |
|---|---|---|---|
| fp16 | 1.000000 / 1.000000 | 0.999999 | top-5 sets identical (4/4 prompts) |
| int8 (wi8fc) | 0.994781 / 0.998148 | 0.986881 | top-1 4/4, ≥3/5 top-5 overlap on all |
## Speed (CPU/XNNPACK)
| Variant | Device | encode_128 | encode_512 |
|---|---|---|---|
| int8 (wi8fc) | Apple-silicon Mac (all threads) | 39 ms (3312 tok/s) | 100 ms (5116 tok/s) |
| int8 (wi8fc) | iPhone 17 Pro (6 threads) | 27 ms | 93 ms (~5500 tok/s) |
On the iPhone 17 Pro the int8 model reproduces the Mac outputs **bit-exactly** (cosine 1.000000, max diff 0.0) across all tested languages and signatures; peak footprint ≈1.0 GiB.
## Usage (Python)
```python
import numpy as np
from ai_edge_litert.interpreter import Interpreter
from tokenizers import Tokenizer
tok = Tokenizer.from_file("tokenizer.json")
it = Interpreter(model_path="LFM2.5-Encoder-230M_wi8fc.tflite")
encode = it.get_signature_runner("encode_128")
ids = tok.encode("On-device embeddings, private and fast.").ids
x = np.zeros((1, 128), np.int32); m = np.zeros((1, 128), np.int32)
x[0, :len(ids)] = ids; m[0, :len(ids)] = 1
h = list(encode(input_ids=x, attention_mask=m).values())[0] # [1, 128, 1024]
emb = h[0, :len(ids)].mean(axis=0) # sentence embedding
```
For masked-token prediction use the `mlm_128` signature and read the logits at the `[MASK]` position. On Android/iOS use the LiteRT runtime's SignatureRunner APIs with the same signature names; the tokenizer is the standard Hugging Face `tokenizer.json` (works with the `tokenizers` libraries for Rust/Swift/Kotlin).
## License
LFM Open License v1.0 (see `LICENSE`, unchanged from the base model). Note the license's commercial-use threshold (Section 5). This repository redistributes converted **Derivative Works** of LiquidAI/LFM2.5-Encoder-230M with modification notices per Section 4; all credit for the model to [Liquid AI](https://www.liquid.ai/).
|