---
license: mit
base_model: minishlab/potion-multilingual-128M
tags:
  - model2vec
  - static-embeddings
  - embeddings
  - multilingual
  - quantized
library_name: model2vec
---

# potion-multilingual-128M-i8-tokenade

An int8 quantisation of [`minishlab/potion-multilingual-128M`](https://huggingface.co/minishlab/potion-multilingual-128M),
plus precomputed tokenizer artifacts for low-memory loading.

All credit for the model itself goes to [the Minish Lab](https://huggingface.co/minishlab)
— this repo adds no training, no distillation and no new data. It only changes
how the same weights are stored and loaded.

## Why

model2vec inference is `tokenize → look up rows → mean-pool`: no context, no
forward pass. Loading it should not cost like a transformer, but in practice a
stock load decodes the whole weight matrix to `f32` and inflates
`tokenizer.json` into an in-memory trie — together about **1 GB of RSS and
~930 ms** before the first vector. That is fine for a long-lived server and
prohibitive for a short-lived CLI process.

These artifacts are the same model arranged so it can be memory-mapped instead:

| | stock load | these artifacts |
|---|---|---|
| RSS | ~1056 MB | **~4.5 MB** |
| first embedding | ~930 ms | **~1 ms** |
| per embedding | ~0.03 ms | ~0.11 ms |

The per-embedding cost regresses (no pre-decoded matrix, no cache); a process
needs roughly 12 700 embeddings in one run before that outweighs the load-time
saving.

## Files

| file | what it is |
|---|---|
| `model.safetensors` | the embedding matrix, `I8`, `[500353, 256]` |
| `vocab.fst` | the Unigram vocab as an FST: token bytes → id |
| `vocab.fst.scores` | `f64` unigram log-probs, one per id, in id order |
| `vocab.fst.norm` | the normalizer / pre-tokenizer spec and decoding metadata |
| `tokenizer.json` | upstream's, unchanged — kept so the FST can be re-derived and verified |
| `config.json` | upstream's, unchanged |

## Quantisation

Symmetric, one global scale (`max|W| / 127`). The scale is deliberately **not**
stored: mean-pooling is linear and model2vec L2-normalises its output, so a
single global factor cancels exactly. Rows can therefore be read straight from
disk as raw `i8`.

This is measured, not assumed. Against `model2vec-rs` reading the f32 original,
over 100 133 text segments extracted from real PDF / DOCX / XLSX / EPUB / ODT /
ODS / ODP / PPTX files:

* token ids: **0 divergences**
* embeddings: worst cosine **0.999994**

Retrieval quality on our internal evaluation is unchanged within noise
(0.570/0.413 vs 0.571/0.414 for f32).

## Caveats

* `[UNK]` handling follows `model2vec-rs` exactly, including that it does *not*
  drop `[UNK]` rows for a Unigram tokenizer (it looks for a string `unk_token`
  field, which Unigram does not emit). If you consume these artifacts with your
  own loader and drop `[UNK]` instead, you will get different vectors on
  unk-heavy text.
* `vocab.fst` maps duplicate tokens to their **last** id, matching how
  `tokenizers` builds its `token_to_ids` map.
* The FST/score/norm layout is ours, not a model2vec standard. Anything reading
  only `model.safetensors` + `tokenizer.json` (including model2vec itself) works
  with this repo unmodified — the i8 tensor is a standard safetensors dtype.

## License

MIT, inherited from the upstream model.