| --- |
| license: mit |
| library_name: transformers.js |
| pipeline_tag: feature-extraction |
| base_model: intfloat/multilingual-e5-small |
| language: |
| - multilingual |
| - en |
| - ja |
| - zh |
| tags: |
| - decidekit |
| - system-one |
| - text-classification |
| - zero-shot-classification |
| - calibration |
| - onnx |
| - on-device |
| thumbnail: https://huggingface.co/debuffjp/decide-multilingual/resolve/main/assets/banner.png |
| --- |
| |
|  |
|
|
| # Decide Multilingual |
|
|
| Decide reads a short piece of text and picks one of your labels, on the user's device, with a confidence you can act on. This is the model for languages other than English, in [decidekit](https://debuff.jp/decide/). |
|
|
| [Docs](https://debuff.jp/decide/docs/) · [Try it in your browser](https://debuff.jp/decide/playground/) · [GitHub](https://github.com/debuffjp/decidekit) · [npm](https://www.npmjs.com/package/decidekit) |
|
|
| ## Use it |
|
|
| ```bash |
| npm install decidekit |
| ``` |
|
|
| ```ts |
| import { createDecider } from "decidekit"; |
| |
| const d = await createDecider({ model: "debuffjp/decide-multilingual" }); |
| const { choice, confidence } = await d.decide("カードが届きません", ["card_arrival", "refund"]); |
| ``` |
|
|
| The files download from this repo on first use, then stay in the browser's cache. To serve them yourself, copy this repo's files to your own host and pass `modelHost` to `createDecider()`. |
|
|
| ## Numbers |
|
|
| | | | |
| |---|---| |
| | MASSIVE intents, Japanese and Chinese, all training data | 76% | |
| | The same with Decide Small | 55% Japanese, 40% Chinese | |
| | Zero-shot calibration error, Japanese | 0.027 | |
| | Download at 8-bit | about 118 MB | |
|
|
| Scripts: [eval/multilingual.ts](https://github.com/debuffjp/decidekit/blob/main/eval/multilingual.ts), [eval/e5-settings.ts](https://github.com/debuffjp/decidekit/blob/main/eval/e5-settings.ts). |
|
|
| ## Limits |
|
|
| - More than three times the download of Decide Small, so prefer that for English. |
| - `withFallback`'s off-topic check (`noneBelow`) never fires with this model. |
| - About a paragraph at a time. Split longer text. |
|
|
| ## What's in this repo |
|
|
| This repo holds the encoder, the part that turns text into vectors. decidekit adds the trained head and the calibrated confidence. |
|
|
| The weights are an unmodified copy of [Xenova/multilingual-e5-small](https://huggingface.co/Xenova/multilingual-e5-small) at revision `761b726`, the ONNX conversion of [intfloat/multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small). We did not train them. They are here so decidekit has a stable, pinned source. MIT licensed, like the original. |
|
|
