File size: 2,586 Bytes
a3e353d 720d457 a3e353d 720d457 a3e353d 720d457 a3e353d 720d457 a3e353d 720d457 a3e353d 720d457 a3e353d 720d457 a3e353d 720d457 a3e353d 720d457 a3e353d 720d457 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | ---
license: mit
library_name: transformers.js
pipeline_tag: feature-extraction
base_model: intfloat/multilingual-e5-small
language:
- multilingual
- en
- ja
- zh
tags:
- decidekit
- system-one
- text-classification
- zero-shot-classification
- calibration
- onnx
- on-device
thumbnail: https://huggingface.co/debuffjp/decide-multilingual/resolve/main/assets/banner.png
---
# Decide Multilingual
Decide reads a short piece of text and picks one of your labels, on the user's device, with a confidence you can act on. This is the model for languages other than English, in [decidekit](https://debuff.jp/decide/).
[Docs](https://debuff.jp/decide/docs/) · [Try it in your browser](https://debuff.jp/decide/playground/) · [GitHub](https://github.com/debuffjp/decidekit) · [npm](https://www.npmjs.com/package/decidekit)
## Use it
```bash
npm install decidekit
```
```ts
import { createDecider } from "decidekit";
const d = await createDecider({ model: "debuffjp/decide-multilingual" });
const { choice, confidence } = await d.decide("カードが届きません", ["card_arrival", "refund"]);
```
The files download from this repo on first use, then stay in the browser's cache. To serve them yourself, copy this repo's files to your own host and pass `modelHost` to `createDecider()`.
## Numbers
| | |
|---|---|
| MASSIVE intents, Japanese and Chinese, all training data | 76% |
| The same with Decide Small | 55% Japanese, 40% Chinese |
| Zero-shot calibration error, Japanese | 0.027 |
| Download at 8-bit | about 118 MB |
Scripts: [eval/multilingual.ts](https://github.com/debuffjp/decidekit/blob/main/eval/multilingual.ts), [eval/e5-settings.ts](https://github.com/debuffjp/decidekit/blob/main/eval/e5-settings.ts).
## Limits
- More than three times the download of Decide Small, so prefer that for English.
- `withFallback`'s off-topic check (`noneBelow`) never fires with this model.
- About a paragraph at a time. Split longer text.
## What's in this repo
This repo holds the encoder, the part that turns text into vectors. decidekit adds the trained head and the calibrated confidence.
The weights are an unmodified copy of [Xenova/multilingual-e5-small](https://huggingface.co/Xenova/multilingual-e5-small) at revision `761b726`, the ONNX conversion of [intfloat/multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small). We did not train them. They are here so decidekit has a stable, pinned source. MIT licensed, like the original.
|