File size: 3,812 Bytes
2d0856a bf0442b | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 | ---
library_name: keras-hub
---
### Model Overview
## Model Summary
Multilingual E5 is a family of multilingual text embedding models from [intfloat ](https://huggingface.co/intfloat), based on the XLM-RoBERTa architecture. These models generate fixed-size sentence embeddings suitable for semantic search, retrieval, clustering, and text classification across 100+ languages.
The E5 models were introduced in ["Multilingual E5 Text Embeddings: A Technical Report"](https://arxiv.org/abs/2402.05672) by Wang et al. The models are trained in two stages: (1) contrastive pre-training on ~1 billion multilingual text pairs, and (2) fine-tuning on labeled datasets. Input text should be prefixed with "query: " or "passage: " for optimal performance.
## Key Features:
- Supports 100+ languages via XLM-RoBERTa backbone
- Trained on ~1 billion multilingual text pairs
- MIT licensed
- Works out of the box for semantic search, retrieval, clustering, and classification
- Model Family: Multilingual E5
- Architecture: XLM-RoBERTa (bidirectional Transformer encoder)
- Languages: 100+
- Task Type: Text Embedding / Sentence Similarity / Retrieval
- Max Sequence Length: 512 tokens
## Model Details
* [Multilingual E5 Quickstart Notebook](coming soon..)
* [Multilingual E5 API Documentation](https://keras.io/keras_hub/api/models/xlm_roberta/)
* [Multilingual E5 Model Card](https://huggingface.co/intfloat/multilingual-e5-small)
* [Multilingual E5 Technical Paper](https://arxiv.org/abs/2402.05672)
* [KerasHub Beginner Guide](https://keras.io/guides/keras_hub/getting_started/)
* [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/)
## Installation
Keras and KerasHub can be installed with:
```
pip install -U -q keras-hub
pip install -U -q keras
```
Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instructions on installing them in another environment see the [Keras Getting Started](https://keras.io/getting_started/) page.
## Presets
## Preset Table
| Preset | Architecture | Pooling | Normalize | Languages | Description |
|---|---|---|---|---|---|
| `multilingual_e5_small` | XLM-RoBERTa | Mean | L2 | 100+ | 12-layer Multilingual E5 small model. Produces 384-dim embeddings. |
| `multilingual_e5_base` | XLM-RoBERTa | Mean | L2 | 100+ | 12-layer Multilingual E5 base model. Produces 768-dim embeddings. |
| `multilingual_e5_large` | XLM-RoBERTa | Mean | L2 | 100+ | 24-layer Multilingual E5 large model. Produces 1024-dim embeddings. |
## Example Usage
```
import keras_hub
# Load the text embedder with preprocessing.
embedder = keras_hub.models.TextEmbedder.from_preset(
"multilingual_e5_small"
)
# Encode queries
query = "query: Which planet is known as the Red Planet?"
q_emb = embedder.encode_text(query)
# Encode documents/passages
documents = [
"passage: Mars is often referred to as the Red Planet.",
"passage: Venus is often called Earth's twin.",
]
d_embs = embedder.encode_text(documents)
```
```
# Load just the backbone for custom architectures.
backbone = keras_hub.models.XLMRobertaBackbone.from_preset(
"multilingual_e5_small",
)
```
## Example Usage with Hugging Face URI
```
import keras_hub
# Load the text embedder with preprocessing.
embedder = keras_hub.models.TextEmbedder.from_preset(
"hf://keras/multilingual_e5_small"
)
# Encode queries
query = "query: Which planet is known as the Red Planet?"
q_emb = embedder.encode_text(query)
# Encode documents/passages
documents = [
"passage: Mars is often referred to as the Red Planet.",
"passage: Venus is often called Earth's twin.",
]
d_embs = embedder.encode_text(documents)
```
```
# Load just the backbone for custom architectures.
backbone = keras_hub.models.XLMRobertaBackbone.from_preset(
"hf://keras/multilingual_e5_small",
)
```
|