Sentence Similarity
sentence-transformers
Safetensors
Dutch
xlm-roberta
trimmed
text-embeddings-inference
Instructions to use alphaedge-ai/multilingual-e5-base-nld-32768 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use alphaedge-ai/multilingual-e5-base-nld-32768 with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("alphaedge-ai/multilingual-e5-base-nld-32768") sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,20 +1,20 @@
|
|
| 1 |
---
|
| 2 |
-
pipeline_tag:
|
| 3 |
language: nld
|
| 4 |
license: mit
|
| 5 |
tags:
|
| 6 |
-
|
| 7 |
-
library_name: transformers
|
| 8 |
base_model: intfloat/multilingual-e5-base
|
| 9 |
base_model_relation: quantized
|
| 10 |
datasets:
|
| 11 |
-
|
| 12 |
---
|
| 13 |
|
| 14 |
# multilingual-e5-base-nld-32768
|
| 15 |
|
| 16 |
This model is a 60.0% smaller version of [intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base)
|
| 17 |
-
optimized for
|
| 18 |
|
| 19 |
**Total vocabulary size**: 32768 tokens (reduced from 250002)
|
| 20 |
**Tokenizer type**: Unigram
|
|
@@ -25,18 +25,14 @@ optimized for multiple languages with vocabulary pruning.
|
|
| 25 |
|
| 26 |
- **nld**: 32768 tokens
|
| 27 |
|
| 28 |
-
This pruned model should perform similarly to the original model for
|
| 29 |
memory footprint. However, it may not perform well for other languages as tokens not commonly used in the selected
|
| 30 |
languages were removed from the vocabulary.
|
| 31 |
|
| 32 |
## Usage
|
| 33 |
|
| 34 |
-
You can use this model with the
|
| 35 |
```python
|
| 36 |
-
from
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
model = AutoModel.from_pretrained(model_name)
|
| 40 |
-
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| 41 |
-
```
|
| 42 |
-
|
|
|
|
| 1 |
---
|
| 2 |
+
pipeline_tag: sentence-similarity
|
| 3 |
language: nld
|
| 4 |
license: mit
|
| 5 |
tags:
|
| 6 |
+
- trimmed
|
| 7 |
+
library_name: sentence-transformers
|
| 8 |
base_model: intfloat/multilingual-e5-base
|
| 9 |
base_model_relation: quantized
|
| 10 |
datasets:
|
| 11 |
+
- Lumberjackk/fineweb-2-trimming
|
| 12 |
---
|
| 13 |
|
| 14 |
# multilingual-e5-base-nld-32768
|
| 15 |
|
| 16 |
This model is a 60.0% smaller version of [intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base)
|
| 17 |
+
optimized for Dutch language via vocabulary pruning.
|
| 18 |
|
| 19 |
**Total vocabulary size**: 32768 tokens (reduced from 250002)
|
| 20 |
**Tokenizer type**: Unigram
|
|
|
|
| 25 |
|
| 26 |
- **nld**: 32768 tokens
|
| 27 |
|
| 28 |
+
This pruned model should perform similarly to the original model for Dutch with a much smaller
|
| 29 |
memory footprint. However, it may not perform well for other languages as tokens not commonly used in the selected
|
| 30 |
languages were removed from the vocabulary.
|
| 31 |
|
| 32 |
## Usage
|
| 33 |
|
| 34 |
+
You can use this model with the SentenceTransformer library:
|
| 35 |
```python
|
| 36 |
+
from sentence_transformers import SentenceTransformer
|
| 37 |
+
model = SentenceTransformer("Lumberjackk/multilingual-e5-base-nld-32768")
|
| 38 |
+
```
|
|
|
|
|
|
|
|
|
|
|
|