lbourdois commited on
Commit
c7be0fc
·
verified ·
1 Parent(s): d93c4a7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -14
README.md CHANGED
@@ -1,20 +1,20 @@
1
  ---
2
- pipeline_tag: fill-mask
3
  language: nld
4
  license: mit
5
  tags:
6
- - trimmed
7
- library_name: transformers
8
  base_model: intfloat/multilingual-e5-base
9
  base_model_relation: quantized
10
  datasets:
11
- - Lumberjackk/fineweb-2-trimming
12
  ---
13
 
14
  # multilingual-e5-base-nld-32768
15
 
16
  This model is a 60.0% smaller version of [intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base)
17
- optimized for multiple languages with vocabulary pruning.
18
 
19
  **Total vocabulary size**: 32768 tokens (reduced from 250002)
20
  **Tokenizer type**: Unigram
@@ -25,18 +25,14 @@ optimized for multiple languages with vocabulary pruning.
25
 
26
  - **nld**: 32768 tokens
27
 
28
- This pruned model should perform similarly to the original model for the selected languages with a much smaller
29
  memory footprint. However, it may not perform well for other languages as tokens not commonly used in the selected
30
  languages were removed from the vocabulary.
31
 
32
  ## Usage
33
 
34
- You can use this model with the Transformers library:
35
  ```python
36
- from transformers import AutoModel, AutoTokenizer
37
-
38
- model_name = "Lumberjackk/multilingual-e5-base-nld-32768"
39
- model = AutoModel.from_pretrained(model_name)
40
- tokenizer = AutoTokenizer.from_pretrained(model_name)
41
- ```
42
-
 
1
  ---
2
+ pipeline_tag: sentence-similarity
3
  language: nld
4
  license: mit
5
  tags:
6
+ - trimmed
7
+ library_name: sentence-transformers
8
  base_model: intfloat/multilingual-e5-base
9
  base_model_relation: quantized
10
  datasets:
11
+ - Lumberjackk/fineweb-2-trimming
12
  ---
13
 
14
  # multilingual-e5-base-nld-32768
15
 
16
  This model is a 60.0% smaller version of [intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base)
17
+ optimized for Dutch language via vocabulary pruning.
18
 
19
  **Total vocabulary size**: 32768 tokens (reduced from 250002)
20
  **Tokenizer type**: Unigram
 
25
 
26
  - **nld**: 32768 tokens
27
 
28
+ This pruned model should perform similarly to the original model for Dutch with a much smaller
29
  memory footprint. However, it may not perform well for other languages as tokens not commonly used in the selected
30
  languages were removed from the vocabulary.
31
 
32
  ## Usage
33
 
34
+ You can use this model with the SentenceTransformer library:
35
  ```python
36
+ from sentence_transformers import SentenceTransformer
37
+ model = SentenceTransformer("Lumberjackk/multilingual-e5-base-nld-32768")
38
+ ```