stephantulkens commited on
Commit
a761dcc
·
verified ·
1 Parent(s): 98e5a10

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,169 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: minishlab/potion-multilingual-128m
3
+ language:
4
+ - en
5
+ - multilingual
6
+ - af
7
+ - am
8
+ - ar
9
+ - az
10
+ - be
11
+ - bg
12
+ - bn
13
+ - ca
14
+ - ceb
15
+ - co
16
+ - cs
17
+ - cy
18
+ - da
19
+ - de
20
+ - el
21
+ - eo
22
+ - es
23
+ - et
24
+ - eu
25
+ - fa
26
+ - fi
27
+ - fil
28
+ - fr
29
+ - fy
30
+ - ga
31
+ - gd
32
+ - gl
33
+ - gu
34
+ - ha
35
+ - haw
36
+ - hi
37
+ - hmn
38
+ - ht
39
+ - hu
40
+ - hy
41
+ - id
42
+ - ig
43
+ - is
44
+ - it
45
+ - iw
46
+ - ja
47
+ - jv
48
+ - ka
49
+ - kk
50
+ - km
51
+ - kn
52
+ - ko
53
+ - ku
54
+ - ky
55
+ - la
56
+ - lb
57
+ - lo
58
+ - lt
59
+ - lv
60
+ - mg
61
+ - mi
62
+ - mk
63
+ - ml
64
+ - mn
65
+ - mr
66
+ - ms
67
+ - mt
68
+ - my
69
+ - ne
70
+ - nl
71
+ - 'no'
72
+ - ny
73
+ - pa
74
+ - pl
75
+ - ps
76
+ - pt
77
+ - ro
78
+ - ru
79
+ - sd
80
+ - si
81
+ - sk
82
+ - sl
83
+ - sm
84
+ - sn
85
+ - so
86
+ - sq
87
+ - sr
88
+ - st
89
+ - su
90
+ - sv
91
+ - sw
92
+ - ta
93
+ - te
94
+ - tg
95
+ - th
96
+ - tr
97
+ - uk
98
+ - ur
99
+ - uz
100
+ - vi
101
+ - xh
102
+ - yi
103
+ - yo
104
+ - zh
105
+ - zu
106
+ library_name: model2vec
107
+ license: mit
108
+ model_name: potion-multilingual-128m-onnx
109
+ tags:
110
+ - embeddings
111
+ - static-embeddings
112
+ - sentence-transformers
113
+ - onnx
114
+ ---
115
+
116
+ # potion-multilingual-128m-onnx Model Card
117
+
118
+ This is an ONNX export of the [minishlab/potion-multilingual-128m](https://huggingface.co/minishlab/potion-multilingual-128m) [Model2Vec](https://github.com/MinishLab/model2vec) model, produced with the [ONNX](https://onnx.ai/) runtime. [Model2Vec](https://github.com/MinishLab/model2vec) models use static embeddings, allowing text embeddings to be computed orders of magnitude faster on both GPU and CPU. This ONNX export lets you run the model with `onnxruntime` or `transformers.js`, without depending on the `model2vec` package.
119
+
120
+ ## Usage
121
+
122
+ ### Using ONNX Runtime
123
+
124
+ ```python
125
+ import onnxruntime as ort
126
+ from transformers import AutoTokenizer
127
+
128
+ tokenizer = AutoTokenizer.from_pretrained("potion-multilingual-128m-onnx")
129
+ session = ort.InferenceSession("model.onnx")
130
+
131
+ encodings = tokenizer(["Example sentence"], padding=True, return_tensors="np")
132
+ embeddings = session.run(None, dict(encodings))[0]
133
+ ```
134
+
135
+ ### Using the original Model2Vec model
136
+
137
+ If you don't need the ONNX runtime, you can load the original model with the [Model2Vec library](https://github.com/MinishLab/model2vec) instead:
138
+ ```python
139
+ from model2vec import StaticModel
140
+
141
+ model = StaticModel.from_pretrained("minishlab/potion-multilingual-128m")
142
+ embeddings = model.encode(["Example sentence"])
143
+ ```
144
+
145
+ ## Additional Resources
146
+
147
+ - [Model2Vec Repo](https://github.com/MinishLab/model2vec)
148
+ - [Model2Vec Base Models](https://huggingface.co/collections/minishlab/model2vec-base-models-66fd9dd9b7c3b3c0f25ca90e)
149
+ - [Model2Vec Results](https://github.com/MinishLab/model2vec/tree/main/results)
150
+ - [Model2Vec Docs](https://minish.ai/packages/model2vec/introduction)
151
+
152
+ ## Library Authors
153
+
154
+ Model2Vec was developed by the [Minish Lab](https://github.com/MinishLab) team consisting of [Stephan Tulkens](https://github.com/stephantul) and [Thomas van Dongen](https://github.com/Pringled).
155
+
156
+ ## Citation
157
+
158
+ Please cite the [Model2Vec repository](https://github.com/MinishLab/model2vec) if you use this model in your work.
159
+ ```
160
+ @software{minishlab2024model2vec,
161
+ author = {Stephan Tulkens and {van Dongen}, Thomas},
162
+ title = {Model2Vec: Fast State-of-the-Art Static Embeddings},
163
+ year = {2024},
164
+ publisher = {Zenodo},
165
+ doi = {10.5281/zenodo.17270888},
166
+ url = {https://github.com/MinishLab/model2vec},
167
+ license = {MIT}
168
+ }
169
+ ```
config.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "pad_token_id": 0
3
+ }
model.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b4e6c98787b3a43d6a29c3ed21fccac74847ab7fa84d77eed5cca8fbe8fa24fb
3
+ size 512372877
special_tokens_map.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "pad_token": "[PAD]"
3
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:07db1d0285c879cd514240759f56cb294c5c19f7a79ea5517e56c350ad65df7b
3
+ size 34630288
tokenizer_config.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "clean_up_tokenization_spaces": false,
4
+ "model_max_length": 32768,
5
+ "pad_to_multiple_of": null,
6
+ "pad_token": "[PAD]",
7
+ "pad_token_type_id": 0,
8
+ "padding_side": "right",
9
+ "tokenizer_class": "TokenizersBackend",
10
+ "unk_token": "[UNK]"
11
+ }