Hugging Face's logo

allegro
/
BiDi-ces-eng

Translation
Transformers
Safetensors
PyTorch
marian
text2text-generation
mt
sentence-piece
multilingual
allegro
laniqo
Model card Files
xet
 Community
1

Add paper link and pipeline tag to model card

#1
by nielsr HF Staff - opened
base: refs/heads/main
←
from: refs/pr/1
Discussion Files changed
+5
-150
Files changed (1) hide show
  1. README.md +5 -150
README.md CHANGED
@@ -1,5 +1,4 @@
1
  ---
2
- license: cc-by-4.0
3
  language:
4
  - cs
5
  - pl
@@ -7,6 +6,7 @@ language:
7
  - sl
8
  - en
9
  library_name: transformers
 
10
  tags:
11
  - translation
12
  - mt
@@ -16,6 +16,8 @@ tags:
16
  - multilingual
17
  - allegro
18
  - laniqo
 
 
19
  ---
20
 
21
  # MultiSlav BiDi Models
@@ -30,7 +32,7 @@ tags:
30
  ___BiDi___ is a collection of Encoder-Decoder vanilla transformer models trained on sentence-level Machine Translation task.
31
  Each model is supporting Bi-Directional translation.
32
 
33
- ___BiDi___ models are part of the [___MultiSlav___ collection](https://huggingface.co/collections/allegro/multislav-6793d6b6419e5963e759a683). More information will be available soon in our upcoming MultiSlav paper.
34
 
35
  Experiments were conducted under research project by [Machine Learning Research](https://ml.allegro.tech/) lab for [Allegro.com](https://ml.allegro.tech/).
36
  Big thanks to [laniqo.com](laniqo.com) for cooperation in the research.
@@ -113,151 +115,4 @@ During the training we used the [MarianNMT](https://marian-nmt.github.io/) frame
113
  Base marian configuration used: [transfromer-big](https://github.com/marian-nmt/marian-dev/blob/master/src/common/aliases.cpp#L113).
114
  All training parameters are listed in table below.
115
 
116
- ### Training hyperparameters:
117
-
118
- | **Hyperparameter** | **Value** |
119
- |----------------------------|------------------------------------------------------------------------------------------------------------|
120
- | Total Parameter Size | 209M |
121
- | Vocab Size | 32k |
122
- | Base Parameters | [Marian transfromer-big](https://github.com/marian-nmt/marian-dev/blob/master/src/common/aliases.cpp#L113) |
123
- | Number of Encoding Layers | 6 |
124
- | Number of Decoding Layers | 6 |
125
- | Model Dimension | 1024 |
126
- | FF Dimension | 4096 |
127
- | Heads | 16 |
128
- | Dropout | 0.1 |
129
- | Batch Size | mini batch fit to VRAM |
130
- | Training Accelerators | 4x A100 40GB |
131
- | Max Length | 100 tokens |
132
- | Optimizer | Adam |
133
- | Warmup steps | 8000 |
134
- | Context | Sentence-level MT |
135
- | Languages Supported | See [Bi-Di models available](#Bi-Di-models-available) |
136
- | Precision | float16 |
137
- | Validation Freq | 3000 steps |
138
- | Stop Metric | ChrF |
139
- | Stop Criterion | 20 Validation steps |
140
-
141
-
142
- ## Training corpora
143
-
144
- The main research question was: "How does adding additional, related languages impact the quality of the model?" - we explored it in the Slavic language family.
145
- ___BiDi___ models are our baseline before expanding the data-regime by using higher-level multilinguality.
146
-
147
- Datasets were downloaded via [MT-Data](https://pypi.org/project/mtdata/0.2.10/) library.
148
- The number of total examples post filtering and deduplication varies, depending on languages supported, see the table below.
149
-
150
- | **Language pair** | **Number of training examples** |
151
- |-------------------|--------------------------------:|
152
- | Czech ↔ Polish | 63M |
153
- | Czech ↔ Slovak | 30M |
154
- | Czech ↔ Slovene | 25M |
155
- | Polish ↔ Slovak | 26M |
156
- | Polish ↔ Slovene | 23M |
157
- | Slovak ↔ Slovene | 18M |
158
- | ---------------- | ------------------------------- |
159
- | Czech ↔ English | 151M |
160
- | English ↔ Polish | 150M |
161
- | English ↔ Slovak | 52M |
162
- | English ↔ Slovene | 40M |
163
-
164
- The datasets used (only applicable to specific directions):
165
-
166
- | **Corpus** |
167
- |----------------------|
168
- | paracrawl |
169
- | opensubtitles |
170
- | multiparacrawl |
171
- | dgt |
172
- | elrc |
173
- | xlent |
174
- | wikititles |
175
- | wmt |
176
- | wikimatrix |
177
- | dcep |
178
- | ELRC |
179
- | tildemodel |
180
- | europarl |
181
- | eesc |
182
- | eubookshop |
183
- | emea |
184
- | jrc_acquis |
185
- | ema |
186
- | qed |
187
- | elitr_eca |
188
- | EU-dcep |
189
- | rapid |
190
- | ecb |
191
- | kde4 |
192
- | news_commentary |
193
- | kde |
194
- | bible_uedin |
195
- | europat |
196
- | elra |
197
- | wikipedia |
198
- | wikimedia |
199
- | tatoeba |
200
- | globalvoices |
201
- | euconst |
202
- | ubuntu |
203
- | php |
204
- | ecdc |
205
- | eac |
206
- | eac_reference |
207
- | gnome |
208
- | EU-eac |
209
- | books |
210
- | EU-ecdc |
211
- | newsdev |
212
- | khresmoi_summary |
213
- | czechtourism |
214
- | khresmoi_summary_dev |
215
- | worldbank |
216
-
217
- ## Evaluation
218
-
219
- Evaluation of the models was performed on [Flores200](https://huggingface.co/datasets/facebook/flores) dataset.
220
- The table below compares performance of the open-source models and all applicable models from our collection.
221
- Metric used: Unbabel/wmt22-comet-da.
222
-
223
- | **Direction** | **CES → ENG** | **CES → POL** | **CES → SLK** | **CES → SLV** | **ENG → CES** | **ENG → POL** | **ENG → SLK** | **ENG → SLV** | **POL → CES** | **POL → ENG** | **POL → SLK** | **POL → SLV** | **SLK → CES** | **SLK → ENG** | **SLK → POL** | **SLK → SLV** | **SLV → CES** | **SLV → ENG** | **SLV → POL** | **SLV → SLK** |
224
- |----------------------------------------------------|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|
225
- | **M2M-100** | 87.0 | 89.0 | 92.1 | 89.7 | 88.6 | 86.4 | 88.4 | 87.3 | 89.6 | 84.6 | 89.4 | 88.4 | 92.7 | 86.8 | 89.1 | 89.6 | 90.3 | 86.4 | 88.7 | 90.1 |
226
- | **NLLB-200** | 88.1 | 88.9 | 91.2 | 88.6 | 90.4 | __88.5__ | 90.1 | 88.8 | 89.4 | __85.8__ | 88.9 | 87.7 | 91.8 | 88.2 | 88.9 | 88.8 | 90.0 | __87.5__ | 88.6 | 89.4 |
227
- | **Seamless-M4T** | 87.5 | 80.9 | 90.8 | 82.0 | __90.7__ | __88.5__ | __90.6__ | __89.6__ | 79.6 | 85.4 | 80.0 | 76.4 | 91.5 | 87.2 | 81.2 | 82.9 | 80.9 | 87.3 | 76.7 | 81.0 |
228
- | **OPUS-MT Sla-Sla** | __88.2__ | 82.8 | - | 83.4 | 89.1 | 85.6 | - | 84.5 | 82.9 | 82.2 | - | 81.2 | - | - | - | - | 83.5 | 84.1 | 80.8 | - |
229
- | **OPUS-MT SK-EN** | - | - | - | - | - | - | 89.5 | - | - | - | - | - | - | __88.4__ | - | - | - | - | - | - |
230
- | _Our contributions:_ | | | | | | | | | | | | | | | | | | | | |
231
- | **BiDi Models**<span style="color:green;">*</span> | 87.5 | 89.4 | 92.4 | 89.8 | 87.8 | 86.2 | 87.2 | 86.6 | 90.0 | 85.0 | 89.1 | 88.4 | 92.9 | 87.3 | 88.8 | 89.4 | 90.0 | 86.9 | 88.1 | 89.1 |
232
- | **P4-pol**<span style="color:red;">◊</span> | - | 89.6 | 90.8 | 88.7 | - | - | - | - | 90.2 | - | 89.8 | 88.7 | 91.0 | - | 89.3 | 88.4 | 89.3 | - | 88.7 | 88.5 |
233
- | **P5-eng**<span style="color:red;">◊</span> | 88.0 | 89.0 | 90.7 | 89.0 | 88.8 | 87.3 | 88.4 | 87.5 | 89.0 | 85.7 | 88.5 | 87.8 | 91.0 | 88.2 | 88.6 | 88.5 | 89.6 | 87.2 | 88.4 | 88.9 |
234
- | **P5-ces**<span style="color:red;">◊</span> | 87.9 | 89.6 | __92.5__ | 89.9 | 88.4 | 85.0 | 87.9 | 85.9 | 90.3 | 84.5 | 89.5 | 88.0 | __93.0__ | 87.8 | 89.4 | 89.8 | 90.3 | 85.7 | 87.9 | 89.8 |
235
- | **MultiSlav-4slav** | - | 89.7 | __92.5__ | 90.0 | - | - | - | - | 90.2 | - | 89.6 | 88.7 | 92.9 | - | 89.4 | 90.1 | __90.6__ | - | 88.9 | __90.2__ |
236
- | **MultiSlav-5lang** | 87.8 | __89.8__ | __92.5__ | __90.1__ | 88.9 | 86.9 | 88.0 | 87.3 | __90.4__ | 85.4 | 89.8 | __88.9__ | 92.9 | 87.8 | __89.6__ | __90.2__ | __90.6__ | 87.0 | __89.2__ | __90.2__ |
237
-
238
- <span style="color:red;">◊</span> system of 2 models *Many2XXX* and *XXX2Many*, see [P5-ces2many](https://huggingface.co/allegro/p5-ces2many)
239
-
240
- <span style="color:green;">*</span> results combined for all bi-directional models; each values for applicable model
241
-
242
- ## Limitations and Biases
243
-
244
- We did not evaluate inherent bias contained in training datasets. It is advised to validate bias of our models in perspective domain. This might be especially problematic in translation from English to Slavic languages, which require explicitly indicated gender and might hallucinate based on bias present in training data.
245
-
246
- ## License
247
-
248
- The model is licensed under CC BY 4.0, which allows for commercial use.
249
-
250
- ## Citation
251
- TO BE UPDATED SOON 🤗
252
-
253
-
254
-
255
- ## Contact Options
256
-
257
- Authors:
258
- - MLR @ Allegro: [Artur Kot](https://linkedin.com/in/arturkot), [Mikołaj Koszowski](https://linkedin.com/in/mkoszowski), [Wojciech Chojnowski](https://linkedin.com/in/wojciech-chojnowski-744702348), [Mieszko Rutkowski](https://linkedin.com/in/mieszko-rutkowski)
259
- - Laniqo.com: [Artur Nowakowski](https://linkedin.com/in/artur-nowakowski-mt), [Kamil Guttmann](https://linkedin.com/in/kamil-guttmann), [Mikołaj Pokrywka](https://linkedin.com/in/mikolaj-pokrywka)
260
-
261
- Please don't hesitate to contact authors if you have any questions or suggestions:
262
- - e-mail: artur.kot@allegro.com or mikolaj.koszowski@allegro.com
263
- - LinkedIn: [Artur Kot](https://linkedin.com/in/arturkot) or [Mikołaj Koszowski](https://linkedin.com/in/mkoszowski)
 
1
  ---
 
2
  language:
3
  - cs
4
  - pl
 
6
  - sl
7
  - en
8
  library_name: transformers
9
+ license: cc-by-4.0
10
  tags:
11
  - translation
12
  - mt
 
16
  - multilingual
17
  - allegro
18
  - laniqo
19
+ pipeline_tag: translation
20
+ paper: https://hf.co/papers/2502.14509
21
  ---
22
 
23
  # MultiSlav BiDi Models
 
32
  ___BiDi___ is a collection of Encoder-Decoder vanilla transformer models trained on sentence-level Machine Translation task.
33
  Each model is supporting Bi-Directional translation.
34
 
35
+ ___BiDi___ models are part of the [___MultiSlav___ collection](https://huggingface.co/collections/allegro/multislav-6793d6b6419e5963e759a683). More information will be available soon in our upcoming MultiSlav paper. [Paper](https://hf.co/papers/2502.14509)
36
 
37
  Experiments were conducted under research project by [Machine Learning Research](https://ml.allegro.tech/) lab for [Allegro.com](https://ml.allegro.tech/).
38
  Big thanks to [laniqo.com](laniqo.com) for cooperation in the research.
 
115
  Base marian configuration used: [transfromer-big](https://github.com/marian-nmt/marian-dev/blob/master/src/common/aliases.cpp#L113).
116
  All training parameters are listed in table below.
117
 
118
+ ### Training hyperparameters: