vrashad commited on
Commit
8b84cfd
·
verified ·
1 Parent(s): dad441e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +139 -1114
README.md CHANGED
@@ -1,1154 +1,179 @@
1
  ---
 
 
 
 
2
  tags:
3
- - sentence-transformers
4
  - sentence-similarity
5
- - feature-extraction
6
- - generated_from_trainer
7
- - dataset_size:8283932
8
- - loss:MSELoss
9
- base_model: sentence-transformers/all-MiniLM-L6-v2
10
- widget:
11
- - source_sentence: Through the Southern Gas Corridor pipeline, gas supply to the European
12
- Union increased from 8.1 billion cubic meters in 2021 to 11.4 billion cubic meters
13
- in 2022.
14
- sentences:
15
- - After this meeting, the monthly amount collected from prosecutors and investigators
16
- for the building was increased from 460 manats to 480 manats.
17
- - Məlik-Aslanov 1919-cu il fevralın 18-dək həm də müvəqqəti olaraq ticarət, sənaye
18
- və ərzaq nazirinin səlahiyyətlərini də yerinə yetirmişdi.
19
- - Üçüncü mərhələdə isə Şura hər bir layihə üzrə təqdim olunmuş ekspert rəyini, QHT-nin
20
- Şuranın maliyyə dəstəyi hesabına əvvəlki illərdə həyata keçirdiyi layihənin icra
21
- vəziyyətini və layihə idarəetmə təcrübəsini nəzərə alaraq yekun qərar qəbul edir.
22
- - source_sentence: '“Azərbaycan Uşaqlar Birliyi”nin sədri Kəmalə Ağazadə isə məsələnin
23
- Elinanın deyil, digər şəxslərin üzərində fokuslanmasının doğru olmadığını bildirdi:
24
- “Elinanın intiharı ilə bağlı məsələ bu gün də sosial şəbəkələrdə xeyli müzakirə
25
- edilir, müxtəlif fikirlər bildirilir.'
26
- sentences:
27
- - 1952-ci ilin aprelindən başlayaraq, "Azərbaycan Kültür Dərnəyi" tərəfindən Ankarada
28
- aylıq "Azərbaycan" jurnalı nəşr olunur.
29
- - G. Məmmədovanın fikrincə abidənin konstruktiv həllinin analizi, kvadrat təməldən
30
- dairəvi dacili və səkkizbucaqlı xarici barabana keçidin yelkənlərlə təmin edilməsinə
31
- əsasən kilsəni təxminən VII-VIII əsrlərə aid etmək mümkündür.
32
- - However, a signature campaign was conducted in the country to hold a referendum
33
- on extending Nursultan Nazarbayev’s term, and nearly 5 million signatures were
34
- collected.
35
- - source_sentence: Thus, we preserve our history, traditions, and culture, and we
36
- do a lot to support each other.
37
- sentences:
38
- - Belə ki, ara yoldan Bakıxanov küçəsinə çıxan “Mercedes”in sürücü Özal Quliyevin
39
- üstünlük nişanının tələbinə əməl etməməsi qəza ilə nəticələnib.
40
- - Bundan başqa, onun sözlərinə görə, OPEK+ razılaşması neft bazarının məhsul artıqlığından
41
- qurtulmasına kömək edib.
42
- - Onun fikrincə, İranın Azərbaycan vilayətləri də “Cənubi Azərbaycan” olmalıdır.
43
- - source_sentence: It's true that, although Shahriyar, who is in the top four alongside
44
- Aronyan in the rankings, couldn't win this match.
45
- sentences:
46
- - After spending a year in exile, his father Sultan Abdul Hamid sent him to Istanbul
47
- along with his sisters Ayşe Sultan and Şadiye Sultan, and asked his brother Sultan
48
- Reşad to arrange their marriages.
49
- - Bu, ilk dəfədir ki ABŞ hərbi qüvvələri Rusiyanın keçən ay gizli olaraq raketlər
50
- yerləşdirilməsini ictimai şəkildə təsdiq edir.
51
- - He noted that the Supreme Court held seven sessions, thoroughly reviewed the lower
52
- court’s investigation, and upheld the death sentence.
53
- - source_sentence: At the same time, it is no secret that Washington’s strategic plans
54
- for the Middle East include changing the current Iranian regime, which opposes
55
- Western interests in the region.
56
- sentences:
57
- - Sürücü Ə.Nəzərovla maşındakı digər sərnişinlər Rahim Mahmudov və Anar Bayramov
58
- isə müxtəlif dərəcəli bədən xəsarətləri ilə Lənkəran Mərkəzi Rayon Xəstəxanasına
59
- yerləşdirilib.
60
- - In addition, Turkey was demanding the territory that included the districts of
61
- Akhaltsikhe, Akhalkalaki, Alexandropol (Gyumri), Surmali, and Nakhchivan.
62
- - Bu vəziyyət kilsə meydanını düzəltdiyindən və qolları bərabər uzunluqda olan xaç
63
- planı aydınlaşmadığı üçün bu plan növü qapalı yunan xaçı planı adlandırılır.
64
  pipeline_tag: sentence-similarity
65
- library_name: sentence-transformers
 
 
 
 
 
 
 
 
 
 
 
 
 
66
  ---
67
 
68
- # SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2
69
 
70
- This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
 
71
 
72
- ## Model Details
73
-
74
- ### Model Description
75
- - **Model Type:** Sentence Transformer
76
- - **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) <!-- at revision c9745ed1d9f207416be6d2e6f8de32d1f16199bf -->
77
- - **Maximum Sequence Length:** 512 tokens
78
- - **Output Dimensionality:** 384 dimensions
79
- - **Similarity Function:** Cosine Similarity
80
- <!-- - **Training Dataset:** Unknown -->
81
- <!-- - **Language:** Unknown -->
82
- <!-- - **License:** Unknown -->
83
-
84
- ### Model Sources
85
-
86
- - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
87
- - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
88
- - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
89
-
90
- ### Full Model Architecture
91
-
92
- ```
93
- SentenceTransformer(
94
- (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
95
- (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
96
- (2): Normalize()
97
- )
98
- ```
99
-
100
- ## Usage
101
 
102
- ### Direct Usage (Sentence Transformers)
103
 
104
- First install the Sentence Transformers library:
105
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
106
  ```bash
107
  pip install -U sentence-transformers
108
  ```
109
 
110
- Then you can load this model and run inference.
111
  ```python
112
  from sentence_transformers import SentenceTransformer
113
 
114
- # Download from the 🤗 Hub
115
- model = SentenceTransformer("sentence_transformers_model_id")
116
- # Run inference
117
- sentences = [
118
- 'At the same time, it is no secret that Washington’s strategic plans for the Middle East include changing the current Iranian regime, which opposes Western interests in the region.',
119
- 'In addition, Turkey was demanding the territory that included the districts of Akhaltsikhe, Akhalkalaki, Alexandropol (Gyumri), Surmali, and Nakhchivan.',
120
- 'Sürücü Ə.Nəzərovla maşındakı digər sərnişinlər Rahim Mahmudov Anar Bayramov isə müxtəlif dərəcəli bədən xəsarətləri ilə Lənkəran Mərkəzi Rayon Xəstəxanasına yerləşdirilib.',
 
 
 
 
 
 
 
 
 
 
 
 
121
  ]
122
- embeddings = model.encode(sentences)
123
- print(embeddings.shape)
124
- # [3, 384]
125
-
126
- # Get the similarity scores for the embeddings
127
- similarities = model.similarity(embeddings, embeddings)
128
- print(similarities.shape)
129
- # [3, 3]
130
- ```
131
-
132
- <!--
133
- ### Direct Usage (Transformers)
134
-
135
- <details><summary>Click to see the direct usage in Transformers</summary>
136
-
137
- </details>
138
- -->
139
-
140
- <!--
141
- ### Downstream Usage (Sentence Transformers)
142
-
143
- You can finetune this model on your own dataset.
144
 
145
- <details><summary>Click to expand</summary>
146
-
147
- </details>
148
- -->
149
-
150
- <!--
151
- ### Out-of-Scope Use
152
-
153
- *List how the model may foreseeably be misused and address what users ought not to do with the model.*
154
- -->
155
 
156
- <!--
157
- ## Bias, Risks and Limitations
 
 
 
 
 
 
 
 
 
 
158
 
159
- *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
160
- -->
161
 
162
- <!--
163
- ### Recommendations
164
 
165
- *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
166
- -->
167
 
168
- ## Training Details
 
 
169
 
170
- ### Training Dataset
171
 
172
- #### Unnamed Dataset
173
 
174
- * Size: 8,283,932 training samples
175
- * Columns: <code>sentence_0</code> and <code>label</code>
176
- * Approximate statistics based on the first 1000 samples:
177
- | | sentence_0 | label |
178
- |:--------|:---------------------------------------------------------------------------------|:-------------------------------------|
179
- | type | string | list |
180
- | details | <ul><li>min: 4 tokens</li><li>mean: 29.8 tokens</li><li>max: 89 tokens</li></ul> | <ul><li>size: 384 elements</li></ul> |
181
- * Samples:
182
- | sentence_0 | label |
183
- |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------|
184
- | <code>“Biz “Hizbullah”a axan maliyyə dəstəyini dayandırmaq istəyirik və bu məqsədlə ABŞ hökuməti tutarlı məlumat qarşılığında 10 milyon dollaradək mükafat verməklə yanaşı digər tədbirlər də görəcək”, - Evanoff belə deyib.</code> | <code>[-0.022054675966501236, 0.0932646170258522, -0.01854480803012848, -0.025271562859416008, 0.028432276099920273, ...]</code> |
185
- | <code>Bu dövləti bu gün müxalifətdə olanlar quranda Əli Həsənovun harada nə işlə məşğul olduğu bəlli deyildi.</code> | <code>[-0.012831359170377254, 0.022371841594576836, -0.0271938294172287, 0.09667906910181046, 0.009270057082176208, ...]</code> |
186
- | <code>APA-nın “Hürriyet” qəzetinə istinadən verdiyi məlumata görə, ABŞ Hərbi Hava Qüvvələrinn Komandanlığı ən son 1991-ci ildə Körfəz savaşında istifadə edilmiş B-52 təyyarələrinin Qətərə göndərildiyini açıqlayıb.</code> | <code>[-0.01321476697921753, 0.06281372904777527, 0.005026344675570726, -0.004140781704336405, 0.04239720478653908, ...]</code> |
187
- * Loss: [<code>MSELoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#mseloss)
188
 
189
  ### Training Hyperparameters
190
- #### Non-Default Hyperparameters
191
-
192
- - `per_device_train_batch_size`: 64
193
- - `per_device_eval_batch_size`: 64
194
- - `multi_dataset_batch_sampler`: round_robin
195
-
196
- #### All Hyperparameters
197
- <details><summary>Click to expand</summary>
198
 
199
- - `overwrite_output_dir`: False
200
- - `do_predict`: False
201
- - `eval_strategy`: no
202
- - `prediction_loss_only`: True
203
- - `per_device_train_batch_size`: 64
204
- - `per_device_eval_batch_size`: 64
205
- - `per_gpu_train_batch_size`: None
206
- - `per_gpu_eval_batch_size`: None
207
- - `gradient_accumulation_steps`: 1
208
- - `eval_accumulation_steps`: None
209
- - `torch_empty_cache_steps`: None
210
- - `learning_rate`: 5e-05
211
- - `weight_decay`: 0.0
212
- - `adam_beta1`: 0.9
213
- - `adam_beta2`: 0.999
214
- - `adam_epsilon`: 1e-08
215
- - `max_grad_norm`: 1
216
- - `num_train_epochs`: 3
217
- - `max_steps`: -1
218
- - `lr_scheduler_type`: linear
219
- - `lr_scheduler_kwargs`: {}
220
- - `warmup_ratio`: 0.0
221
- - `warmup_steps`: 0
222
- - `log_level`: passive
223
- - `log_level_replica`: warning
224
- - `log_on_each_node`: True
225
- - `logging_nan_inf_filter`: True
226
- - `save_safetensors`: True
227
- - `save_on_each_node`: False
228
- - `save_only_model`: False
229
- - `restore_callback_states_from_checkpoint`: False
230
- - `no_cuda`: False
231
- - `use_cpu`: False
232
- - `use_mps_device`: False
233
- - `seed`: 42
234
- - `data_seed`: None
235
- - `jit_mode_eval`: False
236
- - `use_ipex`: False
237
- - `bf16`: False
238
- - `fp16`: False
239
- - `fp16_opt_level`: O1
240
- - `half_precision_backend`: auto
241
- - `bf16_full_eval`: False
242
- - `fp16_full_eval`: False
243
- - `tf32`: None
244
- - `local_rank`: 0
245
- - `ddp_backend`: None
246
- - `tpu_num_cores`: None
247
- - `tpu_metrics_debug`: False
248
- - `debug`: []
249
- - `dataloader_drop_last`: False
250
- - `dataloader_num_workers`: 0
251
- - `dataloader_prefetch_factor`: None
252
- - `past_index`: -1
253
- - `disable_tqdm`: False
254
- - `remove_unused_columns`: True
255
- - `label_names`: None
256
- - `load_best_model_at_end`: False
257
- - `ignore_data_skip`: False
258
- - `fsdp`: []
259
- - `fsdp_min_num_params`: 0
260
- - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
261
- - `fsdp_transformer_layer_cls_to_wrap`: None
262
- - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
263
- - `deepspeed`: None
264
- - `label_smoothing_factor`: 0.0
265
- - `optim`: adamw_torch
266
- - `optim_args`: None
267
- - `adafactor`: False
268
- - `group_by_length`: False
269
- - `length_column_name`: length
270
- - `ddp_find_unused_parameters`: None
271
- - `ddp_bucket_cap_mb`: None
272
- - `ddp_broadcast_buffers`: False
273
- - `dataloader_pin_memory`: True
274
- - `dataloader_persistent_workers`: False
275
- - `skip_memory_metrics`: True
276
- - `use_legacy_prediction_loop`: False
277
- - `push_to_hub`: False
278
- - `resume_from_checkpoint`: None
279
- - `hub_model_id`: None
280
- - `hub_strategy`: every_save
281
- - `hub_private_repo`: None
282
- - `hub_always_push`: False
283
- - `gradient_checkpointing`: False
284
- - `gradient_checkpointing_kwargs`: None
285
- - `include_inputs_for_metrics`: False
286
- - `include_for_metrics`: []
287
- - `eval_do_concat_batches`: True
288
- - `fp16_backend`: auto
289
- - `push_to_hub_model_id`: None
290
- - `push_to_hub_organization`: None
291
- - `mp_parameters`:
292
- - `auto_find_batch_size`: False
293
- - `full_determinism`: False
294
- - `torchdynamo`: None
295
- - `ray_scope`: last
296
- - `ddp_timeout`: 1800
297
- - `torch_compile`: False
298
- - `torch_compile_backend`: None
299
- - `torch_compile_mode`: None
300
- - `include_tokens_per_second`: False
301
- - `include_num_input_tokens_seen`: False
302
- - `neftune_noise_alpha`: None
303
- - `optim_target_modules`: None
304
- - `batch_eval_metrics`: False
305
- - `eval_on_start`: False
306
- - `use_liger_kernel`: False
307
- - `eval_use_gather_object`: False
308
- - `average_tokens_across_devices`: False
309
- - `prompts`: None
310
- - `batch_sampler`: batch_sampler
311
- - `multi_dataset_batch_sampler`: round_robin
312
 
313
- </details>
314
 
315
- ### Training Logs
316
- <details><summary>Click to expand</summary>
317
 
318
- | Epoch | Step | Training Loss |
319
- |:------:|:------:|:-------------:|
320
- | 0.0039 | 500 | 0.0035 |
321
- | 0.0077 | 1000 | 0.0024 |
322
- | 0.0116 | 1500 | 0.0022 |
323
- | 0.0155 | 2000 | 0.002 |
324
- | 0.0193 | 2500 | 0.0019 |
325
- | 0.0232 | 3000 | 0.0019 |
326
- | 0.0270 | 3500 | 0.0018 |
327
- | 0.0309 | 4000 | 0.0018 |
328
- | 0.0348 | 4500 | 0.0017 |
329
- | 0.0386 | 5000 | 0.0017 |
330
- | 0.0425 | 5500 | 0.0016 |
331
- | 0.0464 | 6000 | 0.0016 |
332
- | 0.0502 | 6500 | 0.0016 |
333
- | 0.0541 | 7000 | 0.0016 |
334
- | 0.0579 | 7500 | 0.0015 |
335
- | 0.0618 | 8000 | 0.0015 |
336
- | 0.0657 | 8500 | 0.0015 |
337
- | 0.0695 | 9000 | 0.0014 |
338
- | 0.0734 | 9500 | 0.0014 |
339
- | 0.0773 | 10000 | 0.0014 |
340
- | 0.0811 | 10500 | 0.0013 |
341
- | 0.0850 | 11000 | 0.0013 |
342
- | 0.0888 | 11500 | 0.0013 |
343
- | 0.0927 | 12000 | 0.0012 |
344
- | 0.0966 | 12500 | 0.0012 |
345
- | 0.1004 | 13000 | 0.0012 |
346
- | 0.1043 | 13500 | 0.0012 |
347
- | 0.1082 | 14000 | 0.0011 |
348
- | 0.1120 | 14500 | 0.0011 |
349
- | 0.1159 | 15000 | 0.0011 |
350
- | 0.1197 | 15500 | 0.0011 |
351
- | 0.1236 | 16000 | 0.0011 |
352
- | 0.1275 | 16500 | 0.001 |
353
- | 0.1313 | 17000 | 0.001 |
354
- | 0.1352 | 17500 | 0.001 |
355
- | 0.1391 | 18000 | 0.001 |
356
- | 0.1429 | 18500 | 0.001 |
357
- | 0.1468 | 19000 | 0.0009 |
358
- | 0.1507 | 19500 | 0.0009 |
359
- | 0.1545 | 20000 | 0.0009 |
360
- | 0.1584 | 20500 | 0.0009 |
361
- | 0.1622 | 21000 | 0.0009 |
362
- | 0.1661 | 21500 | 0.0008 |
363
- | 0.1700 | 22000 | 0.0008 |
364
- | 0.1738 | 22500 | 0.0008 |
365
- | 0.1777 | 23000 | 0.0008 |
366
- | 0.1816 | 23500 | 0.0008 |
367
- | 0.1854 | 24000 | 0.0008 |
368
- | 0.1893 | 24500 | 0.0008 |
369
- | 0.1931 | 25000 | 0.0008 |
370
- | 0.1970 | 25500 | 0.0007 |
371
- | 0.2009 | 26000 | 0.0007 |
372
- | 0.2047 | 26500 | 0.0007 |
373
- | 0.2086 | 27000 | 0.0007 |
374
- | 0.2125 | 27500 | 0.0007 |
375
- | 0.2163 | 28000 | 0.0007 |
376
- | 0.2202 | 28500 | 0.0007 |
377
- | 0.2240 | 29000 | 0.0007 |
378
- | 0.2279 | 29500 | 0.0007 |
379
- | 0.2318 | 30000 | 0.0007 |
380
- | 0.2356 | 30500 | 0.0007 |
381
- | 0.2395 | 31000 | 0.0007 |
382
- | 0.2434 | 31500 | 0.0006 |
383
- | 0.2472 | 32000 | 0.0006 |
384
- | 0.2511 | 32500 | 0.0006 |
385
- | 0.2550 | 33000 | 0.0006 |
386
- | 0.2588 | 33500 | 0.0006 |
387
- | 0.2627 | 34000 | 0.0006 |
388
- | 0.2665 | 34500 | 0.0006 |
389
- | 0.2704 | 35000 | 0.0006 |
390
- | 0.2743 | 35500 | 0.0006 |
391
- | 0.2781 | 36000 | 0.0006 |
392
- | 0.2820 | 36500 | 0.0006 |
393
- | 0.2859 | 37000 | 0.0006 |
394
- | 0.2897 | 37500 | 0.0006 |
395
- | 0.2936 | 38000 | 0.0006 |
396
- | 0.2974 | 38500 | 0.0006 |
397
- | 0.3013 | 39000 | 0.0006 |
398
- | 0.3052 | 39500 | 0.0006 |
399
- | 0.3090 | 40000 | 0.0006 |
400
- | 0.3129 | 40500 | 0.0006 |
401
- | 0.3168 | 41000 | 0.0006 |
402
- | 0.3206 | 41500 | 0.0005 |
403
- | 0.3245 | 42000 | 0.0005 |
404
- | 0.3283 | 42500 | 0.0005 |
405
- | 0.3322 | 43000 | 0.0005 |
406
- | 0.3361 | 43500 | 0.0005 |
407
- | 0.3399 | 44000 | 0.0005 |
408
- | 0.3438 | 44500 | 0.0005 |
409
- | 0.3477 | 45000 | 0.0005 |
410
- | 0.3515 | 45500 | 0.0005 |
411
- | 0.3554 | 46000 | 0.0005 |
412
- | 0.3592 | 46500 | 0.0005 |
413
- | 0.3631 | 47000 | 0.0005 |
414
- | 0.3670 | 47500 | 0.0005 |
415
- | 0.3708 | 48000 | 0.0005 |
416
- | 0.3747 | 48500 | 0.0005 |
417
- | 0.3786 | 49000 | 0.0005 |
418
- | 0.3824 | 49500 | 0.0005 |
419
- | 0.3863 | 50000 | 0.0005 |
420
- | 0.3902 | 50500 | 0.0005 |
421
- | 0.3940 | 51000 | 0.0005 |
422
- | 0.3979 | 51500 | 0.0005 |
423
- | 0.4017 | 52000 | 0.0005 |
424
- | 0.4056 | 52500 | 0.0005 |
425
- | 0.4095 | 53000 | 0.0005 |
426
- | 0.4133 | 53500 | 0.0005 |
427
- | 0.4172 | 54000 | 0.0005 |
428
- | 0.4211 | 54500 | 0.0005 |
429
- | 0.4249 | 55000 | 0.0005 |
430
- | 0.4288 | 55500 | 0.0005 |
431
- | 0.4326 | 56000 | 0.0005 |
432
- | 0.4365 | 56500 | 0.0005 |
433
- | 0.4404 | 57000 | 0.0005 |
434
- | 0.4442 | 57500 | 0.0005 |
435
- | 0.4481 | 58000 | 0.0005 |
436
- | 0.4520 | 58500 | 0.0005 |
437
- | 0.4558 | 59000 | 0.0005 |
438
- | 0.4597 | 59500 | 0.0005 |
439
- | 0.4635 | 60000 | 0.0005 |
440
- | 0.4674 | 60500 | 0.0005 |
441
- | 0.4713 | 61000 | 0.0005 |
442
- | 0.4751 | 61500 | 0.0005 |
443
- | 0.4790 | 62000 | 0.0005 |
444
- | 0.4829 | 62500 | 0.0005 |
445
- | 0.4867 | 63000 | 0.0005 |
446
- | 0.4906 | 63500 | 0.0005 |
447
- | 0.4944 | 64000 | 0.0005 |
448
- | 0.4983 | 64500 | 0.0005 |
449
- | 0.5022 | 65000 | 0.0005 |
450
- | 0.5060 | 65500 | 0.0004 |
451
- | 0.5099 | 66000 | 0.0004 |
452
- | 0.5138 | 66500 | 0.0004 |
453
- | 0.5176 | 67000 | 0.0004 |
454
- | 0.5215 | 67500 | 0.0004 |
455
- | 0.5254 | 68000 | 0.0004 |
456
- | 0.5292 | 68500 | 0.0004 |
457
- | 0.5331 | 69000 | 0.0004 |
458
- | 0.5369 | 69500 | 0.0004 |
459
- | 0.5408 | 70000 | 0.0004 |
460
- | 0.5447 | 70500 | 0.0004 |
461
- | 0.5485 | 71000 | 0.0004 |
462
- | 0.5524 | 71500 | 0.0004 |
463
- | 0.5563 | 72000 | 0.0004 |
464
- | 0.5601 | 72500 | 0.0004 |
465
- | 0.5640 | 73000 | 0.0004 |
466
- | 0.5678 | 73500 | 0.0004 |
467
- | 0.5717 | 74000 | 0.0004 |
468
- | 0.5756 | 74500 | 0.0004 |
469
- | 0.5794 | 75000 | 0.0004 |
470
- | 0.5833 | 75500 | 0.0004 |
471
- | 0.5872 | 76000 | 0.0004 |
472
- | 0.5910 | 76500 | 0.0004 |
473
- | 0.5949 | 77000 | 0.0004 |
474
- | 0.5987 | 77500 | 0.0004 |
475
- | 0.6026 | 78000 | 0.0004 |
476
- | 0.6065 | 78500 | 0.0004 |
477
- | 0.6103 | 79000 | 0.0004 |
478
- | 0.6142 | 79500 | 0.0004 |
479
- | 0.6181 | 80000 | 0.0004 |
480
- | 0.6219 | 80500 | 0.0004 |
481
- | 0.6258 | 81000 | 0.0004 |
482
- | 0.6296 | 81500 | 0.0004 |
483
- | 0.6335 | 82000 | 0.0004 |
484
- | 0.6374 | 82500 | 0.0004 |
485
- | 0.6412 | 83000 | 0.0004 |
486
- | 0.6451 | 83500 | 0.0004 |
487
- | 0.6490 | 84000 | 0.0004 |
488
- | 0.6528 | 84500 | 0.0004 |
489
- | 0.6567 | 85000 | 0.0004 |
490
- | 0.6606 | 85500 | 0.0004 |
491
- | 0.6644 | 86000 | 0.0004 |
492
- | 0.6683 | 86500 | 0.0004 |
493
- | 0.6721 | 87000 | 0.0004 |
494
- | 0.6760 | 87500 | 0.0004 |
495
- | 0.6799 | 88000 | 0.0004 |
496
- | 0.6837 | 88500 | 0.0004 |
497
- | 0.6876 | 89000 | 0.0004 |
498
- | 0.6915 | 89500 | 0.0004 |
499
- | 0.6953 | 90000 | 0.0004 |
500
- | 0.6992 | 90500 | 0.0004 |
501
- | 0.7030 | 91000 | 0.0004 |
502
- | 0.7069 | 91500 | 0.0004 |
503
- | 0.7108 | 92000 | 0.0004 |
504
- | 0.7146 | 92500 | 0.0004 |
505
- | 0.7185 | 93000 | 0.0004 |
506
- | 0.7224 | 93500 | 0.0004 |
507
- | 0.7262 | 94000 | 0.0004 |
508
- | 0.7301 | 94500 | 0.0004 |
509
- | 0.7339 | 95000 | 0.0004 |
510
- | 0.7378 | 95500 | 0.0004 |
511
- | 0.7417 | 96000 | 0.0004 |
512
- | 0.7455 | 96500 | 0.0004 |
513
- | 0.7494 | 97000 | 0.0004 |
514
- | 0.7533 | 97500 | 0.0004 |
515
- | 0.7571 | 98000 | 0.0004 |
516
- | 0.7610 | 98500 | 0.0004 |
517
- | 0.7649 | 99000 | 0.0004 |
518
- | 0.7687 | 99500 | 0.0004 |
519
- | 0.7726 | 100000 | 0.0004 |
520
- | 0.7764 | 100500 | 0.0004 |
521
- | 0.7803 | 101000 | 0.0004 |
522
- | 0.7842 | 101500 | 0.0004 |
523
- | 0.7880 | 102000 | 0.0004 |
524
- | 0.7919 | 102500 | 0.0004 |
525
- | 0.7958 | 103000 | 0.0004 |
526
- | 0.7996 | 103500 | 0.0004 |
527
- | 0.8035 | 104000 | 0.0004 |
528
- | 0.8073 | 104500 | 0.0004 |
529
- | 0.8112 | 105000 | 0.0004 |
530
- | 0.8151 | 105500 | 0.0004 |
531
- | 0.8189 | 106000 | 0.0004 |
532
- | 0.8228 | 106500 | 0.0004 |
533
- | 0.8267 | 107000 | 0.0004 |
534
- | 0.8305 | 107500 | 0.0004 |
535
- | 0.8344 | 108000 | 0.0004 |
536
- | 0.8382 | 108500 | 0.0004 |
537
- | 0.8421 | 109000 | 0.0004 |
538
- | 0.8460 | 109500 | 0.0004 |
539
- | 0.8498 | 110000 | 0.0004 |
540
- | 0.8537 | 110500 | 0.0004 |
541
- | 0.8576 | 111000 | 0.0004 |
542
- | 0.8614 | 111500 | 0.0004 |
543
- | 0.8653 | 112000 | 0.0004 |
544
- | 0.8691 | 112500 | 0.0004 |
545
- | 0.8730 | 113000 | 0.0004 |
546
- | 0.8769 | 113500 | 0.0004 |
547
- | 0.8807 | 114000 | 0.0004 |
548
- | 0.8846 | 114500 | 0.0004 |
549
- | 0.8885 | 115000 | 0.0004 |
550
- | 0.8923 | 115500 | 0.0004 |
551
- | 0.8962 | 116000 | 0.0004 |
552
- | 0.9001 | 116500 | 0.0004 |
553
- | 0.9039 | 117000 | 0.0004 |
554
- | 0.9078 | 117500 | 0.0004 |
555
- | 0.9116 | 118000 | 0.0004 |
556
- | 0.9155 | 118500 | 0.0004 |
557
- | 0.9194 | 119000 | 0.0004 |
558
- | 0.9232 | 119500 | 0.0004 |
559
- | 0.9271 | 120000 | 0.0004 |
560
- | 0.9310 | 120500 | 0.0004 |
561
- | 0.9348 | 121000 | 0.0004 |
562
- | 0.9387 | 121500 | 0.0004 |
563
- | 0.9425 | 122000 | 0.0004 |
564
- | 0.9464 | 122500 | 0.0004 |
565
- | 0.9503 | 123000 | 0.0004 |
566
- | 0.9541 | 123500 | 0.0004 |
567
- | 0.9580 | 124000 | 0.0004 |
568
- | 0.9619 | 124500 | 0.0004 |
569
- | 0.9657 | 125000 | 0.0004 |
570
- | 0.9696 | 125500 | 0.0004 |
571
- | 0.9734 | 126000 | 0.0004 |
572
- | 0.9773 | 126500 | 0.0004 |
573
- | 0.9812 | 127000 | 0.0004 |
574
- | 0.9850 | 127500 | 0.0004 |
575
- | 0.9889 | 128000 | 0.0004 |
576
- | 0.9928 | 128500 | 0.0004 |
577
- | 0.9966 | 129000 | 0.0004 |
578
- | 1.0005 | 129500 | 0.0004 |
579
- | 1.0043 | 130000 | 0.0004 |
580
- | 1.0082 | 130500 | 0.0004 |
581
- | 1.0121 | 131000 | 0.0004 |
582
- | 1.0159 | 131500 | 0.0004 |
583
- | 1.0198 | 132000 | 0.0004 |
584
- | 1.0237 | 132500 | 0.0004 |
585
- | 1.0275 | 133000 | 0.0004 |
586
- | 1.0314 | 133500 | 0.0004 |
587
- | 1.0353 | 134000 | 0.0004 |
588
- | 1.0391 | 134500 | 0.0004 |
589
- | 1.0430 | 135000 | 0.0004 |
590
- | 1.0468 | 135500 | 0.0004 |
591
- | 1.0507 | 136000 | 0.0004 |
592
- | 1.0546 | 136500 | 0.0004 |
593
- | 1.0584 | 137000 | 0.0004 |
594
- | 1.0623 | 137500 | 0.0004 |
595
- | 1.0662 | 138000 | 0.0004 |
596
- | 1.0700 | 138500 | 0.0004 |
597
- | 1.0739 | 139000 | 0.0004 |
598
- | 1.0777 | 139500 | 0.0004 |
599
- | 1.0816 | 140000 | 0.0004 |
600
- | 1.0855 | 140500 | 0.0004 |
601
- | 1.0893 | 141000 | 0.0004 |
602
- | 1.0932 | 141500 | 0.0004 |
603
- | 1.0971 | 142000 | 0.0004 |
604
- | 1.1009 | 142500 | 0.0004 |
605
- | 1.1048 | 143000 | 0.0004 |
606
- | 1.1086 | 143500 | 0.0004 |
607
- | 1.1125 | 144000 | 0.0004 |
608
- | 1.1164 | 144500 | 0.0004 |
609
- | 1.1202 | 145000 | 0.0004 |
610
- | 1.1241 | 145500 | 0.0004 |
611
- | 1.1280 | 146000 | 0.0004 |
612
- | 1.1318 | 146500 | 0.0004 |
613
- | 1.1357 | 147000 | 0.0004 |
614
- | 1.1396 | 147500 | 0.0004 |
615
- | 1.1434 | 148000 | 0.0004 |
616
- | 1.1473 | 148500 | 0.0004 |
617
- | 1.1511 | 149000 | 0.0004 |
618
- | 1.1550 | 149500 | 0.0004 |
619
- | 1.1589 | 150000 | 0.0004 |
620
- | 1.1627 | 150500 | 0.0004 |
621
- | 1.1666 | 151000 | 0.0004 |
622
- | 1.1705 | 151500 | 0.0004 |
623
- | 1.1743 | 152000 | 0.0004 |
624
- | 1.1782 | 152500 | 0.0004 |
625
- | 1.1820 | 153000 | 0.0004 |
626
- | 1.1859 | 153500 | 0.0004 |
627
- | 1.1898 | 154000 | 0.0004 |
628
- | 1.1936 | 154500 | 0.0004 |
629
- | 1.1975 | 155000 | 0.0004 |
630
- | 1.2014 | 155500 | 0.0003 |
631
- | 1.2052 | 156000 | 0.0003 |
632
- | 1.2091 | 156500 | 0.0004 |
633
- | 1.2129 | 157000 | 0.0003 |
634
- | 1.2168 | 157500 | 0.0004 |
635
- | 1.2207 | 158000 | 0.0003 |
636
- | 1.2245 | 158500 | 0.0003 |
637
- | 1.2284 | 159000 | 0.0003 |
638
- | 1.2323 | 159500 | 0.0003 |
639
- | 1.2361 | 160000 | 0.0003 |
640
- | 1.2400 | 160500 | 0.0003 |
641
- | 1.2438 | 161000 | 0.0003 |
642
- | 1.2477 | 161500 | 0.0003 |
643
- | 1.2516 | 162000 | 0.0003 |
644
- | 1.2554 | 162500 | 0.0003 |
645
- | 1.2593 | 163000 | 0.0003 |
646
- | 1.2632 | 163500 | 0.0003 |
647
- | 1.2670 | 164000 | 0.0003 |
648
- | 1.2709 | 164500 | 0.0003 |
649
- | 1.2748 | 165000 | 0.0003 |
650
- | 1.2786 | 165500 | 0.0003 |
651
- | 1.2825 | 166000 | 0.0003 |
652
- | 1.2863 | 166500 | 0.0003 |
653
- | 1.2902 | 167000 | 0.0003 |
654
- | 1.2941 | 167500 | 0.0003 |
655
- | 1.2979 | 168000 | 0.0003 |
656
- | 1.3018 | 168500 | 0.0003 |
657
- | 1.3057 | 169000 | 0.0003 |
658
- | 1.3095 | 169500 | 0.0003 |
659
- | 1.3134 | 170000 | 0.0003 |
660
- | 1.3172 | 170500 | 0.0003 |
661
- | 1.3211 | 171000 | 0.0003 |
662
- | 1.3250 | 171500 | 0.0003 |
663
- | 1.3288 | 172000 | 0.0003 |
664
- | 1.3327 | 172500 | 0.0003 |
665
- | 1.3366 | 173000 | 0.0003 |
666
- | 1.3404 | 173500 | 0.0003 |
667
- | 1.3443 | 174000 | 0.0003 |
668
- | 1.3481 | 174500 | 0.0003 |
669
- | 1.3520 | 175000 | 0.0003 |
670
- | 1.3559 | 175500 | 0.0003 |
671
- | 1.3597 | 176000 | 0.0003 |
672
- | 1.3636 | 176500 | 0.0003 |
673
- | 1.3675 | 177000 | 0.0003 |
674
- | 1.3713 | 177500 | 0.0003 |
675
- | 1.3752 | 178000 | 0.0003 |
676
- | 1.3790 | 178500 | 0.0003 |
677
- | 1.3829 | 179000 | 0.0003 |
678
- | 1.3868 | 179500 | 0.0003 |
679
- | 1.3906 | 180000 | 0.0003 |
680
- | 1.3945 | 180500 | 0.0003 |
681
- | 1.3984 | 181000 | 0.0003 |
682
- | 1.4022 | 181500 | 0.0003 |
683
- | 1.4061 | 182000 | 0.0003 |
684
- | 1.4100 | 182500 | 0.0003 |
685
- | 1.4138 | 183000 | 0.0003 |
686
- | 1.4177 | 183500 | 0.0003 |
687
- | 1.4215 | 184000 | 0.0003 |
688
- | 1.4254 | 184500 | 0.0003 |
689
- | 1.4293 | 185000 | 0.0003 |
690
- | 1.4331 | 185500 | 0.0003 |
691
- | 1.4370 | 186000 | 0.0003 |
692
- | 1.4409 | 186500 | 0.0003 |
693
- | 1.4447 | 187000 | 0.0003 |
694
- | 1.4486 | 187500 | 0.0003 |
695
- | 1.4524 | 188000 | 0.0003 |
696
- | 1.4563 | 188500 | 0.0003 |
697
- | 1.4602 | 189000 | 0.0003 |
698
- | 1.4640 | 189500 | 0.0003 |
699
- | 1.4679 | 190000 | 0.0003 |
700
- | 1.4718 | 190500 | 0.0003 |
701
- | 1.4756 | 191000 | 0.0003 |
702
- | 1.4795 | 191500 | 0.0003 |
703
- | 1.4833 | 192000 | 0.0003 |
704
- | 1.4872 | 192500 | 0.0003 |
705
- | 1.4911 | 193000 | 0.0003 |
706
- | 1.4949 | 193500 | 0.0003 |
707
- | 1.4988 | 194000 | 0.0003 |
708
- | 1.5027 | 194500 | 0.0003 |
709
- | 1.5065 | 195000 | 0.0003 |
710
- | 1.5104 | 195500 | 0.0003 |
711
- | 1.5143 | 196000 | 0.0003 |
712
- | 1.5181 | 196500 | 0.0003 |
713
- | 1.5220 | 197000 | 0.0003 |
714
- | 1.5258 | 197500 | 0.0003 |
715
- | 1.5297 | 198000 | 0.0003 |
716
- | 1.5336 | 198500 | 0.0003 |
717
- | 1.5374 | 199000 | 0.0003 |
718
- | 1.5413 | 199500 | 0.0003 |
719
- | 1.5452 | 200000 | 0.0003 |
720
- | 1.5490 | 200500 | 0.0003 |
721
- | 1.5529 | 201000 | 0.0003 |
722
- | 1.5567 | 201500 | 0.0003 |
723
- | 1.5606 | 202000 | 0.0003 |
724
- | 1.5645 | 202500 | 0.0003 |
725
- | 1.5683 | 203000 | 0.0003 |
726
- | 1.5722 | 203500 | 0.0003 |
727
- | 1.5761 | 204000 | 0.0003 |
728
- | 1.5799 | 204500 | 0.0003 |
729
- | 1.5838 | 205000 | 0.0003 |
730
- | 1.5876 | 205500 | 0.0003 |
731
- | 1.5915 | 206000 | 0.0003 |
732
- | 1.5954 | 206500 | 0.0003 |
733
- | 1.5992 | 207000 | 0.0003 |
734
- | 1.6031 | 207500 | 0.0003 |
735
- | 1.6070 | 208000 | 0.0003 |
736
- | 1.6108 | 208500 | 0.0003 |
737
- | 1.6147 | 209000 | 0.0003 |
738
- | 1.6185 | 209500 | 0.0003 |
739
- | 1.6224 | 210000 | 0.0003 |
740
- | 1.6263 | 210500 | 0.0003 |
741
- | 1.6301 | 211000 | 0.0003 |
742
- | 1.6340 | 211500 | 0.0003 |
743
- | 1.6379 | 212000 | 0.0003 |
744
- | 1.6417 | 212500 | 0.0003 |
745
- | 1.6456 | 213000 | 0.0003 |
746
- | 1.6495 | 213500 | 0.0003 |
747
- | 1.6533 | 214000 | 0.0003 |
748
- | 1.6572 | 214500 | 0.0003 |
749
- | 1.6610 | 215000 | 0.0003 |
750
- | 1.6649 | 215500 | 0.0003 |
751
- | 1.6688 | 216000 | 0.0003 |
752
- | 1.6726 | 216500 | 0.0003 |
753
- | 1.6765 | 217000 | 0.0003 |
754
- | 1.6804 | 217500 | 0.0003 |
755
- | 1.6842 | 218000 | 0.0003 |
756
- | 1.6881 | 218500 | 0.0003 |
757
- | 1.6919 | 219000 | 0.0003 |
758
- | 1.6958 | 219500 | 0.0003 |
759
- | 1.6997 | 220000 | 0.0003 |
760
- | 1.7035 | 220500 | 0.0003 |
761
- | 1.7074 | 221000 | 0.0003 |
762
- | 1.7113 | 221500 | 0.0003 |
763
- | 1.7151 | 222000 | 0.0003 |
764
- | 1.7190 | 222500 | 0.0003 |
765
- | 1.7228 | 223000 | 0.0003 |
766
- | 1.7267 | 223500 | 0.0003 |
767
- | 1.7306 | 224000 | 0.0003 |
768
- | 1.7344 | 224500 | 0.0003 |
769
- | 1.7383 | 225000 | 0.0003 |
770
- | 1.7422 | 225500 | 0.0003 |
771
- | 1.7460 | 226000 | 0.0003 |
772
- | 1.7499 | 226500 | 0.0003 |
773
- | 1.7537 | 227000 | 0.0003 |
774
- | 1.7576 | 227500 | 0.0003 |
775
- | 1.7615 | 228000 | 0.0003 |
776
- | 1.7653 | 228500 | 0.0003 |
777
- | 1.7692 | 229000 | 0.0003 |
778
- | 1.7731 | 229500 | 0.0003 |
779
- | 1.7769 | 230000 | 0.0003 |
780
- | 1.7808 | 230500 | 0.0003 |
781
- | 1.7847 | 231000 | 0.0003 |
782
- | 1.7885 | 231500 | 0.0003 |
783
- | 1.7924 | 232000 | 0.0003 |
784
- | 1.7962 | 232500 | 0.0003 |
785
- | 1.8001 | 233000 | 0.0003 |
786
- | 1.8040 | 233500 | 0.0003 |
787
- | 1.8078 | 234000 | 0.0003 |
788
- | 1.8117 | 234500 | 0.0003 |
789
- | 1.8156 | 235000 | 0.0003 |
790
- | 1.8194 | 235500 | 0.0003 |
791
- | 1.8233 | 236000 | 0.0003 |
792
- | 1.8271 | 236500 | 0.0003 |
793
- | 1.8310 | 237000 | 0.0003 |
794
- | 1.8349 | 237500 | 0.0003 |
795
- | 1.8387 | 238000 | 0.0003 |
796
- | 1.8426 | 238500 | 0.0003 |
797
- | 1.8465 | 239000 | 0.0003 |
798
- | 1.8503 | 239500 | 0.0003 |
799
- | 1.8542 | 240000 | 0.0003 |
800
- | 1.8580 | 240500 | 0.0003 |
801
- | 1.8619 | 241000 | 0.0003 |
802
- | 1.8658 | 241500 | 0.0003 |
803
- | 1.8696 | 242000 | 0.0003 |
804
- | 1.8735 | 242500 | 0.0003 |
805
- | 1.8774 | 243000 | 0.0003 |
806
- | 1.8812 | 243500 | 0.0003 |
807
- | 1.8851 | 244000 | 0.0003 |
808
- | 1.8889 | 244500 | 0.0003 |
809
- | 1.8928 | 245000 | 0.0003 |
810
- | 1.8967 | 245500 | 0.0003 |
811
- | 1.9005 | 246000 | 0.0003 |
812
- | 1.9044 | 246500 | 0.0003 |
813
- | 1.9083 | 247000 | 0.0003 |
814
- | 1.9121 | 247500 | 0.0003 |
815
- | 1.9160 | 248000 | 0.0003 |
816
- | 1.9199 | 248500 | 0.0003 |
817
- | 1.9237 | 249000 | 0.0003 |
818
- | 1.9276 | 249500 | 0.0003 |
819
- | 1.9314 | 250000 | 0.0003 |
820
- | 1.9353 | 250500 | 0.0003 |
821
- | 1.9392 | 251000 | 0.0003 |
822
- | 1.9430 | 251500 | 0.0003 |
823
- | 1.9469 | 252000 | 0.0003 |
824
- | 1.9508 | 252500 | 0.0003 |
825
- | 1.9546 | 253000 | 0.0003 |
826
- | 1.9585 | 253500 | 0.0003 |
827
- | 1.9623 | 254000 | 0.0003 |
828
- | 1.9662 | 254500 | 0.0003 |
829
- | 1.9701 | 255000 | 0.0003 |
830
- | 1.9739 | 255500 | 0.0003 |
831
- | 1.9778 | 256000 | 0.0003 |
832
- | 1.9817 | 256500 | 0.0003 |
833
- | 1.9855 | 257000 | 0.0003 |
834
- | 1.9894 | 257500 | 0.0003 |
835
- | 1.9932 | 258000 | 0.0003 |
836
- | 1.9971 | 258500 | 0.0003 |
837
- | 2.0010 | 259000 | 0.0003 |
838
- | 2.0048 | 259500 | 0.0003 |
839
- | 2.0087 | 260000 | 0.0003 |
840
- | 2.0126 | 260500 | 0.0003 |
841
- | 2.0164 | 261000 | 0.0003 |
842
- | 2.0203 | 261500 | 0.0003 |
843
- | 2.0242 | 262000 | 0.0003 |
844
- | 2.0280 | 262500 | 0.0003 |
845
- | 2.0319 | 263000 | 0.0003 |
846
- | 2.0357 | 263500 | 0.0003 |
847
- | 2.0396 | 264000 | 0.0003 |
848
- | 2.0435 | 264500 | 0.0003 |
849
- | 2.0473 | 265000 | 0.0003 |
850
- | 2.0512 | 265500 | 0.0003 |
851
- | 2.0551 | 266000 | 0.0003 |
852
- | 2.0589 | 266500 | 0.0003 |
853
- | 2.0628 | 267000 | 0.0003 |
854
- | 2.0666 | 267500 | 0.0003 |
855
- | 2.0705 | 268000 | 0.0003 |
856
- | 2.0744 | 268500 | 0.0003 |
857
- | 2.0782 | 269000 | 0.0003 |
858
- | 2.0821 | 269500 | 0.0003 |
859
- | 2.0860 | 270000 | 0.0003 |
860
- | 2.0898 | 270500 | 0.0003 |
861
- | 2.0937 | 271000 | 0.0003 |
862
- | 2.0975 | 271500 | 0.0003 |
863
- | 2.1014 | 272000 | 0.0003 |
864
- | 2.1053 | 272500 | 0.0003 |
865
- | 2.1091 | 273000 | 0.0003 |
866
- | 2.1130 | 273500 | 0.0003 |
867
- | 2.1169 | 274000 | 0.0003 |
868
- | 2.1207 | 274500 | 0.0003 |
869
- | 2.1246 | 275000 | 0.0003 |
870
- | 2.1284 | 275500 | 0.0003 |
871
- | 2.1323 | 276000 | 0.0003 |
872
- | 2.1362 | 276500 | 0.0003 |
873
- | 2.1400 | 277000 | 0.0003 |
874
- | 2.1439 | 277500 | 0.0003 |
875
- | 2.1478 | 278000 | 0.0003 |
876
- | 2.1516 | 278500 | 0.0003 |
877
- | 2.1555 | 279000 | 0.0003 |
878
- | 2.1594 | 279500 | 0.0003 |
879
- | 2.1632 | 280000 | 0.0003 |
880
- | 2.1671 | 280500 | 0.0003 |
881
- | 2.1709 | 281000 | 0.0003 |
882
- | 2.1748 | 281500 | 0.0003 |
883
- | 2.1787 | 282000 | 0.0003 |
884
- | 2.1825 | 282500 | 0.0003 |
885
- | 2.1864 | 283000 | 0.0003 |
886
- | 2.1903 | 283500 | 0.0003 |
887
- | 2.1941 | 284000 | 0.0003 |
888
- | 2.1980 | 284500 | 0.0003 |
889
- | 2.2018 | 285000 | 0.0003 |
890
- | 2.2057 | 285500 | 0.0003 |
891
- | 2.2096 | 286000 | 0.0003 |
892
- | 2.2134 | 286500 | 0.0003 |
893
- | 2.2173 | 287000 | 0.0003 |
894
- | 2.2212 | 287500 | 0.0003 |
895
- | 2.2250 | 288000 | 0.0003 |
896
- | 2.2289 | 288500 | 0.0003 |
897
- | 2.2327 | 289000 | 0.0003 |
898
- | 2.2366 | 289500 | 0.0003 |
899
- | 2.2405 | 290000 | 0.0003 |
900
- | 2.2443 | 290500 | 0.0003 |
901
- | 2.2482 | 291000 | 0.0003 |
902
- | 2.2521 | 291500 | 0.0003 |
903
- | 2.2559 | 292000 | 0.0003 |
904
- | 2.2598 | 292500 | 0.0003 |
905
- | 2.2636 | 293000 | 0.0003 |
906
- | 2.2675 | 293500 | 0.0003 |
907
- | 2.2714 | 294000 | 0.0003 |
908
- | 2.2752 | 294500 | 0.0003 |
909
- | 2.2791 | 295000 | 0.0003 |
910
- | 2.2830 | 295500 | 0.0003 |
911
- | 2.2868 | 296000 | 0.0003 |
912
- | 2.2907 | 296500 | 0.0003 |
913
- | 2.2946 | 297000 | 0.0003 |
914
- | 2.2984 | 297500 | 0.0003 |
915
- | 2.3023 | 298000 | 0.0003 |
916
- | 2.3061 | 298500 | 0.0003 |
917
- | 2.3100 | 299000 | 0.0003 |
918
- | 2.3139 | 299500 | 0.0003 |
919
- | 2.3177 | 300000 | 0.0003 |
920
- | 2.3216 | 300500 | 0.0003 |
921
- | 2.3255 | 301000 | 0.0003 |
922
- | 2.3293 | 301500 | 0.0003 |
923
- | 2.3332 | 302000 | 0.0003 |
924
- | 2.3370 | 302500 | 0.0003 |
925
- | 2.3409 | 303000 | 0.0003 |
926
- | 2.3448 | 303500 | 0.0003 |
927
- | 2.3486 | 304000 | 0.0003 |
928
- | 2.3525 | 304500 | 0.0003 |
929
- | 2.3564 | 305000 | 0.0003 |
930
- | 2.3602 | 305500 | 0.0003 |
931
- | 2.3641 | 306000 | 0.0003 |
932
- | 2.3679 | 306500 | 0.0003 |
933
- | 2.3718 | 307000 | 0.0003 |
934
- | 2.3757 | 307500 | 0.0003 |
935
- | 2.3795 | 308000 | 0.0003 |
936
- | 2.3834 | 308500 | 0.0003 |
937
- | 2.3873 | 309000 | 0.0003 |
938
- | 2.3911 | 309500 | 0.0003 |
939
- | 2.3950 | 310000 | 0.0003 |
940
- | 2.3989 | 310500 | 0.0003 |
941
- | 2.4027 | 311000 | 0.0003 |
942
- | 2.4066 | 311500 | 0.0003 |
943
- | 2.4104 | 312000 | 0.0003 |
944
- | 2.4143 | 312500 | 0.0003 |
945
- | 2.4182 | 313000 | 0.0003 |
946
- | 2.4220 | 313500 | 0.0003 |
947
- | 2.4259 | 314000 | 0.0003 |
948
- | 2.4298 | 314500 | 0.0003 |
949
- | 2.4336 | 315000 | 0.0003 |
950
- | 2.4375 | 315500 | 0.0003 |
951
- | 2.4413 | 316000 | 0.0003 |
952
- | 2.4452 | 316500 | 0.0003 |
953
- | 2.4491 | 317000 | 0.0003 |
954
- | 2.4529 | 317500 | 0.0003 |
955
- | 2.4568 | 318000 | 0.0003 |
956
- | 2.4607 | 318500 | 0.0003 |
957
- | 2.4645 | 319000 | 0.0003 |
958
- | 2.4684 | 319500 | 0.0003 |
959
- | 2.4722 | 320000 | 0.0003 |
960
- | 2.4761 | 320500 | 0.0003 |
961
- | 2.4800 | 321000 | 0.0003 |
962
- | 2.4838 | 321500 | 0.0003 |
963
- | 2.4877 | 322000 | 0.0003 |
964
- | 2.4916 | 322500 | 0.0003 |
965
- | 2.4954 | 323000 | 0.0003 |
966
- | 2.4993 | 323500 | 0.0003 |
967
- | 2.5031 | 324000 | 0.0003 |
968
- | 2.5070 | 324500 | 0.0003 |
969
- | 2.5109 | 325000 | 0.0003 |
970
- | 2.5147 | 325500 | 0.0003 |
971
- | 2.5186 | 326000 | 0.0003 |
972
- | 2.5225 | 326500 | 0.0003 |
973
- | 2.5263 | 327000 | 0.0003 |
974
- | 2.5302 | 327500 | 0.0003 |
975
- | 2.5341 | 328000 | 0.0003 |
976
- | 2.5379 | 328500 | 0.0003 |
977
- | 2.5418 | 329000 | 0.0003 |
978
- | 2.5456 | 329500 | 0.0003 |
979
- | 2.5495 | 330000 | 0.0003 |
980
- | 2.5534 | 330500 | 0.0003 |
981
- | 2.5572 | 331000 | 0.0003 |
982
- | 2.5611 | 331500 | 0.0003 |
983
- | 2.5650 | 332000 | 0.0003 |
984
- | 2.5688 | 332500 | 0.0003 |
985
- | 2.5727 | 333000 | 0.0003 |
986
- | 2.5765 | 333500 | 0.0003 |
987
- | 2.5804 | 334000 | 0.0003 |
988
- | 2.5843 | 334500 | 0.0003 |
989
- | 2.5881 | 335000 | 0.0003 |
990
- | 2.5920 | 335500 | 0.0003 |
991
- | 2.5959 | 336000 | 0.0003 |
992
- | 2.5997 | 336500 | 0.0003 |
993
- | 2.6036 | 337000 | 0.0003 |
994
- | 2.6074 | 337500 | 0.0003 |
995
- | 2.6113 | 338000 | 0.0003 |
996
- | 2.6152 | 338500 | 0.0003 |
997
- | 2.6190 | 339000 | 0.0003 |
998
- | 2.6229 | 339500 | 0.0003 |
999
- | 2.6268 | 340000 | 0.0003 |
1000
- | 2.6306 | 340500 | 0.0003 |
1001
- | 2.6345 | 341000 | 0.0003 |
1002
- | 2.6383 | 341500 | 0.0003 |
1003
- | 2.6422 | 342000 | 0.0003 |
1004
- | 2.6461 | 342500 | 0.0003 |
1005
- | 2.6499 | 343000 | 0.0003 |
1006
- | 2.6538 | 343500 | 0.0003 |
1007
- | 2.6577 | 344000 | 0.0003 |
1008
- | 2.6615 | 344500 | 0.0003 |
1009
- | 2.6654 | 345000 | 0.0003 |
1010
- | 2.6693 | 345500 | 0.0003 |
1011
- | 2.6731 | 346000 | 0.0003 |
1012
- | 2.6770 | 346500 | 0.0003 |
1013
- | 2.6808 | 347000 | 0.0003 |
1014
- | 2.6847 | 347500 | 0.0003 |
1015
- | 2.6886 | 348000 | 0.0003 |
1016
- | 2.6924 | 348500 | 0.0003 |
1017
- | 2.6963 | 349000 | 0.0003 |
1018
- | 2.7002 | 349500 | 0.0003 |
1019
- | 2.7040 | 350000 | 0.0003 |
1020
- | 2.7079 | 350500 | 0.0003 |
1021
- | 2.7117 | 351000 | 0.0003 |
1022
- | 2.7156 | 351500 | 0.0003 |
1023
- | 2.7195 | 352000 | 0.0003 |
1024
- | 2.7233 | 352500 | 0.0003 |
1025
- | 2.7272 | 353000 | 0.0003 |
1026
- | 2.7311 | 353500 | 0.0003 |
1027
- | 2.7349 | 354000 | 0.0003 |
1028
- | 2.7388 | 354500 | 0.0003 |
1029
- | 2.7426 | 355000 | 0.0003 |
1030
- | 2.7465 | 355500 | 0.0003 |
1031
- | 2.7504 | 356000 | 0.0003 |
1032
- | 2.7542 | 356500 | 0.0003 |
1033
- | 2.7581 | 357000 | 0.0003 |
1034
- | 2.7620 | 357500 | 0.0003 |
1035
- | 2.7658 | 358000 | 0.0003 |
1036
- | 2.7697 | 358500 | 0.0003 |
1037
- | 2.7736 | 359000 | 0.0003 |
1038
- | 2.7774 | 359500 | 0.0003 |
1039
- | 2.7813 | 360000 | 0.0003 |
1040
- | 2.7851 | 360500 | 0.0003 |
1041
- | 2.7890 | 361000 | 0.0003 |
1042
- | 2.7929 | 361500 | 0.0003 |
1043
- | 2.7967 | 362000 | 0.0003 |
1044
- | 2.8006 | 362500 | 0.0003 |
1045
- | 2.8045 | 363000 | 0.0003 |
1046
- | 2.8083 | 363500 | 0.0003 |
1047
- | 2.8122 | 364000 | 0.0003 |
1048
- | 2.8160 | 364500 | 0.0003 |
1049
- | 2.8199 | 365000 | 0.0003 |
1050
- | 2.8238 | 365500 | 0.0003 |
1051
- | 2.8276 | 366000 | 0.0003 |
1052
- | 2.8315 | 366500 | 0.0003 |
1053
- | 2.8354 | 367000 | 0.0003 |
1054
- | 2.8392 | 367500 | 0.0003 |
1055
- | 2.8431 | 368000 | 0.0003 |
1056
- | 2.8469 | 368500 | 0.0003 |
1057
- | 2.8508 | 369000 | 0.0003 |
1058
- | 2.8547 | 369500 | 0.0003 |
1059
- | 2.8585 | 370000 | 0.0003 |
1060
- | 2.8624 | 370500 | 0.0003 |
1061
- | 2.8663 | 371000 | 0.0003 |
1062
- | 2.8701 | 371500 | 0.0003 |
1063
- | 2.8740 | 372000 | 0.0003 |
1064
- | 2.8778 | 372500 | 0.0003 |
1065
- | 2.8817 | 373000 | 0.0003 |
1066
- | 2.8856 | 373500 | 0.0003 |
1067
- | 2.8894 | 374000 | 0.0003 |
1068
- | 2.8933 | 374500 | 0.0003 |
1069
- | 2.8972 | 375000 | 0.0003 |
1070
- | 2.9010 | 375500 | 0.0003 |
1071
- | 2.9049 | 376000 | 0.0003 |
1072
- | 2.9088 | 376500 | 0.0003 |
1073
- | 2.9126 | 377000 | 0.0003 |
1074
- | 2.9165 | 377500 | 0.0003 |
1075
- | 2.9203 | 378000 | 0.0003 |
1076
- | 2.9242 | 378500 | 0.0003 |
1077
- | 2.9281 | 379000 | 0.0003 |
1078
- | 2.9319 | 379500 | 0.0003 |
1079
- | 2.9358 | 380000 | 0.0003 |
1080
- | 2.9397 | 380500 | 0.0003 |
1081
- | 2.9435 | 381000 | 0.0003 |
1082
- | 2.9474 | 381500 | 0.0003 |
1083
- | 2.9512 | 382000 | 0.0003 |
1084
- | 2.9551 | 382500 | 0.0003 |
1085
- | 2.9590 | 383000 | 0.0003 |
1086
- | 2.9628 | 383500 | 0.0003 |
1087
- | 2.9667 | 384000 | 0.0003 |
1088
- | 2.9706 | 384500 | 0.0003 |
1089
- | 2.9744 | 385000 | 0.0003 |
1090
- | 2.9783 | 385500 | 0.0003 |
1091
- | 2.9821 | 386000 | 0.0003 |
1092
- | 2.9860 | 386500 | 0.0003 |
1093
- | 2.9899 | 387000 | 0.0003 |
1094
- | 2.9937 | 387500 | 0.0003 |
1095
- | 2.9976 | 388000 | 0.0003 |
1096
-
1097
- </details>
1098
-
1099
- ### Framework Versions
1100
- - Python: 3.10.13
1101
- - Sentence Transformers: 4.1.0
1102
- - Transformers: 4.52.4
1103
- - PyTorch: 2.5.1+cu121
1104
- - Accelerate: 1.7.0
1105
- - Datasets: 3.6.0
1106
- - Tokenizers: 0.21.1
1107
-
1108
- ## Citation
1109
-
1110
- ### BibTeX
1111
-
1112
- #### Sentence Transformers
1113
- ```bibtex
1114
- @inproceedings{reimers-2019-sentence-bert,
1115
- title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
1116
- author = "Reimers, Nils and Gurevych, Iryna",
1117
- booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
1118
- month = "11",
1119
- year = "2019",
1120
- publisher = "Association for Computational Linguistics",
1121
- url = "https://arxiv.org/abs/1908.10084",
1122
- }
1123
- ```
1124
-
1125
- #### MSELoss
1126
- ```bibtex
1127
- @inproceedings{reimers-2020-multilingual-sentence-bert,
1128
- title = "Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation",
1129
- author = "Reimers, Nils and Gurevych, Iryna",
1130
- booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing",
1131
- month = "11",
1132
- year = "2020",
1133
- publisher = "Association for Computational Linguistics",
1134
- url = "https://arxiv.org/abs/2004.09813",
1135
- }
1136
- ```
1137
 
1138
- <!--
1139
- ## Glossary
1140
 
1141
- *Clearly define terms in order to be accessible across audiences.*
1142
- -->
1143
 
1144
- <!--
1145
- ## Model Card Authors
1146
 
1147
- *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
1148
- -->
1149
 
1150
- <!--
1151
- ## Model Card Contact
1152
 
1153
- *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
1154
- -->
 
1
  ---
2
+ language:
3
+ - en
4
+ - az
5
+ license: cc-by-4.0
6
  tags:
7
+ - sentence-embeddings
8
  - sentence-similarity
9
+ - text-embeddings
10
+ - bilingual
11
+ - azerbaijani
12
+ - english
13
+ - all-minilm-l6-v2
14
+ - bge-small-en-v1.5
15
+ - distillation
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
  pipeline_tag: sentence-similarity
17
+ model-index:
18
+ - name: Lroc/az-en-MiniLM-L6-v2-30M
19
+ results:
20
+ - task:
21
+ type: Semantic Textual Similarity
22
+ name: Semantic Textual Similarity (Azerbaijani)
23
+ dataset:
24
+ name: Azerbaijani STS Benchmarks (Average)
25
+ type: LocalDoc/Azerbaijani-STS-Average
26
+ metrics:
27
+ - type: Pearson Correlation
28
+ value: 0.7266
29
+ name: Average Pearson
30
+ verified: false
31
  ---
32
 
33
+ # Bilingual Azerbaijani-English Sentence Embedding Model (az-en-MiniLM-L6-v2)
34
 
35
+ This is a sentence-transformer model that maps sentences & paragraphs in **Azerbaijani (az)** and **English (en)** to a 384-dimensional dense vector space.
36
+ It is designed for tasks like semantic textual similarity, semantic search, paraphrase mining, text classification, and clustering for these two languages.
37
 
38
+ The model is based on `sentence-transformers/all-MiniLM-L6-v2` and was fine-tuned using knowledge distillation from the high-performance `BAAI/bge-small-en-v1.5` English embedding model.
39
+ A custom bilingual (Azerbaijani-English) SentencePiece Unigram tokenizer with a vocabulary of ~50k was trained from scratch and is used by this model.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
40
 
 
41
 
42
+ ## Model Details
43
 
44
+ * **Base Architecture:** `sentence-transformers/all-MiniLM-L6-v2` (6 layers, 384 hidden dimension, 12 attention heads)
45
+ * **Parameters:** ~30.2 Million (after vocabulary expansion)
46
+ * **Tokenizer:** Custom bilingual (AZ-EN) SentencePiece Unigram, vocab size ~50k. Available at [LocalDoc/az-en-unigram-tokenizer-50k](https://huggingface.co/LocalDoc/az-en-unigram-tokenizer-50k).
47
+ * **Output Dimension:** 384
48
+ * **Max Sequence Length:** 512 tokens
49
+ * **Training:** Fine-tuned for 3 epochs on a parallel corpus of ~4.14 million Azerbaijani-English sentence pairs using MSELoss for knowledge distillation from `BAAI/bge-small-en-v1.5`.
50
+
51
+ ## Performance on Azerbaijani STS Benchmarks
52
+
53
+ This model demonstrates strong performance on Azerbaijani Semantic Textual Similarity (STS) tasks [LocalDoc-Azerbaijan/STS-Benchmark](https://github.com/LocalDoc-Azerbaijan/STS-Benchmark), achieving results competitive with, and in some cases surpassing, larger multilingual models.
54
+
55
+ The following results were obtained after **3 epochs** of training :
56
+
57
+ | Dataset | Pearson Correlation |
58
+ | :-------------------------------------- | :------------------: |
59
+ | LocalDoc/Azerbaijani-STSBenchmark | 0.7595 |
60
+ | LocalDoc/Azerbaijani-biosses-sts | 0.7410 |
61
+ | LocalDoc/Azerbaijani-sickr-sts | 0.7432 |
62
+ | LocalDoc/Azerbaijani-sts12-sts | 0.7644 |
63
+ | LocalDoc/Azerbaijani-sts13-sts | 0.6336 |
64
+ | LocalDoc/Azerbaijani-sts15-sts | 0.7597 |
65
+ | LocalDoc/Azerbaijani-sts16-sts | 0.6848 |
66
+ | **Average Pearson** | **0.7266** |
67
+
68
+ **Comparison with other models on (assumed) Azerbaijani STS Benchmarks (Average Pearson):**
69
+
70
+ * TEmA-small: `0.7959`
71
+ * Cohere/embed-multilingual-v3.0: `0.7823`
72
+ * BAAI/bge-m3: `0.7577`
73
+ * intfloat/multilingual-e5-large-instruct: `0.7377`
74
+ * Cohere/embed-multilingual-v2.0: `0.7318`
75
+ * intfloat/multilingual-e5-large: `0.7280`
76
+ * OpenAI/text-embedding-3-large: `0.7288`
77
+ * **LocalDoc/az-en-MiniLM-L6-v2: `0.7266`**
78
+ * sentence-transformers/LaBSE: `0.7250`
79
+ * intfloat/multilingual-e5-small: `0.7242`
80
+ * Cohere/embed-multilingual-light-v3.0: `0.7142`
81
+ * intfloat/multilingual-e5-base: `0.6960`
82
+
83
+
84
+ ## How to Use
85
+
86
+ First, install the `sentence-transformers` library:
87
  ```bash
88
  pip install -U sentence-transformers
89
  ```
90
 
 
91
  ```python
92
  from sentence_transformers import SentenceTransformer
93
 
94
+ model_id = "LocalDoc/az-en-MiniLM-L6-v2"
95
+
96
+ try:
97
+ model = SentenceTransformer(model_id)
98
+ print(f"Model {model_id} loaded successfully!")
99
+ except Exception as e:
100
+ print(f"Failed to load model. Ensure the tokenizer 'LocalDoc/az-en-unigram-tokenizer-50k' is accessible and its dependencies (protobuf, sentencepiece_model_pb2.py) are met if loading fails.")
101
+ print(f"Error: {e}")
102
+ # You might need to ensure the tokenizer can be loaded.
103
+ # If the tokenizer requires it (it shouldn't if it's correctly packaged on the Hub by your tokenizer repo):
104
+ # !pip install protobuf
105
+ # !wget -P ./az_en_tokenizer_hf/ https://raw.githubusercontent.com/google/sentencepiece/master/python/src/sentencepiece/sentencepiece_model_pb2.py
106
+ # model = SentenceTransformer(model_id)
107
+
108
+
109
+ # Example Azerbaijani sentences
110
+ sentences_az = [
111
+ "Azərbaycanın paytaxtı Bakı şəhəridir.",
112
+ "Bu gün hava çox istidir."
113
  ]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
114
 
115
+ # Example English sentences
116
+ sentences_en = [
117
+ "The capital of Azerbaijan is the city of Baku.",
118
+ "The weather is very hot today.",
119
+ "I enjoy reading books."
120
+ ]
 
 
 
 
121
 
122
+ print("\nEncoding Azerbaijani sentences...")
123
+ embeddings_az = model.encode(sentences_az)
124
+ for sent, emb in zip(sentences_az, embeddings_az):
125
+ print(f"Sentence: {sent}")
126
+ print(f"Embedding shape: {emb.shape}, first 3 dims: {emb[:3]}\n")
127
+
128
+ print("Encoding English sentences...")
129
+ embeddings_en = model.encode(sentences_en)
130
+ for sent, emb in zip(sentences_en, embeddings_en):
131
+ print(f"Sentence: {sent}")
132
+ print(f"Embedding shape: {emb.shape}, first 3 dims: {emb[:3]}\n")
133
+ ```
134
 
135
+ # Example of calculating similarity
 
136
 
137
+ ```python
138
+ from sentence_transformers.util import cos_sim
139
 
140
+ similarity_matrix = cos_sim(embeddings_az[0], embeddings_en[0])
141
+ print(f"Similarity between '{sentences_az[0]}' and '{sentences_en[0]}': {similarity_matrix.item():.4f}")
142
 
143
+ similarity_matrix_diff = cos_sim(embeddings_az[0], embeddings_en[2])
144
+ print(f"Similarity between '{sentences_az[0]}' and '{sentences_en[2]}': {similarity_matrix_diff.item():.4f}")
145
+ ```
146
 
147
+ ## Training
148
 
149
+ This model was fine-tuned from `sentence-transformers/all-MiniLM-L6-v2` using a **knowledge distillation** setup.
150
 
151
+ - **Teacher Model:** [`BAAI/bge-small-en-v1.5`](https://huggingface.co/BAAI/bge-small-en-v1.5) (used to generate target embeddings for English sentences).
152
+ - **Student Model:** Initialized from `sentence-transformers/all-MiniLM-L6-v2`.
153
+ - **Tokenizer:** A custom bilingual (Azerbaijani-English) [SentencePiece Unigram tokenizer](https://huggingface.co/LocalDoc/az-en-unigram-tokenizer-50k) (`LocalDoc/az-en-unigram-tokenizer-50k`) was used.
154
+ The student model's token embedding layer was resized to match the new vocabulary size (~50k).
155
+ - **Training Data:** A parallel corpus of approximately **4.14 million Azerbaijani-English sentence pairs**.
156
+ - **Loss Function:** `MSELoss` — the student model was trained to produce embeddings for both Azerbaijani and English sentences that are similar to the teacher model's embeddings for the corresponding **English** sentences.
 
 
 
 
 
 
 
 
157
 
158
  ### Training Hyperparameters
 
 
 
 
 
 
 
 
159
 
160
+ - **Epochs:** 3
161
+ - **Batch Size:** 64
162
+ - **Max Sequence Length:** 512
163
+ - **Learning Rate:** 3e-4
164
+ - **Warmup Ratio:** 0.15
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
165
 
 
166
 
167
+ ## CC BY 4.0 License — What It Allows
 
168
 
169
+ The **Creative Commons Attribution 4.0 International (CC BY 4.0)** license allows:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
170
 
171
+ You are free to use, modify, and distribute the model — even for commercial purposes — as long as you give proper credit to the original creator.
 
172
 
173
+ For more information, please refer to the <a target="_blank" href="https://creativecommons.org/licenses/by/4.0/deed.en">CC BY 4.0 license</a>.
 
174
 
 
 
175
 
176
+ ## Contact
 
177
 
178
+ For more information, questions, or issues, please contact LocalDoc at [v.resad.89@gmail.com].
 
179