Hugging Face's logo

Starkdoorstep12
/
entity-resolution-biencoder

Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
dense
Generated from Trainer
dataset_size:623829
loss:MultipleNegativesRankingLoss
Eval Results (legacy)
text-embeddings-inference
Model card Files
xet
 Community
Starkdoorstep12 commited on
Commit
7b0e569
·
verified ·
1 Parent(s): dac3bb8

Upload folder using huggingface_hub

Browse files
Files changed (31) hide show
  1. .gitattributes +2 -0
  2. 1_Pooling/config.json +5 -0
  3. 2_Dense/config.json +8 -0
  4. 2_Dense/model.safetensors +3 -0
  5. 3_Normalize/config.json +4 -0
  6. README.md +860 -0
  7. checkpoint-19496/1_Pooling/config.json +5 -0
  8. checkpoint-19496/2_Dense/config.json +8 -0
  9. checkpoint-19496/2_Dense/model.safetensors +3 -0
  10. checkpoint-19496/3_Normalize/config.json +4 -0
  11. checkpoint-19496/README.md +860 -0
  12. checkpoint-19496/config.json +36 -0
  13. checkpoint-19496/config_sentence_transformers.json +14 -0
  14. checkpoint-19496/model.safetensors +3 -0
  15. checkpoint-19496/modules.json +26 -0
  16. checkpoint-19496/optimizer.pt +3 -0
  17. checkpoint-19496/rng_state.pth +3 -0
  18. checkpoint-19496/scheduler.pt +3 -0
  19. checkpoint-19496/sentence_bert_config.json +10 -0
  20. checkpoint-19496/tokenizer.json +3 -0
  21. checkpoint-19496/tokenizer_config.json +18 -0
  22. checkpoint-19496/trainer_state.json +2801 -0
  23. checkpoint-19496/training_args.bin +3 -0
  24. config.json +36 -0
  25. config_sentence_transformers.json +14 -0
  26. eval/Information-Retrieval_evaluation_biencoder-val_results.csv +3 -0
  27. model.safetensors +3 -0
  28. modules.json +26 -0
  29. sentence_bert_config.json +10 -0
  30. tokenizer.json +3 -0
  31. tokenizer_config.json +18 -0
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ checkpoint-19496/tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
1_Pooling/config.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "embedding_dimension": 768,
3
+ "pooling_mode": "cls",
4
+ "include_prompt": true
5
+ }
2_Dense/config.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "in_features": 768,
3
+ "out_features": 768,
4
+ "bias": true,
5
+ "activation_function": "torch.nn.modules.activation.Tanh",
6
+ "module_input_name": "sentence_embedding",
7
+ "module_output_name": "sentence_embedding"
8
+ }
2_Dense/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ffb7f44524dc870beaae5f90090cef75366ae2387053696c402b575cc624d41b
3
+ size 2362528
3_Normalize/config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "module_input_name": "sentence_embedding",
3
+ "module_output_name": "sentence_embedding"
4
+ }
README.md ADDED
@@ -0,0 +1,860 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - sentence-transformers
4
+ - sentence-similarity
5
+ - feature-extraction
6
+ - dense
7
+ - generated_from_trainer
8
+ - dataset_size:623829
9
+ - loss:MultipleNegativesRankingLoss
10
+ base_model: sentence-transformers/LaBSE
11
+ widget:
12
+ - source_sentence: Uptown Pub! | 1589 Clover Lane, Wichita Falls, TX
13
+ sentences:
14
+ - Moa LLC Services | 1964 341, VEEDERSBURG, IN
15
+ - Wong Peak-(One) | 2658 61, Vincennes, Indiana
16
+ - 'Uptown Pub! Trading | '
17
+ - source_sentence: Village Of | 375 Broadway, Unit 2-G08, Lynnfield, MA
18
+ sentences:
19
+ - ROCKY GUILD | 260 2ND AVENUE, FORSYTH, MT
20
+ - 'Village Of Center | '
21
+ - Vio L.L.C. | 415 2TH AVENUE, WASHINGTON, IA
22
+ - source_sentence: St. Fellowship | 1822 Mayfair Drive, City Of Appleton, WI
23
+ sentences:
24
+ - Sanders Adgno | 3606 LORNE DR, ENDWELL, NY
25
+ - 'Quality Opendoor | '
26
+ - 'St. Fellowship | '
27
+ - source_sentence: Ross and Trinidad Inc | AR, Bryant, 6328 Lena Drive
28
+ sentences:
29
+ - 'STEINMEYER,-WINI, SERVICES | '
30
+ - 'Ross and Trinidad Inc. | '
31
+ - Zium [Ltd.] | 3215- CROSSFELL RD, SRING, TX
32
+ - source_sentence: Stag | 122 Bruce Street, Scotia, NY
33
+ sentences:
34
+ - Stag Corporation | BRUCE STREET, SCOTIA, NY
35
+ - ORTHOPEDIC PHYSICIANS | 75, ALTOONA, AL
36
+ - '#2 HUB CO | 142 COTTAGE STREET, LEWISTON, ME'
37
+ pipeline_tag: sentence-similarity
38
+ library_name: sentence-transformers
39
+ metrics:
40
+ - cosine_accuracy@1
41
+ - cosine_accuracy@5
42
+ - cosine_accuracy@10
43
+ - cosine_accuracy@30
44
+ - cosine_precision@1
45
+ - cosine_precision@3
46
+ - cosine_precision@5
47
+ - cosine_precision@10
48
+ - cosine_recall@1
49
+ - cosine_recall@3
50
+ - cosine_recall@5
51
+ - cosine_recall@10
52
+ - cosine_ndcg@10
53
+ - cosine_mrr@10
54
+ - cosine_map@100
55
+ model-index:
56
+ - name: SentenceTransformer based on sentence-transformers/LaBSE
57
+ results:
58
+ - task:
59
+ type: information-retrieval
60
+ name: Information Retrieval
61
+ dataset:
62
+ name: biencoder val
63
+ type: biencoder-val
64
+ metrics:
65
+ - type: cosine_accuracy@1
66
+ value: 0.929
67
+ name: Cosine Accuracy@1
68
+ - type: cosine_accuracy@5
69
+ value: 1.0
70
+ name: Cosine Accuracy@5
71
+ - type: cosine_accuracy@10
72
+ value: 1.0
73
+ name: Cosine Accuracy@10
74
+ - type: cosine_accuracy@30
75
+ value: 1.0
76
+ name: Cosine Accuracy@30
77
+ - type: cosine_precision@1
78
+ value: 0.929
79
+ name: Cosine Precision@1
80
+ - type: cosine_precision@3
81
+ value: 0.33333333333333326
82
+ name: Cosine Precision@3
83
+ - type: cosine_precision@5
84
+ value: 0.20000000000000004
85
+ name: Cosine Precision@5
86
+ - type: cosine_precision@10
87
+ value: 0.10000000000000002
88
+ name: Cosine Precision@10
89
+ - type: cosine_recall@1
90
+ value: 0.929
91
+ name: Cosine Recall@1
92
+ - type: cosine_recall@3
93
+ value: 1.0
94
+ name: Cosine Recall@3
95
+ - type: cosine_recall@5
96
+ value: 1.0
97
+ name: Cosine Recall@5
98
+ - type: cosine_recall@10
99
+ value: 1.0
100
+ name: Cosine Recall@10
101
+ - type: cosine_ndcg@10
102
+ value: 0.9732722934892877
103
+ name: Cosine Ndcg@10
104
+ - type: cosine_mrr@10
105
+ value: 0.9638333333333332
106
+ name: Cosine Mrr@10
107
+ - type: cosine_map@100
108
+ value: 0.9638333333333332
109
+ name: Cosine Map@100
110
+ ---
111
+
112
+ # SentenceTransformer based on sentence-transformers/LaBSE
113
+
114
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/LaBSE](https://huggingface.co/sentence-transformers/LaBSE). It maps inputs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.
115
+
116
+ ## Model Details
117
+
118
+ ### Model Description
119
+ - **Model Type:** Sentence Transformer
120
+ - **Base model:** [sentence-transformers/LaBSE](https://huggingface.co/sentence-transformers/LaBSE) <!-- at revision 836121a0533e5664b21c7aacc5d22951f2b8b25b -->
121
+ - **Maximum Sequence Length:** 256 tokens
122
+ - **Output Dimensionality:** 768 dimensions
123
+ - **Similarity Function:** Cosine Similarity
124
+ - **Supported Modality:** Text
125
+ <!-- - **Training Dataset:** Unknown -->
126
+ <!-- - **Language:** Unknown -->
127
+ <!-- - **License:** Unknown -->
128
+
129
+ ### Model Sources
130
+
131
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
132
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
133
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
134
+
135
+ ### Full Model Architecture
136
+
137
+ ```
138
+ SentenceTransformer(
139
+ (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
140
+ (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True})
141
+ (2): Dense({'in_features': 768, 'out_features': 768, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh', 'module_input_name': 'sentence_embedding', 'module_output_name': 'sentence_embedding'})
142
+ (3): Normalize({'module_input_name': 'sentence_embedding', 'module_output_name': 'sentence_embedding'})
143
+ )
144
+ ```
145
+
146
+ ## Usage
147
+
148
+ ### Direct Usage (Sentence Transformers)
149
+
150
+ First install the Sentence Transformers library:
151
+
152
+ ```bash
153
+ pip install -U sentence-transformers
154
+ ```
155
+ Then you can load this model and run inference.
156
+ ```python
157
+ from sentence_transformers import SentenceTransformer
158
+
159
+ # Download from the 🤗 Hub
160
+ model = SentenceTransformer("sentence_transformers_model_id")
161
+ # Run inference
162
+ queries = [
163
+ 'Stag | 122 Bruce Street, Scotia, NY',
164
+ ]
165
+ documents = [
166
+ 'Stag Corporation | BRUCE STREET, SCOTIA, NY',
167
+ '#2 HUB CO | 142 COTTAGE STREET, LEWISTON, ME',
168
+ 'ORTHOPEDIC PHYSICIANS | 75, ALTOONA, AL',
169
+ ]
170
+ query_embeddings = model.encode_query(queries)
171
+ document_embeddings = model.encode_document(documents)
172
+ print(query_embeddings.shape, document_embeddings.shape)
173
+ # [1, 768] [3, 768]
174
+
175
+ # Get the similarity scores for the embeddings
176
+ similarities = model.similarity(query_embeddings, document_embeddings)
177
+ print(similarities)
178
+ # tensor([[ 0.7677, 0.1634, -0.0688]])
179
+ ```
180
+ <!--
181
+ ### Direct Usage (Transformers)
182
+
183
+ <details><summary>Click to see the direct usage in Transformers</summary>
184
+
185
+ </details>
186
+ -->
187
+
188
+ <!--
189
+ ### Downstream Usage (Sentence Transformers)
190
+
191
+ You can finetune this model on your own dataset.
192
+
193
+ <details><summary>Click to expand</summary>
194
+
195
+ </details>
196
+ -->
197
+
198
+ <!--
199
+ ### Out-of-Scope Use
200
+
201
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
202
+ -->
203
+
204
+ ## Evaluation
205
+
206
+ ### Metrics
207
+
208
+ #### Information Retrieval
209
+
210
+ * Dataset: `biencoder-val`
211
+ * Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.sentence_transformer.evaluation.InformationRetrievalEvaluator)
212
+
213
+ | Metric | Value |
214
+ |:--------------------|:-----------|
215
+ | cosine_accuracy@1 | 0.929 |
216
+ | cosine_accuracy@5 | 1.0 |
217
+ | cosine_accuracy@10 | 1.0 |
218
+ | cosine_accuracy@30 | 1.0 |
219
+ | cosine_precision@1 | 0.929 |
220
+ | cosine_precision@3 | 0.3333 |
221
+ | cosine_precision@5 | 0.2 |
222
+ | cosine_precision@10 | 0.1 |
223
+ | cosine_recall@1 | 0.929 |
224
+ | cosine_recall@3 | 1.0 |
225
+ | cosine_recall@5 | 1.0 |
226
+ | cosine_recall@10 | 1.0 |
227
+ | **cosine_ndcg@10** | **0.9733** |
228
+ | cosine_mrr@10 | 0.9638 |
229
+ | cosine_map@100 | 0.9638 |
230
+
231
+ <!--
232
+ ## Bias, Risks and Limitations
233
+
234
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
235
+ -->
236
+
237
+ <!--
238
+ ### Recommendations
239
+
240
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
241
+ -->
242
+
243
+ ## Training Details
244
+
245
+ ### Training Dataset
246
+
247
+ #### Unnamed Dataset
248
+
249
+ * Size: 623,829 training samples
250
+ * Columns: <code>anchor</code> and <code>positive</code>
251
+ * Approximate statistics based on the first 100 samples:
252
+ | | anchor | positive |
253
+ |:---------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
254
+ | type | string | string |
255
+ | modality | text | text |
256
+ | details | <ul><li>min: 18 tokens</li><li>mean: 31.59 tokens</li><li>max: 55 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 33.45 tokens</li><li>max: 64 tokens</li></ul> |
257
+ * Samples:
258
+ | anchor | positive |
259
+ |:----------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------|
260
+ | <code>Bangalore South Automotive Private Limited \| C-901, Goyalco Orchid, Lakeview, Bangalore South, Bangalore, Karnataka</code> | <code>Bangalore South Automotive Limited Private \| C-901., GOYALCO ORCHID, LAKEVIEW, BANGALORE SOUTH, ಕರ್ನಾಟಕ</code> |
261
+ | <code>Bangalore South Automotive Private Limited \| C-901, Goyalco Orchid, Lakeview, Bangalore South, Bangalore, Karnataka</code> | <code>Bangalore Sóuth Automotive Private Limited \| </code> |
262
+ | <code>Software Pinnacle Tourism \| 3016/5, 1St Floor, Ranjit Nagar, New Delhi, Central Delhi, Delhi</code> | <code>Software Pinnacle Tóurism \| A-3016/5, 1ST FLOOR, RANJIT NAGAR, NEW DELHI, दिल्ली</code> |
263
+ * Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
264
+ ```json
265
+ {
266
+ "scale": 20.0,
267
+ "similarity_fct": "cos_sim",
268
+ "gather_across_devices": false,
269
+ "directions": [
270
+ "query_to_doc"
271
+ ],
272
+ "partition_mode": "joint",
273
+ "hardness_mode": null,
274
+ "hardness_strength": 0.0
275
+ }
276
+ ```
277
+
278
+ ### Training Hyperparameters
279
+ #### Non-Default Hyperparameters
280
+
281
+ - `per_device_train_batch_size`: 64
282
+ - `num_train_epochs`: 2
283
+ - `learning_rate`: 2e-05
284
+ - `warmup_steps`: 0.1
285
+
286
+ #### All Hyperparameters
287
+ <details><summary>Click to expand</summary>
288
+
289
+ - `per_device_train_batch_size`: 64
290
+ - `num_train_epochs`: 2
291
+ - `max_steps`: -1
292
+ - `learning_rate`: 2e-05
293
+ - `lr_scheduler_type`: linear
294
+ - `lr_scheduler_kwargs`: None
295
+ - `warmup_steps`: 0.1
296
+ - `optim`: adamw_torch_fused
297
+ - `optim_args`: None
298
+ - `weight_decay`: 0.0
299
+ - `adam_beta1`: 0.9
300
+ - `adam_beta2`: 0.999
301
+ - `adam_epsilon`: 1e-08
302
+ - `optim_target_modules`: None
303
+ - `gradient_accumulation_steps`: 1
304
+ - `average_tokens_across_devices`: True
305
+ - `max_grad_norm`: 1.0
306
+ - `label_smoothing_factor`: 0.0
307
+ - `bf16`: False
308
+ - `fp16`: False
309
+ - `bf16_full_eval`: False
310
+ - `fp16_full_eval`: False
311
+ - `tf32`: None
312
+ - `gradient_checkpointing`: False
313
+ - `gradient_checkpointing_kwargs`: None
314
+ - `torch_compile`: False
315
+ - `torch_compile_backend`: None
316
+ - `torch_compile_mode`: None
317
+ - `use_liger_kernel`: False
318
+ - `liger_kernel_config`: None
319
+ - `use_cache`: False
320
+ - `neftune_noise_alpha`: None
321
+ - `torch_empty_cache_steps`: None
322
+ - `auto_find_batch_size`: False
323
+ - `log_on_each_node`: True
324
+ - `logging_nan_inf_filter`: True
325
+ - `include_num_input_tokens_seen`: no
326
+ - `log_level`: passive
327
+ - `log_level_replica`: warning
328
+ - `disable_tqdm`: False
329
+ - `project`: huggingface
330
+ - `trackio_space_id`: None
331
+ - `trackio_bucket_id`: None
332
+ - `trackio_static_space_id`: None
333
+ - `per_device_eval_batch_size`: 8
334
+ - `prediction_loss_only`: True
335
+ - `eval_on_start`: False
336
+ - `eval_do_concat_batches`: True
337
+ - `eval_use_gather_object`: False
338
+ - `eval_accumulation_steps`: None
339
+ - `include_for_metrics`: []
340
+ - `batch_eval_metrics`: False
341
+ - `save_only_model`: False
342
+ - `save_on_each_node`: False
343
+ - `enable_jit_checkpoint`: False
344
+ - `push_to_hub`: False
345
+ - `hub_private_repo`: None
346
+ - `hub_model_id`: None
347
+ - `hub_strategy`: every_save
348
+ - `hub_always_push`: False
349
+ - `hub_revision`: None
350
+ - `load_best_model_at_end`: False
351
+ - `ignore_data_skip`: False
352
+ - `restore_callback_states_from_checkpoint`: False
353
+ - `full_determinism`: False
354
+ - `seed`: 42
355
+ - `data_seed`: None
356
+ - `use_cpu`: False
357
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
358
+ - `parallelism_config`: None
359
+ - `dataloader_drop_last`: False
360
+ - `dataloader_num_workers`: 0
361
+ - `dataloader_pin_memory`: True
362
+ - `dataloader_persistent_workers`: False
363
+ - `dataloader_prefetch_factor`: None
364
+ - `dataloader_multiprocessing_context`: None
365
+ - `dataloader_in_order`: True
366
+ - `remove_unused_columns`: True
367
+ - `label_names`: None
368
+ - `train_sampling_strategy`: random
369
+ - `length_column_name`: length
370
+ - `ddp_find_unused_parameters`: None
371
+ - `ddp_bucket_cap_mb`: None
372
+ - `ddp_broadcast_buffers`: False
373
+ - `ddp_static_graph`: None
374
+ - `ddp_backend`: None
375
+ - `ddp_timeout`: 1800
376
+ - `fsdp`: None
377
+ - `fsdp_config`: None
378
+ - `deepspeed`: None
379
+ - `debug`: []
380
+ - `skip_memory_metrics`: True
381
+ - `do_predict`: False
382
+ - `resume_from_checkpoint`: None
383
+ - `local_rank`: -1
384
+ - `prompts`: None
385
+ - `batch_sampler`: batch_sampler
386
+ - `multi_dataset_batch_sampler`: proportional
387
+ - `router_mapping`: {}
388
+ - `learning_rate_mapping`: {}
389
+ - `warmup_ratio`: None
390
+
391
+ </details>
392
+
393
+ ### Training Logs
394
+ <details><summary>Click to expand</summary>
395
+
396
+ | Epoch | Step | Training Loss | biencoder-val_cosine_ndcg@10 |
397
+ |:------:|:-----:|:-------------:|:----------------------------:|
398
+ | 0.0051 | 50 | 0.2099 | - |
399
+ | 0.0103 | 100 | 0.0801 | - |
400
+ | 0.0154 | 150 | 0.0250 | - |
401
+ | 0.0205 | 200 | 0.0196 | - |
402
+ | 0.0256 | 250 | 0.0155 | - |
403
+ | 0.0308 | 300 | 0.0151 | - |
404
+ | 0.0359 | 350 | 0.0150 | - |
405
+ | 0.0410 | 400 | 0.0117 | - |
406
+ | 0.0462 | 450 | 0.0124 | - |
407
+ | 0.0513 | 500 | 0.0111 | - |
408
+ | 0.0564 | 550 | 0.0070 | - |
409
+ | 0.0616 | 600 | 0.0107 | - |
410
+ | 0.0667 | 650 | 0.0091 | - |
411
+ | 0.0718 | 700 | 0.0060 | - |
412
+ | 0.0769 | 750 | 0.0094 | - |
413
+ | 0.0821 | 800 | 0.0067 | - |
414
+ | 0.0872 | 850 | 0.0061 | - |
415
+ | 0.0923 | 900 | 0.0076 | - |
416
+ | 0.0975 | 950 | 0.0060 | - |
417
+ | 0.1026 | 1000 | 0.0047 | - |
418
+ | 0.1077 | 1050 | 0.0071 | - |
419
+ | 0.1128 | 1100 | 0.0063 | - |
420
+ | 0.1180 | 1150 | 0.0046 | - |
421
+ | 0.1231 | 1200 | 0.0039 | - |
422
+ | 0.1282 | 1250 | 0.0049 | - |
423
+ | 0.1334 | 1300 | 0.0055 | - |
424
+ | 0.1385 | 1350 | 0.0038 | - |
425
+ | 0.1436 | 1400 | 0.0041 | - |
426
+ | 0.1487 | 1450 | 0.0060 | - |
427
+ | 0.1539 | 1500 | 0.0055 | - |
428
+ | 0.1590 | 1550 | 0.0053 | - |
429
+ | 0.1641 | 1600 | 0.0047 | - |
430
+ | 0.1693 | 1650 | 0.0044 | - |
431
+ | 0.1744 | 1700 | 0.0071 | - |
432
+ | 0.1795 | 1750 | 0.0044 | - |
433
+ | 0.1847 | 1800 | 0.0025 | - |
434
+ | 0.1898 | 1850 | 0.0055 | - |
435
+ | 0.1949 | 1900 | 0.0053 | - |
436
+ | 0.2000 | 1950 | 0.0045 | - |
437
+ | 0.2052 | 2000 | 0.0047 | - |
438
+ | 0.2103 | 2050 | 0.0048 | - |
439
+ | 0.2154 | 2100 | 0.0041 | - |
440
+ | 0.2206 | 2150 | 0.0030 | - |
441
+ | 0.2257 | 2200 | 0.0037 | - |
442
+ | 0.2308 | 2250 | 0.0047 | - |
443
+ | 0.2359 | 2300 | 0.0042 | - |
444
+ | 0.2411 | 2350 | 0.0027 | - |
445
+ | 0.2462 | 2400 | 0.0047 | - |
446
+ | 0.2513 | 2450 | 0.0028 | - |
447
+ | 0.2565 | 2500 | 0.0043 | - |
448
+ | 0.2616 | 2550 | 0.0063 | - |
449
+ | 0.2667 | 2600 | 0.0027 | - |
450
+ | 0.2719 | 2650 | 0.0021 | - |
451
+ | 0.2770 | 2700 | 0.0028 | - |
452
+ | 0.2821 | 2750 | 0.0024 | - |
453
+ | 0.2872 | 2800 | 0.0045 | - |
454
+ | 0.2924 | 2850 | 0.0033 | - |
455
+ | 0.2975 | 2900 | 0.0048 | - |
456
+ | 0.3026 | 2950 | 0.0021 | - |
457
+ | 0.3078 | 3000 | 0.0031 | - |
458
+ | 0.3129 | 3050 | 0.0027 | - |
459
+ | 0.3180 | 3100 | 0.0033 | - |
460
+ | 0.3231 | 3150 | 0.0040 | - |
461
+ | 0.3283 | 3200 | 0.0012 | - |
462
+ | 0.3334 | 3250 | 0.0031 | - |
463
+ | 0.3385 | 3300 | 0.0033 | - |
464
+ | 0.3437 | 3350 | 0.0020 | - |
465
+ | 0.3488 | 3400 | 0.0028 | - |
466
+ | 0.3539 | 3450 | 0.0022 | - |
467
+ | 0.3590 | 3500 | 0.0025 | - |
468
+ | 0.3642 | 3550 | 0.0026 | - |
469
+ | 0.3693 | 3600 | 0.0028 | - |
470
+ | 0.3744 | 3650 | 0.0035 | - |
471
+ | 0.3796 | 3700 | 0.0023 | - |
472
+ | 0.3847 | 3750 | 0.0029 | - |
473
+ | 0.3898 | 3800 | 0.0031 | - |
474
+ | 0.3950 | 3850 | 0.0042 | - |
475
+ | 0.4001 | 3900 | 0.0027 | - |
476
+ | 0.4052 | 3950 | 0.0035 | - |
477
+ | 0.4103 | 4000 | 0.0026 | - |
478
+ | 0.4155 | 4050 | 0.0017 | - |
479
+ | 0.4206 | 4100 | 0.0022 | - |
480
+ | 0.4257 | 4150 | 0.0020 | - |
481
+ | 0.4309 | 4200 | 0.0025 | - |
482
+ | 0.4360 | 4250 | 0.0024 | - |
483
+ | 0.4411 | 4300 | 0.0025 | - |
484
+ | 0.4462 | 4350 | 0.0025 | - |
485
+ | 0.4514 | 4400 | 0.0021 | - |
486
+ | 0.4565 | 4450 | 0.0011 | - |
487
+ | 0.4616 | 4500 | 0.0018 | - |
488
+ | 0.4668 | 4550 | 0.0039 | - |
489
+ | 0.4719 | 4600 | 0.0016 | - |
490
+ | 0.4770 | 4650 | 0.0023 | - |
491
+ | 0.4822 | 4700 | 0.0012 | - |
492
+ | 0.4873 | 4750 | 0.0027 | - |
493
+ | 0.4924 | 4800 | 0.0020 | - |
494
+ | 0.4975 | 4850 | 0.0022 | - |
495
+ | 0.5027 | 4900 | 0.0022 | - |
496
+ | 0.5078 | 4950 | 0.0016 | - |
497
+ | 0.5129 | 5000 | 0.0027 | - |
498
+ | 0.5181 | 5050 | 0.0017 | - |
499
+ | 0.5232 | 5100 | 0.0020 | - |
500
+ | 0.5283 | 5150 | 0.0024 | - |
501
+ | 0.5334 | 5200 | 0.0024 | - |
502
+ | 0.5386 | 5250 | 0.0024 | - |
503
+ | 0.5437 | 5300 | 0.0022 | - |
504
+ | 0.5488 | 5350 | 0.0029 | - |
505
+ | 0.5540 | 5400 | 0.0016 | - |
506
+ | 0.5591 | 5450 | 0.0010 | - |
507
+ | 0.5642 | 5500 | 0.0017 | - |
508
+ | 0.5693 | 5550 | 0.0029 | - |
509
+ | 0.5745 | 5600 | 0.0017 | - |
510
+ | 0.5796 | 5650 | 0.0021 | - |
511
+ | 0.5847 | 5700 | 0.0019 | - |
512
+ | 0.5899 | 5750 | 0.0021 | - |
513
+ | 0.5950 | 5800 | 0.0030 | - |
514
+ | 0.6001 | 5850 | 0.0015 | - |
515
+ | 0.6053 | 5900 | 0.0028 | - |
516
+ | 0.6104 | 5950 | 0.0022 | - |
517
+ | 0.6155 | 6000 | 0.0020 | - |
518
+ | 0.6206 | 6050 | 0.0021 | - |
519
+ | 0.6258 | 6100 | 0.0017 | - |
520
+ | 0.6309 | 6150 | 0.0021 | - |
521
+ | 0.6360 | 6200 | 0.0015 | - |
522
+ | 0.6412 | 6250 | 0.0010 | - |
523
+ | 0.6463 | 6300 | 0.0018 | - |
524
+ | 0.6514 | 6350 | 0.0011 | - |
525
+ | 0.6565 | 6400 | 0.0035 | - |
526
+ | 0.6617 | 6450 | 0.0015 | - |
527
+ | 0.6668 | 6500 | 0.0022 | - |
528
+ | 0.6719 | 6550 | 0.0019 | - |
529
+ | 0.6771 | 6600 | 0.0017 | - |
530
+ | 0.6822 | 6650 | 0.0011 | - |
531
+ | 0.6873 | 6700 | 0.0021 | - |
532
+ | 0.6924 | 6750 | 0.0017 | - |
533
+ | 0.6976 | 6800 | 0.0014 | - |
534
+ | 0.7027 | 6850 | 0.0024 | - |
535
+ | 0.7078 | 6900 | 0.0012 | - |
536
+ | 0.7130 | 6950 | 0.0032 | - |
537
+ | 0.7181 | 7000 | 0.0020 | - |
538
+ | 0.7232 | 7050 | 0.0013 | - |
539
+ | 0.7284 | 7100 | 0.0038 | - |
540
+ | 0.7335 | 7150 | 0.0018 | - |
541
+ | 0.7386 | 7200 | 0.0019 | - |
542
+ | 0.7437 | 7250 | 0.0038 | - |
543
+ | 0.7489 | 7300 | 0.0013 | - |
544
+ | 0.7540 | 7350 | 0.0025 | - |
545
+ | 0.7591 | 7400 | 0.0022 | - |
546
+ | 0.7643 | 7450 | 0.0014 | - |
547
+ | 0.7694 | 7500 | 0.0015 | - |
548
+ | 0.7745 | 7550 | 0.0030 | - |
549
+ | 0.7796 | 7600 | 0.0016 | - |
550
+ | 0.7848 | 7650 | 0.0017 | - |
551
+ | 0.7899 | 7700 | 0.0013 | - |
552
+ | 0.7950 | 7750 | 0.0036 | - |
553
+ | 0.8002 | 7800 | 0.0022 | - |
554
+ | 0.8053 | 7850 | 0.0015 | - |
555
+ | 0.8104 | 7900 | 0.0010 | - |
556
+ | 0.8156 | 7950 | 0.0021 | - |
557
+ | 0.8207 | 8000 | 0.0019 | - |
558
+ | 0.8258 | 8050 | 0.0011 | - |
559
+ | 0.8309 | 8100 | 0.0012 | - |
560
+ | 0.8361 | 8150 | 0.0026 | - |
561
+ | 0.8412 | 8200 | 0.0011 | - |
562
+ | 0.8463 | 8250 | 0.0019 | - |
563
+ | 0.8515 | 8300 | 0.0017 | - |
564
+ | 0.8566 | 8350 | 0.0011 | - |
565
+ | 0.8617 | 8400 | 0.0012 | - |
566
+ | 0.8668 | 8450 | 0.0018 | - |
567
+ | 0.8720 | 8500 | 0.0008 | - |
568
+ | 0.8771 | 8550 | 0.0017 | - |
569
+ | 0.8822 | 8600 | 0.0023 | - |
570
+ | 0.8874 | 8650 | 0.0010 | - |
571
+ | 0.8925 | 8700 | 0.0011 | - |
572
+ | 0.8976 | 8750 | 0.0022 | - |
573
+ | 0.9027 | 8800 | 0.0023 | - |
574
+ | 0.9079 | 8850 | 0.0013 | - |
575
+ | 0.9130 | 8900 | 0.0015 | - |
576
+ | 0.9181 | 8950 | 0.0034 | - |
577
+ | 0.9233 | 9000 | 0.0011 | - |
578
+ | 0.9284 | 9050 | 0.0018 | - |
579
+ | 0.9335 | 9100 | 0.0013 | - |
580
+ | 0.9387 | 9150 | 0.0009 | - |
581
+ | 0.9438 | 9200 | 0.0008 | - |
582
+ | 0.9489 | 9250 | 0.0013 | - |
583
+ | 0.9540 | 9300 | 0.0012 | - |
584
+ | 0.9592 | 9350 | 0.0012 | - |
585
+ | 0.9643 | 9400 | 0.0008 | - |
586
+ | 0.9694 | 9450 | 0.0011 | - |
587
+ | 0.9746 | 9500 | 0.0008 | - |
588
+ | 0.9797 | 9550 | 0.0015 | - |
589
+ | 0.9848 | 9600 | 0.0014 | - |
590
+ | 0.9899 | 9650 | 0.0013 | - |
591
+ | 0.9951 | 9700 | 0.0006 | - |
592
+ | 1.0 | 9748 | - | 0.9733 |
593
+ | 1.0002 | 9750 | 0.0014 | - |
594
+ | 1.0053 | 9800 | 0.0006 | - |
595
+ | 1.0105 | 9850 | 0.0005 | - |
596
+ | 1.0156 | 9900 | 0.0011 | - |
597
+ | 1.0207 | 9950 | 0.0011 | - |
598
+ | 1.0259 | 10000 | 0.0009 | - |
599
+ | 1.0310 | 10050 | 0.0013 | - |
600
+ | 1.0361 | 10100 | 0.0008 | - |
601
+ | 1.0412 | 10150 | 0.0008 | - |
602
+ | 1.0464 | 10200 | 0.0006 | - |
603
+ | 1.0515 | 10250 | 0.0006 | - |
604
+ | 1.0566 | 10300 | 0.0010 | - |
605
+ | 1.0618 | 10350 | 0.0026 | - |
606
+ | 1.0669 | 10400 | 0.0007 | - |
607
+ | 1.0720 | 10450 | 0.0007 | - |
608
+ | 1.0771 | 10500 | 0.0010 | - |
609
+ | 1.0823 | 10550 | 0.0016 | - |
610
+ | 1.0874 | 10600 | 0.0014 | - |
611
+ | 1.0925 | 10650 | 0.0007 | - |
612
+ | 1.0977 | 10700 | 0.0006 | - |
613
+ | 1.1028 | 10750 | 0.0021 | - |
614
+ | 1.1079 | 10800 | 0.0027 | - |
615
+ | 1.1130 | 10850 | 0.0006 | - |
616
+ | 1.1182 | 10900 | 0.0006 | - |
617
+ | 1.1233 | 10950 | 0.0012 | - |
618
+ | 1.1284 | 11000 | 0.0016 | - |
619
+ | 1.1336 | 11050 | 0.0015 | - |
620
+ | 1.1387 | 11100 | 0.0026 | - |
621
+ | 1.1438 | 11150 | 0.0008 | - |
622
+ | 1.1490 | 11200 | 0.0007 | - |
623
+ | 1.1541 | 11250 | 0.0007 | - |
624
+ | 1.1592 | 11300 | 0.0012 | - |
625
+ | 1.1643 | 11350 | 0.0010 | - |
626
+ | 1.1695 | 11400 | 0.0005 | - |
627
+ | 1.1746 | 11450 | 0.0008 | - |
628
+ | 1.1797 | 11500 | 0.0006 | - |
629
+ | 1.1849 | 11550 | 0.0005 | - |
630
+ | 1.1900 | 11600 | 0.0016 | - |
631
+ | 1.1951 | 11650 | 0.0009 | - |
632
+ | 1.2002 | 11700 | 0.0011 | - |
633
+ | 1.2054 | 11750 | 0.0006 | - |
634
+ | 1.2105 | 11800 | 0.0014 | - |
635
+ | 1.2156 | 11850 | 0.0006 | - |
636
+ | 1.2208 | 11900 | 0.0006 | - |
637
+ | 1.2259 | 11950 | 0.0015 | - |
638
+ | 1.2310 | 12000 | 0.0005 | - |
639
+ | 1.2362 | 12050 | 0.0037 | - |
640
+ | 1.2413 | 12100 | 0.0006 | - |
641
+ | 1.2464 | 12150 | 0.0005 | - |
642
+ | 1.2515 | 12200 | 0.0007 | - |
643
+ | 1.2567 | 12250 | 0.0016 | - |
644
+ | 1.2618 | 12300 | 0.0006 | - |
645
+ | 1.2669 | 12350 | 0.0015 | - |
646
+ | 1.2721 | 12400 | 0.0005 | - |
647
+ | 1.2772 | 12450 | 0.0007 | - |
648
+ | 1.2823 | 12500 | 0.0006 | - |
649
+ | 1.2874 | 12550 | 0.0009 | - |
650
+ | 1.2926 | 12600 | 0.0005 | - |
651
+ | 1.2977 | 12650 | 0.0004 | - |
652
+ | 1.3028 | 12700 | 0.0011 | - |
653
+ | 1.3080 | 12750 | 0.0006 | - |
654
+ | 1.3131 | 12800 | 0.0006 | - |
655
+ | 1.3182 | 12850 | 0.0006 | - |
656
+ | 1.3233 | 12900 | 0.0004 | - |
657
+ | 1.3285 | 12950 | 0.0015 | - |
658
+ | 1.3336 | 13000 | 0.0013 | - |
659
+ | 1.3387 | 13050 | 0.0008 | - |
660
+ | 1.3439 | 13100 | 0.0010 | - |
661
+ | 1.3490 | 13150 | 0.0007 | - |
662
+ | 1.3541 | 13200 | 0.0013 | - |
663
+ | 1.3593 | 13250 | 0.0005 | - |
664
+ | 1.3644 | 13300 | 0.0017 | - |
665
+ | 1.3695 | 13350 | 0.0020 | - |
666
+ | 1.3746 | 13400 | 0.0004 | - |
667
+ | 1.3798 | 13450 | 0.0004 | - |
668
+ | 1.3849 | 13500 | 0.0009 | - |
669
+ | 1.3900 | 13550 | 0.0004 | - |
670
+ | 1.3952 | 13600 | 0.0009 | - |
671
+ | 1.4003 | 13650 | 0.0004 | - |
672
+ | 1.4054 | 13700 | 0.0007 | - |
673
+ | 1.4105 | 13750 | 0.0013 | - |
674
+ | 1.4157 | 13800 | 0.0007 | - |
675
+ | 1.4208 | 13850 | 0.0010 | - |
676
+ | 1.4259 | 13900 | 0.0010 | - |
677
+ | 1.4311 | 13950 | 0.0008 | - |
678
+ | 1.4362 | 14000 | 0.0009 | - |
679
+ | 1.4413 | 14050 | 0.0015 | - |
680
+ | 1.4465 | 14100 | 0.0008 | - |
681
+ | 1.4516 | 14150 | 0.0007 | - |
682
+ | 1.4567 | 14200 | 0.0007 | - |
683
+ | 1.4618 | 14250 | 0.0011 | - |
684
+ | 1.4670 | 14300 | 0.0023 | - |
685
+ | 1.4721 | 14350 | 0.0011 | - |
686
+ | 1.4772 | 14400 | 0.0004 | - |
687
+ | 1.4824 | 14450 | 0.0009 | - |
688
+ | 1.4875 | 14500 | 0.0005 | - |
689
+ | 1.4926 | 14550 | 0.0009 | - |
690
+ | 1.4977 | 14600 | 0.0004 | - |
691
+ | 1.5029 | 14650 | 0.0004 | - |
692
+ | 1.5080 | 14700 | 0.0004 | - |
693
+ | 1.5131 | 14750 | 0.0011 | - |
694
+ | 1.5183 | 14800 | 0.0020 | - |
695
+ | 1.5234 | 14850 | 0.0007 | - |
696
+ | 1.5285 | 14900 | 0.0005 | - |
697
+ | 1.5336 | 14950 | 0.0004 | - |
698
+ | 1.5388 | 15000 | 0.0004 | - |
699
+ | 1.5439 | 15050 | 0.0009 | - |
700
+ | 1.5490 | 15100 | 0.0005 | - |
701
+ | 1.5542 | 15150 | 0.0006 | - |
702
+ | 1.5593 | 15200 | 0.0008 | - |
703
+ | 1.5644 | 15250 | 0.0006 | - |
704
+ | 1.5696 | 15300 | 0.0023 | - |
705
+ | 1.5747 | 15350 | 0.0006 | - |
706
+ | 1.5798 | 15400 | 0.0006 | - |
707
+ | 1.5849 | 15450 | 0.0013 | - |
708
+ | 1.5901 | 15500 | 0.0006 | - |
709
+ | 1.5952 | 15550 | 0.0004 | - |
710
+ | 1.6003 | 15600 | 0.0020 | - |
711
+ | 1.6055 | 15650 | 0.0016 | - |
712
+ | 1.6106 | 15700 | 0.0005 | - |
713
+ | 1.6157 | 15750 | 0.0004 | - |
714
+ | 1.6208 | 15800 | 0.0011 | - |
715
+ | 1.6260 | 15850 | 0.0014 | - |
716
+ | 1.6311 | 15900 | 0.0005 | - |
717
+ | 1.6362 | 15950 | 0.0011 | - |
718
+ | 1.6414 | 16000 | 0.0007 | - |
719
+ | 1.6465 | 16050 | 0.0005 | - |
720
+ | 1.6516 | 16100 | 0.0008 | - |
721
+ | 1.6568 | 16150 | 0.0003 | - |
722
+ | 1.6619 | 16200 | 0.0012 | - |
723
+ | 1.6670 | 16250 | 0.0004 | - |
724
+ | 1.6721 | 16300 | 0.0003 | - |
725
+ | 1.6773 | 16350 | 0.0003 | - |
726
+ | 1.6824 | 16400 | 0.0022 | - |
727
+ | 1.6875 | 16450 | 0.0004 | - |
728
+ | 1.6927 | 16500 | 0.0005 | - |
729
+ | 1.6978 | 16550 | 0.0004 | - |
730
+ | 1.7029 | 16600 | 0.0006 | - |
731
+ | 1.7080 | 16650 | 0.0007 | - |
732
+ | 1.7132 | 16700 | 0.0009 | - |
733
+ | 1.7183 | 16750 | 0.0004 | - |
734
+ | 1.7234 | 16800 | 0.0014 | - |
735
+ | 1.7286 | 16850 | 0.0012 | - |
736
+ | 1.7337 | 16900 | 0.0005 | - |
737
+ | 1.7388 | 16950 | 0.0008 | - |
738
+ | 1.7439 | 17000 | 0.0006 | - |
739
+ | 1.7491 | 17050 | 0.0007 | - |
740
+ | 1.7542 | 17100 | 0.0004 | - |
741
+ | 1.7593 | 17150 | 0.0012 | - |
742
+ | 1.7645 | 17200 | 0.0013 | - |
743
+ | 1.7696 | 17250 | 0.0010 | - |
744
+ | 1.7747 | 17300 | 0.0009 | - |
745
+ | 1.7799 | 17350 | 0.0004 | - |
746
+ | 1.7850 | 17400 | 0.0003 | - |
747
+ | 1.7901 | 17450 | 0.0007 | - |
748
+ | 1.7952 | 17500 | 0.0004 | - |
749
+ | 1.8004 | 17550 | 0.0004 | - |
750
+ | 1.8055 | 17600 | 0.0005 | - |
751
+ | 1.8106 | 17650 | 0.0004 | - |
752
+ | 1.8158 | 17700 | 0.0003 | - |
753
+ | 1.8209 | 17750 | 0.0004 | - |
754
+ | 1.8260 | 17800 | 0.0014 | - |
755
+ | 1.8311 | 17850 | 0.0004 | - |
756
+ | 1.8363 | 17900 | 0.0008 | - |
757
+ | 1.8414 | 17950 | 0.0004 | - |
758
+ | 1.8465 | 18000 | 0.0004 | - |
759
+ | 1.8517 | 18050 | 0.0007 | - |
760
+ | 1.8568 | 18100 | 0.0003 | - |
761
+ | 1.8619 | 18150 | 0.0004 | - |
762
+ | 1.8670 | 18200 | 0.0019 | - |
763
+ | 1.8722 | 18250 | 0.0005 | - |
764
+ | 1.8773 | 18300 | 0.0007 | - |
765
+ | 1.8824 | 18350 | 0.0004 | - |
766
+ | 1.8876 | 18400 | 0.0014 | - |
767
+ | 1.8927 | 18450 | 0.0006 | - |
768
+ | 1.8978 | 18500 | 0.0004 | - |
769
+ | 1.9030 | 18550 | 0.0004 | - |
770
+ | 1.9081 | 18600 | 0.0003 | - |
771
+ | 1.9132 | 18650 | 0.0006 | - |
772
+ | 1.9183 | 18700 | 0.0008 | - |
773
+ | 1.9235 | 18750 | 0.0005 | - |
774
+ | 1.9286 | 18800 | 0.0003 | - |
775
+ | 1.9337 | 18850 | 0.0003 | - |
776
+ | 1.9389 | 18900 | 0.0003 | - |
777
+ | 1.9440 | 18950 | 0.0007 | - |
778
+ | 1.9491 | 19000 | 0.0003 | - |
779
+ | 1.9542 | 19050 | 0.0008 | - |
780
+ | 1.9594 | 19100 | 0.0007 | - |
781
+ | 1.9645 | 19150 | 0.0004 | - |
782
+ | 1.9696 | 19200 | 0.0004 | - |
783
+ | 1.9748 | 19250 | 0.0014 | - |
784
+ | 1.9799 | 19300 | 0.0009 | - |
785
+ | 1.9850 | 19350 | 0.0004 | - |
786
+ | 1.9902 | 19400 | 0.0012 | - |
787
+ | 1.9953 | 19450 | 0.0010 | - |
788
+ | 2.0 | 19496 | - | 0.9733 |
789
+
790
+ </details>
791
+
792
+ ### Training Time
793
+ - **Training**: 1.0 hours
794
+ - **Evaluation**: 1.8 seconds
795
+ - **Total**: 1.0 hours
796
+
797
+ ### Framework Versions
798
+ - Python: 3.10.12
799
+ - Sentence Transformers: 6.1.0
800
+ - Transformers: 5.17.0
801
+ - PyTorch: 2.14.0+cu130
802
+ - Accelerate: 1.15.0
803
+ - Datasets: 5.0.1
804
+ - Tokenizers: 0.23.2
805
+
806
+ ## Additional Resources
807
+
808
+ - [Training and Finetuning Embedding Models with Sentence Transformers](https://huggingface.co/blog/train-sentence-transformers): the end-to-end guide for training or finetuning Sentence Transformer models.
809
+ - [Introduction to Matryoshka Embedding Models](https://huggingface.co/blog/matryoshka): variable-size embeddings that can be truncated with minimal quality loss.
810
+ - [Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval](https://huggingface.co/blog/embedding-quantization): post-training compression of embedding vectors.
811
+ - [Multimodal Embedding & Reranker Models with Sentence Transformers](https://huggingface.co/blog/multimodal-sentence-transformers): use text, image, audio, and video models through the same API.
812
+ - [Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers](https://huggingface.co/blog/train-multimodal-sentence-transformers): train multimodal embedding models, with a Visual Document Retrieval walkthrough.
813
+
814
+ ## Citation
815
+
816
+ ### BibTeX
817
+
818
+ #### Sentence Transformers
819
+ ```bibtex
820
+ @inproceedings{reimers-2019-sentence-bert,
821
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
822
+ author = "Reimers, Nils and Gurevych, Iryna",
823
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
824
+ month = "11",
825
+ year = "2019",
826
+ publisher = "Association for Computational Linguistics",
827
+ url = "https://arxiv.org/abs/1908.10084",
828
+ }
829
+ ```
830
+
831
+ #### MultipleNegativesRankingLoss
832
+ ```bibtex
833
+ @misc{oord2019representationlearningcontrastivepredictive,
834
+ title={Representation Learning with Contrastive Predictive Coding},
835
+ author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
836
+ year={2019},
837
+ eprint={1807.03748},
838
+ archivePrefix={arXiv},
839
+ primaryClass={cs.LG},
840
+ url={https://arxiv.org/abs/1807.03748},
841
+ }
842
+ ```
843
+
844
+ <!--
845
+ ## Glossary
846
+
847
+ *Clearly define terms in order to be accessible across audiences.*
848
+ -->
849
+
850
+ <!--
851
+ ## Model Card Authors
852
+
853
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
854
+ -->
855
+
856
+ <!--
857
+ ## Model Card Contact
858
+
859
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
860
+ -->
checkpoint-19496/1_Pooling/config.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "embedding_dimension": 768,
3
+ "pooling_mode": "cls",
4
+ "include_prompt": true
5
+ }
checkpoint-19496/2_Dense/config.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "in_features": 768,
3
+ "out_features": 768,
4
+ "bias": true,
5
+ "activation_function": "torch.nn.modules.activation.Tanh",
6
+ "module_input_name": "sentence_embedding",
7
+ "module_output_name": "sentence_embedding"
8
+ }
checkpoint-19496/2_Dense/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ffb7f44524dc870beaae5f90090cef75366ae2387053696c402b575cc624d41b
3
+ size 2362528
checkpoint-19496/3_Normalize/config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "module_input_name": "sentence_embedding",
3
+ "module_output_name": "sentence_embedding"
4
+ }
checkpoint-19496/README.md ADDED
@@ -0,0 +1,860 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - sentence-transformers
4
+ - sentence-similarity
5
+ - feature-extraction
6
+ - dense
7
+ - generated_from_trainer
8
+ - dataset_size:623829
9
+ - loss:MultipleNegativesRankingLoss
10
+ base_model: sentence-transformers/LaBSE
11
+ widget:
12
+ - source_sentence: Uptown Pub! | 1589 Clover Lane, Wichita Falls, TX
13
+ sentences:
14
+ - Moa LLC Services | 1964 341, VEEDERSBURG, IN
15
+ - Wong Peak-(One) | 2658 61, Vincennes, Indiana
16
+ - 'Uptown Pub! Trading | '
17
+ - source_sentence: Village Of | 375 Broadway, Unit 2-G08, Lynnfield, MA
18
+ sentences:
19
+ - ROCKY GUILD | 260 2ND AVENUE, FORSYTH, MT
20
+ - 'Village Of Center | '
21
+ - Vio L.L.C. | 415 2TH AVENUE, WASHINGTON, IA
22
+ - source_sentence: St. Fellowship | 1822 Mayfair Drive, City Of Appleton, WI
23
+ sentences:
24
+ - Sanders Adgno | 3606 LORNE DR, ENDWELL, NY
25
+ - 'Quality Opendoor | '
26
+ - 'St. Fellowship | '
27
+ - source_sentence: Ross and Trinidad Inc | AR, Bryant, 6328 Lena Drive
28
+ sentences:
29
+ - 'STEINMEYER,-WINI, SERVICES | '
30
+ - 'Ross and Trinidad Inc. | '
31
+ - Zium [Ltd.] | 3215- CROSSFELL RD, SRING, TX
32
+ - source_sentence: Stag | 122 Bruce Street, Scotia, NY
33
+ sentences:
34
+ - Stag Corporation | BRUCE STREET, SCOTIA, NY
35
+ - ORTHOPEDIC PHYSICIANS | 75, ALTOONA, AL
36
+ - '#2 HUB CO | 142 COTTAGE STREET, LEWISTON, ME'
37
+ pipeline_tag: sentence-similarity
38
+ library_name: sentence-transformers
39
+ metrics:
40
+ - cosine_accuracy@1
41
+ - cosine_accuracy@5
42
+ - cosine_accuracy@10
43
+ - cosine_accuracy@30
44
+ - cosine_precision@1
45
+ - cosine_precision@3
46
+ - cosine_precision@5
47
+ - cosine_precision@10
48
+ - cosine_recall@1
49
+ - cosine_recall@3
50
+ - cosine_recall@5
51
+ - cosine_recall@10
52
+ - cosine_ndcg@10
53
+ - cosine_mrr@10
54
+ - cosine_map@100
55
+ model-index:
56
+ - name: SentenceTransformer based on sentence-transformers/LaBSE
57
+ results:
58
+ - task:
59
+ type: information-retrieval
60
+ name: Information Retrieval
61
+ dataset:
62
+ name: biencoder val
63
+ type: biencoder-val
64
+ metrics:
65
+ - type: cosine_accuracy@1
66
+ value: 0.929
67
+ name: Cosine Accuracy@1
68
+ - type: cosine_accuracy@5
69
+ value: 1.0
70
+ name: Cosine Accuracy@5
71
+ - type: cosine_accuracy@10
72
+ value: 1.0
73
+ name: Cosine Accuracy@10
74
+ - type: cosine_accuracy@30
75
+ value: 1.0
76
+ name: Cosine Accuracy@30
77
+ - type: cosine_precision@1
78
+ value: 0.929
79
+ name: Cosine Precision@1
80
+ - type: cosine_precision@3
81
+ value: 0.33333333333333326
82
+ name: Cosine Precision@3
83
+ - type: cosine_precision@5
84
+ value: 0.20000000000000004
85
+ name: Cosine Precision@5
86
+ - type: cosine_precision@10
87
+ value: 0.10000000000000002
88
+ name: Cosine Precision@10
89
+ - type: cosine_recall@1
90
+ value: 0.929
91
+ name: Cosine Recall@1
92
+ - type: cosine_recall@3
93
+ value: 1.0
94
+ name: Cosine Recall@3
95
+ - type: cosine_recall@5
96
+ value: 1.0
97
+ name: Cosine Recall@5
98
+ - type: cosine_recall@10
99
+ value: 1.0
100
+ name: Cosine Recall@10
101
+ - type: cosine_ndcg@10
102
+ value: 0.9732722934892877
103
+ name: Cosine Ndcg@10
104
+ - type: cosine_mrr@10
105
+ value: 0.9638333333333332
106
+ name: Cosine Mrr@10
107
+ - type: cosine_map@100
108
+ value: 0.9638333333333332
109
+ name: Cosine Map@100
110
+ ---
111
+
112
+ # SentenceTransformer based on sentence-transformers/LaBSE
113
+
114
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/LaBSE](https://huggingface.co/sentence-transformers/LaBSE). It maps inputs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.
115
+
116
+ ## Model Details
117
+
118
+ ### Model Description
119
+ - **Model Type:** Sentence Transformer
120
+ - **Base model:** [sentence-transformers/LaBSE](https://huggingface.co/sentence-transformers/LaBSE) <!-- at revision 836121a0533e5664b21c7aacc5d22951f2b8b25b -->
121
+ - **Maximum Sequence Length:** 256 tokens
122
+ - **Output Dimensionality:** 768 dimensions
123
+ - **Similarity Function:** Cosine Similarity
124
+ - **Supported Modality:** Text
125
+ <!-- - **Training Dataset:** Unknown -->
126
+ <!-- - **Language:** Unknown -->
127
+ <!-- - **License:** Unknown -->
128
+
129
+ ### Model Sources
130
+
131
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
132
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
133
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
134
+
135
+ ### Full Model Architecture
136
+
137
+ ```
138
+ SentenceTransformer(
139
+ (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
140
+ (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True})
141
+ (2): Dense({'in_features': 768, 'out_features': 768, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh', 'module_input_name': 'sentence_embedding', 'module_output_name': 'sentence_embedding'})
142
+ (3): Normalize({'module_input_name': 'sentence_embedding', 'module_output_name': 'sentence_embedding'})
143
+ )
144
+ ```
145
+
146
+ ## Usage
147
+
148
+ ### Direct Usage (Sentence Transformers)
149
+
150
+ First install the Sentence Transformers library:
151
+
152
+ ```bash
153
+ pip install -U sentence-transformers
154
+ ```
155
+ Then you can load this model and run inference.
156
+ ```python
157
+ from sentence_transformers import SentenceTransformer
158
+
159
+ # Download from the 🤗 Hub
160
+ model = SentenceTransformer("sentence_transformers_model_id")
161
+ # Run inference
162
+ queries = [
163
+ 'Stag | 122 Bruce Street, Scotia, NY',
164
+ ]
165
+ documents = [
166
+ 'Stag Corporation | BRUCE STREET, SCOTIA, NY',
167
+ '#2 HUB CO | 142 COTTAGE STREET, LEWISTON, ME',
168
+ 'ORTHOPEDIC PHYSICIANS | 75, ALTOONA, AL',
169
+ ]
170
+ query_embeddings = model.encode_query(queries)
171
+ document_embeddings = model.encode_document(documents)
172
+ print(query_embeddings.shape, document_embeddings.shape)
173
+ # [1, 768] [3, 768]
174
+
175
+ # Get the similarity scores for the embeddings
176
+ similarities = model.similarity(query_embeddings, document_embeddings)
177
+ print(similarities)
178
+ # tensor([[ 0.7677, 0.1634, -0.0688]])
179
+ ```
180
+ <!--
181
+ ### Direct Usage (Transformers)
182
+
183
+ <details><summary>Click to see the direct usage in Transformers</summary>
184
+
185
+ </details>
186
+ -->
187
+
188
+ <!--
189
+ ### Downstream Usage (Sentence Transformers)
190
+
191
+ You can finetune this model on your own dataset.
192
+
193
+ <details><summary>Click to expand</summary>
194
+
195
+ </details>
196
+ -->
197
+
198
+ <!--
199
+ ### Out-of-Scope Use
200
+
201
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
202
+ -->
203
+
204
+ ## Evaluation
205
+
206
+ ### Metrics
207
+
208
+ #### Information Retrieval
209
+
210
+ * Dataset: `biencoder-val`
211
+ * Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.sentence_transformer.evaluation.InformationRetrievalEvaluator)
212
+
213
+ | Metric | Value |
214
+ |:--------------------|:-----------|
215
+ | cosine_accuracy@1 | 0.929 |
216
+ | cosine_accuracy@5 | 1.0 |
217
+ | cosine_accuracy@10 | 1.0 |
218
+ | cosine_accuracy@30 | 1.0 |
219
+ | cosine_precision@1 | 0.929 |
220
+ | cosine_precision@3 | 0.3333 |
221
+ | cosine_precision@5 | 0.2 |
222
+ | cosine_precision@10 | 0.1 |
223
+ | cosine_recall@1 | 0.929 |
224
+ | cosine_recall@3 | 1.0 |
225
+ | cosine_recall@5 | 1.0 |
226
+ | cosine_recall@10 | 1.0 |
227
+ | **cosine_ndcg@10** | **0.9733** |
228
+ | cosine_mrr@10 | 0.9638 |
229
+ | cosine_map@100 | 0.9638 |
230
+
231
+ <!--
232
+ ## Bias, Risks and Limitations
233
+
234
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
235
+ -->
236
+
237
+ <!--
238
+ ### Recommendations
239
+
240
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
241
+ -->
242
+
243
+ ## Training Details
244
+
245
+ ### Training Dataset
246
+
247
+ #### Unnamed Dataset
248
+
249
+ * Size: 623,829 training samples
250
+ * Columns: <code>anchor</code> and <code>positive</code>
251
+ * Approximate statistics based on the first 100 samples:
252
+ | | anchor | positive |
253
+ |:---------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
254
+ | type | string | string |
255
+ | modality | text | text |
256
+ | details | <ul><li>min: 18 tokens</li><li>mean: 31.59 tokens</li><li>max: 55 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 33.45 tokens</li><li>max: 64 tokens</li></ul> |
257
+ * Samples:
258
+ | anchor | positive |
259
+ |:----------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------|
260
+ | <code>Bangalore South Automotive Private Limited \| C-901, Goyalco Orchid, Lakeview, Bangalore South, Bangalore, Karnataka</code> | <code>Bangalore South Automotive Limited Private \| C-901., GOYALCO ORCHID, LAKEVIEW, BANGALORE SOUTH, ಕರ್ನಾಟಕ</code> |
261
+ | <code>Bangalore South Automotive Private Limited \| C-901, Goyalco Orchid, Lakeview, Bangalore South, Bangalore, Karnataka</code> | <code>Bangalore Sóuth Automotive Private Limited \| </code> |
262
+ | <code>Software Pinnacle Tourism \| 3016/5, 1St Floor, Ranjit Nagar, New Delhi, Central Delhi, Delhi</code> | <code>Software Pinnacle Tóurism \| A-3016/5, 1ST FLOOR, RANJIT NAGAR, NEW DELHI, दिल्ली</code> |
263
+ * Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
264
+ ```json
265
+ {
266
+ "scale": 20.0,
267
+ "similarity_fct": "cos_sim",
268
+ "gather_across_devices": false,
269
+ "directions": [
270
+ "query_to_doc"
271
+ ],
272
+ "partition_mode": "joint",
273
+ "hardness_mode": null,
274
+ "hardness_strength": 0.0
275
+ }
276
+ ```
277
+
278
+ ### Training Hyperparameters
279
+ #### Non-Default Hyperparameters
280
+
281
+ - `per_device_train_batch_size`: 64
282
+ - `num_train_epochs`: 2
283
+ - `learning_rate`: 2e-05
284
+ - `warmup_steps`: 0.1
285
+
286
+ #### All Hyperparameters
287
+ <details><summary>Click to expand</summary>
288
+
289
+ - `per_device_train_batch_size`: 64
290
+ - `num_train_epochs`: 2
291
+ - `max_steps`: -1
292
+ - `learning_rate`: 2e-05
293
+ - `lr_scheduler_type`: linear
294
+ - `lr_scheduler_kwargs`: None
295
+ - `warmup_steps`: 0.1
296
+ - `optim`: adamw_torch_fused
297
+ - `optim_args`: None
298
+ - `weight_decay`: 0.0
299
+ - `adam_beta1`: 0.9
300
+ - `adam_beta2`: 0.999
301
+ - `adam_epsilon`: 1e-08
302
+ - `optim_target_modules`: None
303
+ - `gradient_accumulation_steps`: 1
304
+ - `average_tokens_across_devices`: True
305
+ - `max_grad_norm`: 1.0
306
+ - `label_smoothing_factor`: 0.0
307
+ - `bf16`: False
308
+ - `fp16`: False
309
+ - `bf16_full_eval`: False
310
+ - `fp16_full_eval`: False
311
+ - `tf32`: None
312
+ - `gradient_checkpointing`: False
313
+ - `gradient_checkpointing_kwargs`: None
314
+ - `torch_compile`: False
315
+ - `torch_compile_backend`: None
316
+ - `torch_compile_mode`: None
317
+ - `use_liger_kernel`: False
318
+ - `liger_kernel_config`: None
319
+ - `use_cache`: False
320
+ - `neftune_noise_alpha`: None
321
+ - `torch_empty_cache_steps`: None
322
+ - `auto_find_batch_size`: False
323
+ - `log_on_each_node`: True
324
+ - `logging_nan_inf_filter`: True
325
+ - `include_num_input_tokens_seen`: no
326
+ - `log_level`: passive
327
+ - `log_level_replica`: warning
328
+ - `disable_tqdm`: False
329
+ - `project`: huggingface
330
+ - `trackio_space_id`: None
331
+ - `trackio_bucket_id`: None
332
+ - `trackio_static_space_id`: None
333
+ - `per_device_eval_batch_size`: 8
334
+ - `prediction_loss_only`: True
335
+ - `eval_on_start`: False
336
+ - `eval_do_concat_batches`: True
337
+ - `eval_use_gather_object`: False
338
+ - `eval_accumulation_steps`: None
339
+ - `include_for_metrics`: []
340
+ - `batch_eval_metrics`: False
341
+ - `save_only_model`: False
342
+ - `save_on_each_node`: False
343
+ - `enable_jit_checkpoint`: False
344
+ - `push_to_hub`: False
345
+ - `hub_private_repo`: None
346
+ - `hub_model_id`: None
347
+ - `hub_strategy`: every_save
348
+ - `hub_always_push`: False
349
+ - `hub_revision`: None
350
+ - `load_best_model_at_end`: False
351
+ - `ignore_data_skip`: False
352
+ - `restore_callback_states_from_checkpoint`: False
353
+ - `full_determinism`: False
354
+ - `seed`: 42
355
+ - `data_seed`: None
356
+ - `use_cpu`: False
357
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
358
+ - `parallelism_config`: None
359
+ - `dataloader_drop_last`: False
360
+ - `dataloader_num_workers`: 0
361
+ - `dataloader_pin_memory`: True
362
+ - `dataloader_persistent_workers`: False
363
+ - `dataloader_prefetch_factor`: None
364
+ - `dataloader_multiprocessing_context`: None
365
+ - `dataloader_in_order`: True
366
+ - `remove_unused_columns`: True
367
+ - `label_names`: None
368
+ - `train_sampling_strategy`: random
369
+ - `length_column_name`: length
370
+ - `ddp_find_unused_parameters`: None
371
+ - `ddp_bucket_cap_mb`: None
372
+ - `ddp_broadcast_buffers`: False
373
+ - `ddp_static_graph`: None
374
+ - `ddp_backend`: None
375
+ - `ddp_timeout`: 1800
376
+ - `fsdp`: None
377
+ - `fsdp_config`: None
378
+ - `deepspeed`: None
379
+ - `debug`: []
380
+ - `skip_memory_metrics`: True
381
+ - `do_predict`: False
382
+ - `resume_from_checkpoint`: None
383
+ - `local_rank`: -1
384
+ - `prompts`: None
385
+ - `batch_sampler`: batch_sampler
386
+ - `multi_dataset_batch_sampler`: proportional
387
+ - `router_mapping`: {}
388
+ - `learning_rate_mapping`: {}
389
+ - `warmup_ratio`: None
390
+
391
+ </details>
392
+
393
+ ### Training Logs
394
+ <details><summary>Click to expand</summary>
395
+
396
+ | Epoch | Step | Training Loss | biencoder-val_cosine_ndcg@10 |
397
+ |:------:|:-----:|:-------------:|:----------------------------:|
398
+ | 0.0051 | 50 | 0.2099 | - |
399
+ | 0.0103 | 100 | 0.0801 | - |
400
+ | 0.0154 | 150 | 0.0250 | - |
401
+ | 0.0205 | 200 | 0.0196 | - |
402
+ | 0.0256 | 250 | 0.0155 | - |
403
+ | 0.0308 | 300 | 0.0151 | - |
404
+ | 0.0359 | 350 | 0.0150 | - |
405
+ | 0.0410 | 400 | 0.0117 | - |
406
+ | 0.0462 | 450 | 0.0124 | - |
407
+ | 0.0513 | 500 | 0.0111 | - |
408
+ | 0.0564 | 550 | 0.0070 | - |
409
+ | 0.0616 | 600 | 0.0107 | - |
410
+ | 0.0667 | 650 | 0.0091 | - |
411
+ | 0.0718 | 700 | 0.0060 | - |
412
+ | 0.0769 | 750 | 0.0094 | - |
413
+ | 0.0821 | 800 | 0.0067 | - |
414
+ | 0.0872 | 850 | 0.0061 | - |
415
+ | 0.0923 | 900 | 0.0076 | - |
416
+ | 0.0975 | 950 | 0.0060 | - |
417
+ | 0.1026 | 1000 | 0.0047 | - |
418
+ | 0.1077 | 1050 | 0.0071 | - |
419
+ | 0.1128 | 1100 | 0.0063 | - |
420
+ | 0.1180 | 1150 | 0.0046 | - |
421
+ | 0.1231 | 1200 | 0.0039 | - |
422
+ | 0.1282 | 1250 | 0.0049 | - |
423
+ | 0.1334 | 1300 | 0.0055 | - |
424
+ | 0.1385 | 1350 | 0.0038 | - |
425
+ | 0.1436 | 1400 | 0.0041 | - |
426
+ | 0.1487 | 1450 | 0.0060 | - |
427
+ | 0.1539 | 1500 | 0.0055 | - |
428
+ | 0.1590 | 1550 | 0.0053 | - |
429
+ | 0.1641 | 1600 | 0.0047 | - |
430
+ | 0.1693 | 1650 | 0.0044 | - |
431
+ | 0.1744 | 1700 | 0.0071 | - |
432
+ | 0.1795 | 1750 | 0.0044 | - |
433
+ | 0.1847 | 1800 | 0.0025 | - |
434
+ | 0.1898 | 1850 | 0.0055 | - |
435
+ | 0.1949 | 1900 | 0.0053 | - |
436
+ | 0.2000 | 1950 | 0.0045 | - |
437
+ | 0.2052 | 2000 | 0.0047 | - |
438
+ | 0.2103 | 2050 | 0.0048 | - |
439
+ | 0.2154 | 2100 | 0.0041 | - |
440
+ | 0.2206 | 2150 | 0.0030 | - |
441
+ | 0.2257 | 2200 | 0.0037 | - |
442
+ | 0.2308 | 2250 | 0.0047 | - |
443
+ | 0.2359 | 2300 | 0.0042 | - |
444
+ | 0.2411 | 2350 | 0.0027 | - |
445
+ | 0.2462 | 2400 | 0.0047 | - |
446
+ | 0.2513 | 2450 | 0.0028 | - |
447
+ | 0.2565 | 2500 | 0.0043 | - |
448
+ | 0.2616 | 2550 | 0.0063 | - |
449
+ | 0.2667 | 2600 | 0.0027 | - |
450
+ | 0.2719 | 2650 | 0.0021 | - |
451
+ | 0.2770 | 2700 | 0.0028 | - |
452
+ | 0.2821 | 2750 | 0.0024 | - |
453
+ | 0.2872 | 2800 | 0.0045 | - |
454
+ | 0.2924 | 2850 | 0.0033 | - |
455
+ | 0.2975 | 2900 | 0.0048 | - |
456
+ | 0.3026 | 2950 | 0.0021 | - |
457
+ | 0.3078 | 3000 | 0.0031 | - |
458
+ | 0.3129 | 3050 | 0.0027 | - |
459
+ | 0.3180 | 3100 | 0.0033 | - |
460
+ | 0.3231 | 3150 | 0.0040 | - |
461
+ | 0.3283 | 3200 | 0.0012 | - |
462
+ | 0.3334 | 3250 | 0.0031 | - |
463
+ | 0.3385 | 3300 | 0.0033 | - |
464
+ | 0.3437 | 3350 | 0.0020 | - |
465
+ | 0.3488 | 3400 | 0.0028 | - |
466
+ | 0.3539 | 3450 | 0.0022 | - |
467
+ | 0.3590 | 3500 | 0.0025 | - |
468
+ | 0.3642 | 3550 | 0.0026 | - |
469
+ | 0.3693 | 3600 | 0.0028 | - |
470
+ | 0.3744 | 3650 | 0.0035 | - |
471
+ | 0.3796 | 3700 | 0.0023 | - |
472
+ | 0.3847 | 3750 | 0.0029 | - |
473
+ | 0.3898 | 3800 | 0.0031 | - |
474
+ | 0.3950 | 3850 | 0.0042 | - |
475
+ | 0.4001 | 3900 | 0.0027 | - |
476
+ | 0.4052 | 3950 | 0.0035 | - |
477
+ | 0.4103 | 4000 | 0.0026 | - |
478
+ | 0.4155 | 4050 | 0.0017 | - |
479
+ | 0.4206 | 4100 | 0.0022 | - |
480
+ | 0.4257 | 4150 | 0.0020 | - |
481
+ | 0.4309 | 4200 | 0.0025 | - |
482
+ | 0.4360 | 4250 | 0.0024 | - |
483
+ | 0.4411 | 4300 | 0.0025 | - |
484
+ | 0.4462 | 4350 | 0.0025 | - |
485
+ | 0.4514 | 4400 | 0.0021 | - |
486
+ | 0.4565 | 4450 | 0.0011 | - |
487
+ | 0.4616 | 4500 | 0.0018 | - |
488
+ | 0.4668 | 4550 | 0.0039 | - |
489
+ | 0.4719 | 4600 | 0.0016 | - |
490
+ | 0.4770 | 4650 | 0.0023 | - |
491
+ | 0.4822 | 4700 | 0.0012 | - |
492
+ | 0.4873 | 4750 | 0.0027 | - |
493
+ | 0.4924 | 4800 | 0.0020 | - |
494
+ | 0.4975 | 4850 | 0.0022 | - |
495
+ | 0.5027 | 4900 | 0.0022 | - |
496
+ | 0.5078 | 4950 | 0.0016 | - |
497
+ | 0.5129 | 5000 | 0.0027 | - |
498
+ | 0.5181 | 5050 | 0.0017 | - |
499
+ | 0.5232 | 5100 | 0.0020 | - |
500
+ | 0.5283 | 5150 | 0.0024 | - |
501
+ | 0.5334 | 5200 | 0.0024 | - |
502
+ | 0.5386 | 5250 | 0.0024 | - |
503
+ | 0.5437 | 5300 | 0.0022 | - |
504
+ | 0.5488 | 5350 | 0.0029 | - |
505
+ | 0.5540 | 5400 | 0.0016 | - |
506
+ | 0.5591 | 5450 | 0.0010 | - |
507
+ | 0.5642 | 5500 | 0.0017 | - |
508
+ | 0.5693 | 5550 | 0.0029 | - |
509
+ | 0.5745 | 5600 | 0.0017 | - |
510
+ | 0.5796 | 5650 | 0.0021 | - |
511
+ | 0.5847 | 5700 | 0.0019 | - |
512
+ | 0.5899 | 5750 | 0.0021 | - |
513
+ | 0.5950 | 5800 | 0.0030 | - |
514
+ | 0.6001 | 5850 | 0.0015 | - |
515
+ | 0.6053 | 5900 | 0.0028 | - |
516
+ | 0.6104 | 5950 | 0.0022 | - |
517
+ | 0.6155 | 6000 | 0.0020 | - |
518
+ | 0.6206 | 6050 | 0.0021 | - |
519
+ | 0.6258 | 6100 | 0.0017 | - |
520
+ | 0.6309 | 6150 | 0.0021 | - |
521
+ | 0.6360 | 6200 | 0.0015 | - |
522
+ | 0.6412 | 6250 | 0.0010 | - |
523
+ | 0.6463 | 6300 | 0.0018 | - |
524
+ | 0.6514 | 6350 | 0.0011 | - |
525
+ | 0.6565 | 6400 | 0.0035 | - |
526
+ | 0.6617 | 6450 | 0.0015 | - |
527
+ | 0.6668 | 6500 | 0.0022 | - |
528
+ | 0.6719 | 6550 | 0.0019 | - |
529
+ | 0.6771 | 6600 | 0.0017 | - |
530
+ | 0.6822 | 6650 | 0.0011 | - |
531
+ | 0.6873 | 6700 | 0.0021 | - |
532
+ | 0.6924 | 6750 | 0.0017 | - |
533
+ | 0.6976 | 6800 | 0.0014 | - |
534
+ | 0.7027 | 6850 | 0.0024 | - |
535
+ | 0.7078 | 6900 | 0.0012 | - |
536
+ | 0.7130 | 6950 | 0.0032 | - |
537
+ | 0.7181 | 7000 | 0.0020 | - |
538
+ | 0.7232 | 7050 | 0.0013 | - |
539
+ | 0.7284 | 7100 | 0.0038 | - |
540
+ | 0.7335 | 7150 | 0.0018 | - |
541
+ | 0.7386 | 7200 | 0.0019 | - |
542
+ | 0.7437 | 7250 | 0.0038 | - |
543
+ | 0.7489 | 7300 | 0.0013 | - |
544
+ | 0.7540 | 7350 | 0.0025 | - |
545
+ | 0.7591 | 7400 | 0.0022 | - |
546
+ | 0.7643 | 7450 | 0.0014 | - |
547
+ | 0.7694 | 7500 | 0.0015 | - |
548
+ | 0.7745 | 7550 | 0.0030 | - |
549
+ | 0.7796 | 7600 | 0.0016 | - |
550
+ | 0.7848 | 7650 | 0.0017 | - |
551
+ | 0.7899 | 7700 | 0.0013 | - |
552
+ | 0.7950 | 7750 | 0.0036 | - |
553
+ | 0.8002 | 7800 | 0.0022 | - |
554
+ | 0.8053 | 7850 | 0.0015 | - |
555
+ | 0.8104 | 7900 | 0.0010 | - |
556
+ | 0.8156 | 7950 | 0.0021 | - |
557
+ | 0.8207 | 8000 | 0.0019 | - |
558
+ | 0.8258 | 8050 | 0.0011 | - |
559
+ | 0.8309 | 8100 | 0.0012 | - |
560
+ | 0.8361 | 8150 | 0.0026 | - |
561
+ | 0.8412 | 8200 | 0.0011 | - |
562
+ | 0.8463 | 8250 | 0.0019 | - |
563
+ | 0.8515 | 8300 | 0.0017 | - |
564
+ | 0.8566 | 8350 | 0.0011 | - |
565
+ | 0.8617 | 8400 | 0.0012 | - |
566
+ | 0.8668 | 8450 | 0.0018 | - |
567
+ | 0.8720 | 8500 | 0.0008 | - |
568
+ | 0.8771 | 8550 | 0.0017 | - |
569
+ | 0.8822 | 8600 | 0.0023 | - |
570
+ | 0.8874 | 8650 | 0.0010 | - |
571
+ | 0.8925 | 8700 | 0.0011 | - |
572
+ | 0.8976 | 8750 | 0.0022 | - |
573
+ | 0.9027 | 8800 | 0.0023 | - |
574
+ | 0.9079 | 8850 | 0.0013 | - |
575
+ | 0.9130 | 8900 | 0.0015 | - |
576
+ | 0.9181 | 8950 | 0.0034 | - |
577
+ | 0.9233 | 9000 | 0.0011 | - |
578
+ | 0.9284 | 9050 | 0.0018 | - |
579
+ | 0.9335 | 9100 | 0.0013 | - |
580
+ | 0.9387 | 9150 | 0.0009 | - |
581
+ | 0.9438 | 9200 | 0.0008 | - |
582
+ | 0.9489 | 9250 | 0.0013 | - |
583
+ | 0.9540 | 9300 | 0.0012 | - |
584
+ | 0.9592 | 9350 | 0.0012 | - |
585
+ | 0.9643 | 9400 | 0.0008 | - |
586
+ | 0.9694 | 9450 | 0.0011 | - |
587
+ | 0.9746 | 9500 | 0.0008 | - |
588
+ | 0.9797 | 9550 | 0.0015 | - |
589
+ | 0.9848 | 9600 | 0.0014 | - |
590
+ | 0.9899 | 9650 | 0.0013 | - |
591
+ | 0.9951 | 9700 | 0.0006 | - |
592
+ | 1.0 | 9748 | - | 0.9733 |
593
+ | 1.0002 | 9750 | 0.0014 | - |
594
+ | 1.0053 | 9800 | 0.0006 | - |
595
+ | 1.0105 | 9850 | 0.0005 | - |
596
+ | 1.0156 | 9900 | 0.0011 | - |
597
+ | 1.0207 | 9950 | 0.0011 | - |
598
+ | 1.0259 | 10000 | 0.0009 | - |
599
+ | 1.0310 | 10050 | 0.0013 | - |
600
+ | 1.0361 | 10100 | 0.0008 | - |
601
+ | 1.0412 | 10150 | 0.0008 | - |
602
+ | 1.0464 | 10200 | 0.0006 | - |
603
+ | 1.0515 | 10250 | 0.0006 | - |
604
+ | 1.0566 | 10300 | 0.0010 | - |
605
+ | 1.0618 | 10350 | 0.0026 | - |
606
+ | 1.0669 | 10400 | 0.0007 | - |
607
+ | 1.0720 | 10450 | 0.0007 | - |
608
+ | 1.0771 | 10500 | 0.0010 | - |
609
+ | 1.0823 | 10550 | 0.0016 | - |
610
+ | 1.0874 | 10600 | 0.0014 | - |
611
+ | 1.0925 | 10650 | 0.0007 | - |
612
+ | 1.0977 | 10700 | 0.0006 | - |
613
+ | 1.1028 | 10750 | 0.0021 | - |
614
+ | 1.1079 | 10800 | 0.0027 | - |
615
+ | 1.1130 | 10850 | 0.0006 | - |
616
+ | 1.1182 | 10900 | 0.0006 | - |
617
+ | 1.1233 | 10950 | 0.0012 | - |
618
+ | 1.1284 | 11000 | 0.0016 | - |
619
+ | 1.1336 | 11050 | 0.0015 | - |
620
+ | 1.1387 | 11100 | 0.0026 | - |
621
+ | 1.1438 | 11150 | 0.0008 | - |
622
+ | 1.1490 | 11200 | 0.0007 | - |
623
+ | 1.1541 | 11250 | 0.0007 | - |
624
+ | 1.1592 | 11300 | 0.0012 | - |
625
+ | 1.1643 | 11350 | 0.0010 | - |
626
+ | 1.1695 | 11400 | 0.0005 | - |
627
+ | 1.1746 | 11450 | 0.0008 | - |
628
+ | 1.1797 | 11500 | 0.0006 | - |
629
+ | 1.1849 | 11550 | 0.0005 | - |
630
+ | 1.1900 | 11600 | 0.0016 | - |
631
+ | 1.1951 | 11650 | 0.0009 | - |
632
+ | 1.2002 | 11700 | 0.0011 | - |
633
+ | 1.2054 | 11750 | 0.0006 | - |
634
+ | 1.2105 | 11800 | 0.0014 | - |
635
+ | 1.2156 | 11850 | 0.0006 | - |
636
+ | 1.2208 | 11900 | 0.0006 | - |
637
+ | 1.2259 | 11950 | 0.0015 | - |
638
+ | 1.2310 | 12000 | 0.0005 | - |
639
+ | 1.2362 | 12050 | 0.0037 | - |
640
+ | 1.2413 | 12100 | 0.0006 | - |
641
+ | 1.2464 | 12150 | 0.0005 | - |
642
+ | 1.2515 | 12200 | 0.0007 | - |
643
+ | 1.2567 | 12250 | 0.0016 | - |
644
+ | 1.2618 | 12300 | 0.0006 | - |
645
+ | 1.2669 | 12350 | 0.0015 | - |
646
+ | 1.2721 | 12400 | 0.0005 | - |
647
+ | 1.2772 | 12450 | 0.0007 | - |
648
+ | 1.2823 | 12500 | 0.0006 | - |
649
+ | 1.2874 | 12550 | 0.0009 | - |
650
+ | 1.2926 | 12600 | 0.0005 | - |
651
+ | 1.2977 | 12650 | 0.0004 | - |
652
+ | 1.3028 | 12700 | 0.0011 | - |
653
+ | 1.3080 | 12750 | 0.0006 | - |
654
+ | 1.3131 | 12800 | 0.0006 | - |
655
+ | 1.3182 | 12850 | 0.0006 | - |
656
+ | 1.3233 | 12900 | 0.0004 | - |
657
+ | 1.3285 | 12950 | 0.0015 | - |
658
+ | 1.3336 | 13000 | 0.0013 | - |
659
+ | 1.3387 | 13050 | 0.0008 | - |
660
+ | 1.3439 | 13100 | 0.0010 | - |
661
+ | 1.3490 | 13150 | 0.0007 | - |
662
+ | 1.3541 | 13200 | 0.0013 | - |
663
+ | 1.3593 | 13250 | 0.0005 | - |
664
+ | 1.3644 | 13300 | 0.0017 | - |
665
+ | 1.3695 | 13350 | 0.0020 | - |
666
+ | 1.3746 | 13400 | 0.0004 | - |
667
+ | 1.3798 | 13450 | 0.0004 | - |
668
+ | 1.3849 | 13500 | 0.0009 | - |
669
+ | 1.3900 | 13550 | 0.0004 | - |
670
+ | 1.3952 | 13600 | 0.0009 | - |
671
+ | 1.4003 | 13650 | 0.0004 | - |
672
+ | 1.4054 | 13700 | 0.0007 | - |
673
+ | 1.4105 | 13750 | 0.0013 | - |
674
+ | 1.4157 | 13800 | 0.0007 | - |
675
+ | 1.4208 | 13850 | 0.0010 | - |
676
+ | 1.4259 | 13900 | 0.0010 | - |
677
+ | 1.4311 | 13950 | 0.0008 | - |
678
+ | 1.4362 | 14000 | 0.0009 | - |
679
+ | 1.4413 | 14050 | 0.0015 | - |
680
+ | 1.4465 | 14100 | 0.0008 | - |
681
+ | 1.4516 | 14150 | 0.0007 | - |
682
+ | 1.4567 | 14200 | 0.0007 | - |
683
+ | 1.4618 | 14250 | 0.0011 | - |
684
+ | 1.4670 | 14300 | 0.0023 | - |
685
+ | 1.4721 | 14350 | 0.0011 | - |
686
+ | 1.4772 | 14400 | 0.0004 | - |
687
+ | 1.4824 | 14450 | 0.0009 | - |
688
+ | 1.4875 | 14500 | 0.0005 | - |
689
+ | 1.4926 | 14550 | 0.0009 | - |
690
+ | 1.4977 | 14600 | 0.0004 | - |
691
+ | 1.5029 | 14650 | 0.0004 | - |
692
+ | 1.5080 | 14700 | 0.0004 | - |
693
+ | 1.5131 | 14750 | 0.0011 | - |
694
+ | 1.5183 | 14800 | 0.0020 | - |
695
+ | 1.5234 | 14850 | 0.0007 | - |
696
+ | 1.5285 | 14900 | 0.0005 | - |
697
+ | 1.5336 | 14950 | 0.0004 | - |
698
+ | 1.5388 | 15000 | 0.0004 | - |
699
+ | 1.5439 | 15050 | 0.0009 | - |
700
+ | 1.5490 | 15100 | 0.0005 | - |
701
+ | 1.5542 | 15150 | 0.0006 | - |
702
+ | 1.5593 | 15200 | 0.0008 | - |
703
+ | 1.5644 | 15250 | 0.0006 | - |
704
+ | 1.5696 | 15300 | 0.0023 | - |
705
+ | 1.5747 | 15350 | 0.0006 | - |
706
+ | 1.5798 | 15400 | 0.0006 | - |
707
+ | 1.5849 | 15450 | 0.0013 | - |
708
+ | 1.5901 | 15500 | 0.0006 | - |
709
+ | 1.5952 | 15550 | 0.0004 | - |
710
+ | 1.6003 | 15600 | 0.0020 | - |
711
+ | 1.6055 | 15650 | 0.0016 | - |
712
+ | 1.6106 | 15700 | 0.0005 | - |
713
+ | 1.6157 | 15750 | 0.0004 | - |
714
+ | 1.6208 | 15800 | 0.0011 | - |
715
+ | 1.6260 | 15850 | 0.0014 | - |
716
+ | 1.6311 | 15900 | 0.0005 | - |
717
+ | 1.6362 | 15950 | 0.0011 | - |
718
+ | 1.6414 | 16000 | 0.0007 | - |
719
+ | 1.6465 | 16050 | 0.0005 | - |
720
+ | 1.6516 | 16100 | 0.0008 | - |
721
+ | 1.6568 | 16150 | 0.0003 | - |
722
+ | 1.6619 | 16200 | 0.0012 | - |
723
+ | 1.6670 | 16250 | 0.0004 | - |
724
+ | 1.6721 | 16300 | 0.0003 | - |
725
+ | 1.6773 | 16350 | 0.0003 | - |
726
+ | 1.6824 | 16400 | 0.0022 | - |
727
+ | 1.6875 | 16450 | 0.0004 | - |
728
+ | 1.6927 | 16500 | 0.0005 | - |
729
+ | 1.6978 | 16550 | 0.0004 | - |
730
+ | 1.7029 | 16600 | 0.0006 | - |
731
+ | 1.7080 | 16650 | 0.0007 | - |
732
+ | 1.7132 | 16700 | 0.0009 | - |
733
+ | 1.7183 | 16750 | 0.0004 | - |
734
+ | 1.7234 | 16800 | 0.0014 | - |
735
+ | 1.7286 | 16850 | 0.0012 | - |
736
+ | 1.7337 | 16900 | 0.0005 | - |
737
+ | 1.7388 | 16950 | 0.0008 | - |
738
+ | 1.7439 | 17000 | 0.0006 | - |
739
+ | 1.7491 | 17050 | 0.0007 | - |
740
+ | 1.7542 | 17100 | 0.0004 | - |
741
+ | 1.7593 | 17150 | 0.0012 | - |
742
+ | 1.7645 | 17200 | 0.0013 | - |
743
+ | 1.7696 | 17250 | 0.0010 | - |
744
+ | 1.7747 | 17300 | 0.0009 | - |
745
+ | 1.7799 | 17350 | 0.0004 | - |
746
+ | 1.7850 | 17400 | 0.0003 | - |
747
+ | 1.7901 | 17450 | 0.0007 | - |
748
+ | 1.7952 | 17500 | 0.0004 | - |
749
+ | 1.8004 | 17550 | 0.0004 | - |
750
+ | 1.8055 | 17600 | 0.0005 | - |
751
+ | 1.8106 | 17650 | 0.0004 | - |
752
+ | 1.8158 | 17700 | 0.0003 | - |
753
+ | 1.8209 | 17750 | 0.0004 | - |
754
+ | 1.8260 | 17800 | 0.0014 | - |
755
+ | 1.8311 | 17850 | 0.0004 | - |
756
+ | 1.8363 | 17900 | 0.0008 | - |
757
+ | 1.8414 | 17950 | 0.0004 | - |
758
+ | 1.8465 | 18000 | 0.0004 | - |
759
+ | 1.8517 | 18050 | 0.0007 | - |
760
+ | 1.8568 | 18100 | 0.0003 | - |
761
+ | 1.8619 | 18150 | 0.0004 | - |
762
+ | 1.8670 | 18200 | 0.0019 | - |
763
+ | 1.8722 | 18250 | 0.0005 | - |
764
+ | 1.8773 | 18300 | 0.0007 | - |
765
+ | 1.8824 | 18350 | 0.0004 | - |
766
+ | 1.8876 | 18400 | 0.0014 | - |
767
+ | 1.8927 | 18450 | 0.0006 | - |
768
+ | 1.8978 | 18500 | 0.0004 | - |
769
+ | 1.9030 | 18550 | 0.0004 | - |
770
+ | 1.9081 | 18600 | 0.0003 | - |
771
+ | 1.9132 | 18650 | 0.0006 | - |
772
+ | 1.9183 | 18700 | 0.0008 | - |
773
+ | 1.9235 | 18750 | 0.0005 | - |
774
+ | 1.9286 | 18800 | 0.0003 | - |
775
+ | 1.9337 | 18850 | 0.0003 | - |
776
+ | 1.9389 | 18900 | 0.0003 | - |
777
+ | 1.9440 | 18950 | 0.0007 | - |
778
+ | 1.9491 | 19000 | 0.0003 | - |
779
+ | 1.9542 | 19050 | 0.0008 | - |
780
+ | 1.9594 | 19100 | 0.0007 | - |
781
+ | 1.9645 | 19150 | 0.0004 | - |
782
+ | 1.9696 | 19200 | 0.0004 | - |
783
+ | 1.9748 | 19250 | 0.0014 | - |
784
+ | 1.9799 | 19300 | 0.0009 | - |
785
+ | 1.9850 | 19350 | 0.0004 | - |
786
+ | 1.9902 | 19400 | 0.0012 | - |
787
+ | 1.9953 | 19450 | 0.0010 | - |
788
+ | 2.0 | 19496 | - | 0.9733 |
789
+
790
+ </details>
791
+
792
+ ### Training Time
793
+ - **Training**: 1.0 hours
794
+ - **Evaluation**: 1.8 seconds
795
+ - **Total**: 1.0 hours
796
+
797
+ ### Framework Versions
798
+ - Python: 3.10.12
799
+ - Sentence Transformers: 6.1.0
800
+ - Transformers: 5.17.0
801
+ - PyTorch: 2.14.0+cu130
802
+ - Accelerate: 1.15.0
803
+ - Datasets: 5.0.1
804
+ - Tokenizers: 0.23.2
805
+
806
+ ## Additional Resources
807
+
808
+ - [Training and Finetuning Embedding Models with Sentence Transformers](https://huggingface.co/blog/train-sentence-transformers): the end-to-end guide for training or finetuning Sentence Transformer models.
809
+ - [Introduction to Matryoshka Embedding Models](https://huggingface.co/blog/matryoshka): variable-size embeddings that can be truncated with minimal quality loss.
810
+ - [Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval](https://huggingface.co/blog/embedding-quantization): post-training compression of embedding vectors.
811
+ - [Multimodal Embedding & Reranker Models with Sentence Transformers](https://huggingface.co/blog/multimodal-sentence-transformers): use text, image, audio, and video models through the same API.
812
+ - [Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers](https://huggingface.co/blog/train-multimodal-sentence-transformers): train multimodal embedding models, with a Visual Document Retrieval walkthrough.
813
+
814
+ ## Citation
815
+
816
+ ### BibTeX
817
+
818
+ #### Sentence Transformers
819
+ ```bibtex
820
+ @inproceedings{reimers-2019-sentence-bert,
821
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
822
+ author = "Reimers, Nils and Gurevych, Iryna",
823
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
824
+ month = "11",
825
+ year = "2019",
826
+ publisher = "Association for Computational Linguistics",
827
+ url = "https://arxiv.org/abs/1908.10084",
828
+ }
829
+ ```
830
+
831
+ #### MultipleNegativesRankingLoss
832
+ ```bibtex
833
+ @misc{oord2019representationlearningcontrastivepredictive,
834
+ title={Representation Learning with Contrastive Predictive Coding},
835
+ author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
836
+ year={2019},
837
+ eprint={1807.03748},
838
+ archivePrefix={arXiv},
839
+ primaryClass={cs.LG},
840
+ url={https://arxiv.org/abs/1807.03748},
841
+ }
842
+ ```
843
+
844
+ <!--
845
+ ## Glossary
846
+
847
+ *Clearly define terms in order to be accessible across audiences.*
848
+ -->
849
+
850
+ <!--
851
+ ## Model Card Authors
852
+
853
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
854
+ -->
855
+
856
+ <!--
857
+ ## Model Card Contact
858
+
859
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
860
+ -->
checkpoint-19496/config.json ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_cross_attention": false,
3
+ "architectures": [
4
+ "BertModel"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": null,
8
+ "classifier_dropout": null,
9
+ "directionality": "bidi",
10
+ "dtype": "float32",
11
+ "eos_token_id": null,
12
+ "gradient_checkpointing": false,
13
+ "hidden_act": "gelu",
14
+ "hidden_dropout_prob": 0.1,
15
+ "hidden_size": 768,
16
+ "initializer_range": 0.02,
17
+ "intermediate_size": 3072,
18
+ "is_decoder": false,
19
+ "layer_norm_eps": 1e-12,
20
+ "max_position_embeddings": 512,
21
+ "model_type": "bert",
22
+ "num_attention_heads": 12,
23
+ "num_hidden_layers": 12,
24
+ "pad_token_id": 0,
25
+ "pooler_fc_size": 768,
26
+ "pooler_num_attention_heads": 12,
27
+ "pooler_num_fc_layers": 3,
28
+ "pooler_size_per_head": 128,
29
+ "pooler_type": "first_token_transform",
30
+ "position_embedding_type": "absolute",
31
+ "tie_word_embeddings": true,
32
+ "transformers_version": "5.17.0",
33
+ "type_vocab_size": 2,
34
+ "use_cache": false,
35
+ "vocab_size": 501153
36
+ }
checkpoint-19496/config_sentence_transformers.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "pytorch": "2.14.0+cu130",
4
+ "sentence_transformers": "6.1.0",
5
+ "transformers": "5.17.0"
6
+ },
7
+ "default_prompt_name": null,
8
+ "model_type": "SentenceTransformer",
9
+ "prompts": {
10
+ "document": "",
11
+ "query": ""
12
+ },
13
+ "similarity_fn_name": "cosine"
14
+ }
checkpoint-19496/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:880ba175dc3dbb5fee1d6fafe965ccf86f9a8e8971a6250d01f650e46c5f008c
3
+ size 1883730160
checkpoint-19496/modules.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.base.modules.transformer.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.sentence_transformer.modules.pooling.Pooling"
13
+ },
14
+ {
15
+ "idx": 2,
16
+ "name": "2",
17
+ "path": "2_Dense",
18
+ "type": "sentence_transformers.base.modules.dense.Dense"
19
+ },
20
+ {
21
+ "idx": 3,
22
+ "name": "3",
23
+ "path": "3_Normalize",
24
+ "type": "sentence_transformers.base.modules.normalize.Normalize"
25
+ }
26
+ ]
checkpoint-19496/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb09b6e752ace5a049d758bf2ce82f0a5e94c9c6bd47a1e9536eab835fb8daf4
3
+ size 3767584523
checkpoint-19496/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:04b0f9853d00e8940a64f75e141ae88d69d2ed3f7a052d04d7353f45fae2ac64
3
+ size 14645
checkpoint-19496/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b54815ced3359243f2f8e5ff19ee8610116d09f243ae28e62664a006b387e55f
3
+ size 1465
checkpoint-19496/sentence_bert_config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "transformer_task": "feature-extraction",
3
+ "modality_config": {
4
+ "text": {
5
+ "method": "forward",
6
+ "method_output_name": "last_hidden_state"
7
+ }
8
+ },
9
+ "module_output_name": "token_embeddings"
10
+ }
checkpoint-19496/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:92262b29204f8fdc169a63f9005a0e311a16262cef4d96ecfe2a7ed638662ed3
3
+ size 13632172
checkpoint-19496/tokenizer_config.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "cls_token": "[CLS]",
4
+ "do_basic_tokenize": true,
5
+ "do_lower_case": false,
6
+ "full_tokenizer_file": null,
7
+ "is_local": false,
8
+ "local_files_only": false,
9
+ "mask_token": "[MASK]",
10
+ "model_max_length": 256,
11
+ "never_split": null,
12
+ "pad_token": "[PAD]",
13
+ "sep_token": "[SEP]",
14
+ "strip_accents": null,
15
+ "tokenize_chinese_chars": true,
16
+ "tokenizer_class": "BertTokenizer",
17
+ "unk_token": "[UNK]"
18
+ }
checkpoint-19496/trainer_state.json ADDED
@@ -0,0 +1,2801 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_global_step": null,
3
+ "best_metric": null,
4
+ "best_model_checkpoint": null,
5
+ "epoch": 2.0,
6
+ "eval_steps": 500,
7
+ "global_step": 19496,
8
+ "is_hyper_param_search": false,
9
+ "is_local_process_zero": true,
10
+ "is_world_process_zero": true,
11
+ "log_history": [
12
+ {
13
+ "epoch": 0.0051292572835453425,
14
+ "grad_norm": 5.146663188934326,
15
+ "learning_rate": 5.025641025641026e-07,
16
+ "loss": 0.20988283157348633,
17
+ "step": 50
18
+ },
19
+ {
20
+ "epoch": 0.010258514567090685,
21
+ "grad_norm": 1.9344255924224854,
22
+ "learning_rate": 1.0153846153846155e-06,
23
+ "loss": 0.08006068229675294,
24
+ "step": 100
25
+ },
26
+ {
27
+ "epoch": 0.015387771850636028,
28
+ "grad_norm": 0.7599691152572632,
29
+ "learning_rate": 1.5282051282051283e-06,
30
+ "loss": 0.025034008026123048,
31
+ "step": 150
32
+ },
33
+ {
34
+ "epoch": 0.02051702913418137,
35
+ "grad_norm": 2.4555349349975586,
36
+ "learning_rate": 2.041025641025641e-06,
37
+ "loss": 0.019598073959350586,
38
+ "step": 200
39
+ },
40
+ {
41
+ "epoch": 0.02564628641772671,
42
+ "grad_norm": 0.22511054575443268,
43
+ "learning_rate": 2.5538461538461538e-06,
44
+ "loss": 0.01550939679145813,
45
+ "step": 250
46
+ },
47
+ {
48
+ "epoch": 0.030775543701272057,
49
+ "grad_norm": 1.7008031606674194,
50
+ "learning_rate": 3.066666666666667e-06,
51
+ "loss": 0.015118303298950196,
52
+ "step": 300
53
+ },
54
+ {
55
+ "epoch": 0.0359048009848174,
56
+ "grad_norm": 0.2420973777770996,
57
+ "learning_rate": 3.57948717948718e-06,
58
+ "loss": 0.0150359308719635,
59
+ "step": 350
60
+ },
61
+ {
62
+ "epoch": 0.04103405826836274,
63
+ "grad_norm": 0.07540615648031235,
64
+ "learning_rate": 4.0923076923076925e-06,
65
+ "loss": 0.011664544343948363,
66
+ "step": 400
67
+ },
68
+ {
69
+ "epoch": 0.046163315551908085,
70
+ "grad_norm": 0.33558762073516846,
71
+ "learning_rate": 4.605128205128205e-06,
72
+ "loss": 0.012408304214477538,
73
+ "step": 450
74
+ },
75
+ {
76
+ "epoch": 0.05129257283545342,
77
+ "grad_norm": 1.9506994485855103,
78
+ "learning_rate": 5.1179487179487186e-06,
79
+ "loss": 0.01108514666557312,
80
+ "step": 500
81
+ },
82
+ {
83
+ "epoch": 0.05642183011899877,
84
+ "grad_norm": 2.12082576751709,
85
+ "learning_rate": 5.63076923076923e-06,
86
+ "loss": 0.007002769708633423,
87
+ "step": 550
88
+ },
89
+ {
90
+ "epoch": 0.061551087402544113,
91
+ "grad_norm": 0.08876100182533264,
92
+ "learning_rate": 6.143589743589744e-06,
93
+ "loss": 0.010732929706573486,
94
+ "step": 600
95
+ },
96
+ {
97
+ "epoch": 0.06668034468608945,
98
+ "grad_norm": 0.06329763680696487,
99
+ "learning_rate": 6.656410256410257e-06,
100
+ "loss": 0.009123514890670776,
101
+ "step": 650
102
+ },
103
+ {
104
+ "epoch": 0.0718096019696348,
105
+ "grad_norm": 0.10696346312761307,
106
+ "learning_rate": 7.169230769230769e-06,
107
+ "loss": 0.006033726930618286,
108
+ "step": 700
109
+ },
110
+ {
111
+ "epoch": 0.07693885925318014,
112
+ "grad_norm": 0.18690240383148193,
113
+ "learning_rate": 7.682051282051283e-06,
114
+ "loss": 0.009448485970497132,
115
+ "step": 750
116
+ },
117
+ {
118
+ "epoch": 0.08206811653672548,
119
+ "grad_norm": 0.13596756756305695,
120
+ "learning_rate": 8.194871794871795e-06,
121
+ "loss": 0.006707255840301514,
122
+ "step": 800
123
+ },
124
+ {
125
+ "epoch": 0.08719737382027082,
126
+ "grad_norm": 0.32657232880592346,
127
+ "learning_rate": 8.707692307692309e-06,
128
+ "loss": 0.006077980995178223,
129
+ "step": 850
130
+ },
131
+ {
132
+ "epoch": 0.09232663110381617,
133
+ "grad_norm": 0.5776617527008057,
134
+ "learning_rate": 9.220512820512822e-06,
135
+ "loss": 0.0075683963298797605,
136
+ "step": 900
137
+ },
138
+ {
139
+ "epoch": 0.09745588838736151,
140
+ "grad_norm": 0.24212700128555298,
141
+ "learning_rate": 9.733333333333334e-06,
142
+ "loss": 0.005986063480377197,
143
+ "step": 950
144
+ },
145
+ {
146
+ "epoch": 0.10258514567090685,
147
+ "grad_norm": 0.5887935161590576,
148
+ "learning_rate": 1.0246153846153847e-05,
149
+ "loss": 0.004652740359306335,
150
+ "step": 1000
151
+ },
152
+ {
153
+ "epoch": 0.1077144029544522,
154
+ "grad_norm": 1.7308635711669922,
155
+ "learning_rate": 1.0758974358974359e-05,
156
+ "loss": 0.0070805585384368895,
157
+ "step": 1050
158
+ },
159
+ {
160
+ "epoch": 0.11284366023799754,
161
+ "grad_norm": 0.3150564134120941,
162
+ "learning_rate": 1.1271794871794873e-05,
163
+ "loss": 0.006270056962966919,
164
+ "step": 1100
165
+ },
166
+ {
167
+ "epoch": 0.11797291752154287,
168
+ "grad_norm": 0.1266297996044159,
169
+ "learning_rate": 1.1784615384615386e-05,
170
+ "loss": 0.004561504721641541,
171
+ "step": 1150
172
+ },
173
+ {
174
+ "epoch": 0.12310217480508823,
175
+ "grad_norm": 0.1587323397397995,
176
+ "learning_rate": 1.22974358974359e-05,
177
+ "loss": 0.003875747323036194,
178
+ "step": 1200
179
+ },
180
+ {
181
+ "epoch": 0.12823143208863358,
182
+ "grad_norm": 1.8177411556243896,
183
+ "learning_rate": 1.2810256410256411e-05,
184
+ "loss": 0.004878196120262146,
185
+ "step": 1250
186
+ },
187
+ {
188
+ "epoch": 0.1333606893721789,
189
+ "grad_norm": 0.07512801885604858,
190
+ "learning_rate": 1.3323076923076925e-05,
191
+ "loss": 0.005473270416259766,
192
+ "step": 1300
193
+ },
194
+ {
195
+ "epoch": 0.13848994665572426,
196
+ "grad_norm": 0.09478379040956497,
197
+ "learning_rate": 1.3835897435897437e-05,
198
+ "loss": 0.003836294412612915,
199
+ "step": 1350
200
+ },
201
+ {
202
+ "epoch": 0.1436192039392696,
203
+ "grad_norm": 0.20241230726242065,
204
+ "learning_rate": 1.434871794871795e-05,
205
+ "loss": 0.004140390455722809,
206
+ "step": 1400
207
+ },
208
+ {
209
+ "epoch": 0.14874846122281493,
210
+ "grad_norm": 0.28742000460624695,
211
+ "learning_rate": 1.4861538461538464e-05,
212
+ "loss": 0.005977784991264343,
213
+ "step": 1450
214
+ },
215
+ {
216
+ "epoch": 0.15387771850636028,
217
+ "grad_norm": 1.9182926416397095,
218
+ "learning_rate": 1.5374358974358975e-05,
219
+ "loss": 0.00550079345703125,
220
+ "step": 1500
221
+ },
222
+ {
223
+ "epoch": 0.1590069757899056,
224
+ "grad_norm": 0.23799288272857666,
225
+ "learning_rate": 1.588717948717949e-05,
226
+ "loss": 0.00528115451335907,
227
+ "step": 1550
228
+ },
229
+ {
230
+ "epoch": 0.16413623307345096,
231
+ "grad_norm": 0.18093019723892212,
232
+ "learning_rate": 1.64e-05,
233
+ "loss": 0.004683617353439331,
234
+ "step": 1600
235
+ },
236
+ {
237
+ "epoch": 0.1692654903569963,
238
+ "grad_norm": 0.6764870882034302,
239
+ "learning_rate": 1.6912820512820512e-05,
240
+ "loss": 0.004430403709411621,
241
+ "step": 1650
242
+ },
243
+ {
244
+ "epoch": 0.17439474764054164,
245
+ "grad_norm": 0.030442317947745323,
246
+ "learning_rate": 1.7425641025641026e-05,
247
+ "loss": 0.007149366140365601,
248
+ "step": 1700
249
+ },
250
+ {
251
+ "epoch": 0.179524004924087,
252
+ "grad_norm": 0.12910808622837067,
253
+ "learning_rate": 1.793846153846154e-05,
254
+ "loss": 0.004426850080490112,
255
+ "step": 1750
256
+ },
257
+ {
258
+ "epoch": 0.18465326220763234,
259
+ "grad_norm": 0.03711940720677376,
260
+ "learning_rate": 1.8451282051282053e-05,
261
+ "loss": 0.002525296211242676,
262
+ "step": 1800
263
+ },
264
+ {
265
+ "epoch": 0.18978251949117766,
266
+ "grad_norm": 0.209708109498024,
267
+ "learning_rate": 1.8964102564102566e-05,
268
+ "loss": 0.005534772872924805,
269
+ "step": 1850
270
+ },
271
+ {
272
+ "epoch": 0.19491177677472302,
273
+ "grad_norm": 0.07900407165288925,
274
+ "learning_rate": 1.9476923076923076e-05,
275
+ "loss": 0.00532868504524231,
276
+ "step": 1900
277
+ },
278
+ {
279
+ "epoch": 0.20004103405826837,
280
+ "grad_norm": 0.04088887572288513,
281
+ "learning_rate": 1.998974358974359e-05,
282
+ "loss": 0.004520394206047058,
283
+ "step": 1950
284
+ },
285
+ {
286
+ "epoch": 0.2051702913418137,
287
+ "grad_norm": 0.044649723917245865,
288
+ "learning_rate": 1.9944146814088682e-05,
289
+ "loss": 0.004650795459747314,
290
+ "step": 2000
291
+ },
292
+ {
293
+ "epoch": 0.21029954862535905,
294
+ "grad_norm": 0.36089038848876953,
295
+ "learning_rate": 1.98871537672404e-05,
296
+ "loss": 0.004750135838985443,
297
+ "step": 2050
298
+ },
299
+ {
300
+ "epoch": 0.2154288059089044,
301
+ "grad_norm": 0.2627781331539154,
302
+ "learning_rate": 1.9830160720392112e-05,
303
+ "loss": 0.004101980328559875,
304
+ "step": 2100
305
+ },
306
+ {
307
+ "epoch": 0.22055806319244972,
308
+ "grad_norm": 0.0719594806432724,
309
+ "learning_rate": 1.977316767354383e-05,
310
+ "loss": 0.002980492115020752,
311
+ "step": 2150
312
+ },
313
+ {
314
+ "epoch": 0.22568732047599507,
315
+ "grad_norm": 0.027993543073534966,
316
+ "learning_rate": 1.9716174626695546e-05,
317
+ "loss": 0.0036524400115013123,
318
+ "step": 2200
319
+ },
320
+ {
321
+ "epoch": 0.23081657775954043,
322
+ "grad_norm": 0.020856564864516258,
323
+ "learning_rate": 1.965918157984726e-05,
324
+ "loss": 0.004709515571594238,
325
+ "step": 2250
326
+ },
327
+ {
328
+ "epoch": 0.23594583504308575,
329
+ "grad_norm": 0.7141599655151367,
330
+ "learning_rate": 1.9602188532998976e-05,
331
+ "loss": 0.004219242334365845,
332
+ "step": 2300
333
+ },
334
+ {
335
+ "epoch": 0.2410750923266311,
336
+ "grad_norm": 0.6264314651489258,
337
+ "learning_rate": 1.954519548615069e-05,
338
+ "loss": 0.0027445003390312193,
339
+ "step": 2350
340
+ },
341
+ {
342
+ "epoch": 0.24620434961017645,
343
+ "grad_norm": 0.0843123346567154,
344
+ "learning_rate": 1.9488202439302406e-05,
345
+ "loss": 0.004736282825469971,
346
+ "step": 2400
347
+ },
348
+ {
349
+ "epoch": 0.2513336068937218,
350
+ "grad_norm": 0.03579830378293991,
351
+ "learning_rate": 1.943120939245412e-05,
352
+ "loss": 0.002808626592159271,
353
+ "step": 2450
354
+ },
355
+ {
356
+ "epoch": 0.25646286417726716,
357
+ "grad_norm": 1.2769086360931396,
358
+ "learning_rate": 1.937421634560584e-05,
359
+ "loss": 0.0043040883541107175,
360
+ "step": 2500
361
+ },
362
+ {
363
+ "epoch": 0.26159212146081245,
364
+ "grad_norm": 0.6006304621696472,
365
+ "learning_rate": 1.931722329875755e-05,
366
+ "loss": 0.00633010983467102,
367
+ "step": 2550
368
+ },
369
+ {
370
+ "epoch": 0.2667213787443578,
371
+ "grad_norm": 0.4219711422920227,
372
+ "learning_rate": 1.926023025190927e-05,
373
+ "loss": 0.002694355249404907,
374
+ "step": 2600
375
+ },
376
+ {
377
+ "epoch": 0.27185063602790316,
378
+ "grad_norm": 0.03629787266254425,
379
+ "learning_rate": 1.9203237205060985e-05,
380
+ "loss": 0.0020533058047294615,
381
+ "step": 2650
382
+ },
383
+ {
384
+ "epoch": 0.2769798933114485,
385
+ "grad_norm": 0.5271649360656738,
386
+ "learning_rate": 1.91462441582127e-05,
387
+ "loss": 0.002846062183380127,
388
+ "step": 2700
389
+ },
390
+ {
391
+ "epoch": 0.28210915059499386,
392
+ "grad_norm": 0.038686707615852356,
393
+ "learning_rate": 1.9089251111364415e-05,
394
+ "loss": 0.002439509928226471,
395
+ "step": 2750
396
+ },
397
+ {
398
+ "epoch": 0.2872384078785392,
399
+ "grad_norm": 0.0804801732301712,
400
+ "learning_rate": 1.903225806451613e-05,
401
+ "loss": 0.00447674959897995,
402
+ "step": 2800
403
+ },
404
+ {
405
+ "epoch": 0.2923676651620845,
406
+ "grad_norm": 0.11198680847883224,
407
+ "learning_rate": 1.8975265017667846e-05,
408
+ "loss": 0.0033469206094741823,
409
+ "step": 2850
410
+ },
411
+ {
412
+ "epoch": 0.29749692244562986,
413
+ "grad_norm": 0.2559853196144104,
414
+ "learning_rate": 1.891827197081956e-05,
415
+ "loss": 0.004776168465614319,
416
+ "step": 2900
417
+ },
418
+ {
419
+ "epoch": 0.3026261797291752,
420
+ "grad_norm": 0.9892205595970154,
421
+ "learning_rate": 1.886127892397128e-05,
422
+ "loss": 0.0021404142677783965,
423
+ "step": 2950
424
+ },
425
+ {
426
+ "epoch": 0.30775543701272057,
427
+ "grad_norm": 0.028234116733074188,
428
+ "learning_rate": 1.880428587712299e-05,
429
+ "loss": 0.0031156307458877564,
430
+ "step": 3000
431
+ },
432
+ {
433
+ "epoch": 0.3128846942962659,
434
+ "grad_norm": 0.08867978304624557,
435
+ "learning_rate": 1.874729283027471e-05,
436
+ "loss": 0.002720406651496887,
437
+ "step": 3050
438
+ },
439
+ {
440
+ "epoch": 0.3180139515798112,
441
+ "grad_norm": 0.4465535283088684,
442
+ "learning_rate": 1.8690299783426425e-05,
443
+ "loss": 0.0033028775453567503,
444
+ "step": 3100
445
+ },
446
+ {
447
+ "epoch": 0.32314320886335657,
448
+ "grad_norm": 0.11233843117952347,
449
+ "learning_rate": 1.863330673657814e-05,
450
+ "loss": 0.004013978838920593,
451
+ "step": 3150
452
+ },
453
+ {
454
+ "epoch": 0.3282724661469019,
455
+ "grad_norm": 0.06376665830612183,
456
+ "learning_rate": 1.8576313689729855e-05,
457
+ "loss": 0.0012492486834526063,
458
+ "step": 3200
459
+ },
460
+ {
461
+ "epoch": 0.33340172343044727,
462
+ "grad_norm": 0.03352602571249008,
463
+ "learning_rate": 1.851932064288157e-05,
464
+ "loss": 0.0030529314279556276,
465
+ "step": 3250
466
+ },
467
+ {
468
+ "epoch": 0.3385309807139926,
469
+ "grad_norm": 1.316901683807373,
470
+ "learning_rate": 1.8462327596033285e-05,
471
+ "loss": 0.003336084485054016,
472
+ "step": 3300
473
+ },
474
+ {
475
+ "epoch": 0.343660237997538,
476
+ "grad_norm": 0.5216875076293945,
477
+ "learning_rate": 1.8405334549185e-05,
478
+ "loss": 0.0019701987504959105,
479
+ "step": 3350
480
+ },
481
+ {
482
+ "epoch": 0.3487894952810833,
483
+ "grad_norm": 0.008256763219833374,
484
+ "learning_rate": 1.834834150233672e-05,
485
+ "loss": 0.002782421410083771,
486
+ "step": 3400
487
+ },
488
+ {
489
+ "epoch": 0.3539187525646286,
490
+ "grad_norm": 0.14605538547039032,
491
+ "learning_rate": 1.829134845548843e-05,
492
+ "loss": 0.0022422271966934205,
493
+ "step": 3450
494
+ },
495
+ {
496
+ "epoch": 0.359048009848174,
497
+ "grad_norm": 0.05085963010787964,
498
+ "learning_rate": 1.823435540864015e-05,
499
+ "loss": 0.002529405951499939,
500
+ "step": 3500
501
+ },
502
+ {
503
+ "epoch": 0.36417726713171933,
504
+ "grad_norm": 0.0359930619597435,
505
+ "learning_rate": 1.8177362361791864e-05,
506
+ "loss": 0.0026357889175415037,
507
+ "step": 3550
508
+ },
509
+ {
510
+ "epoch": 0.3693065244152647,
511
+ "grad_norm": 0.23114101588726044,
512
+ "learning_rate": 1.812036931494358e-05,
513
+ "loss": 0.0027996709942817687,
514
+ "step": 3600
515
+ },
516
+ {
517
+ "epoch": 0.37443578169881003,
518
+ "grad_norm": 0.021300168707966805,
519
+ "learning_rate": 1.8063376268095294e-05,
520
+ "loss": 0.0034693616628646852,
521
+ "step": 3650
522
+ },
523
+ {
524
+ "epoch": 0.37956503898235533,
525
+ "grad_norm": 0.11509425938129425,
526
+ "learning_rate": 1.800638322124701e-05,
527
+ "loss": 0.0023235560953617095,
528
+ "step": 3700
529
+ },
530
+ {
531
+ "epoch": 0.3846942962659007,
532
+ "grad_norm": 0.6359381079673767,
533
+ "learning_rate": 1.7949390174398724e-05,
534
+ "loss": 0.002906927764415741,
535
+ "step": 3750
536
+ },
537
+ {
538
+ "epoch": 0.38982355354944603,
539
+ "grad_norm": 0.030257439240813255,
540
+ "learning_rate": 1.789239712755044e-05,
541
+ "loss": 0.0030827236175537107,
542
+ "step": 3800
543
+ },
544
+ {
545
+ "epoch": 0.3949528108329914,
546
+ "grad_norm": 0.19199493527412415,
547
+ "learning_rate": 1.7835404080702158e-05,
548
+ "loss": 0.0042154985666275025,
549
+ "step": 3850
550
+ },
551
+ {
552
+ "epoch": 0.40008206811653674,
553
+ "grad_norm": 0.01845471002161503,
554
+ "learning_rate": 1.777841103385387e-05,
555
+ "loss": 0.002743417024612427,
556
+ "step": 3900
557
+ },
558
+ {
559
+ "epoch": 0.4052113254000821,
560
+ "grad_norm": 0.019544444978237152,
561
+ "learning_rate": 1.7721417987005588e-05,
562
+ "loss": 0.003500168323516846,
563
+ "step": 3950
564
+ },
565
+ {
566
+ "epoch": 0.4103405826836274,
567
+ "grad_norm": 1.0970685482025146,
568
+ "learning_rate": 1.7664424940157303e-05,
569
+ "loss": 0.0025902286171913145,
570
+ "step": 4000
571
+ },
572
+ {
573
+ "epoch": 0.41546983996717274,
574
+ "grad_norm": 0.0196465365588665,
575
+ "learning_rate": 1.760743189330902e-05,
576
+ "loss": 0.0017264434695243836,
577
+ "step": 4050
578
+ },
579
+ {
580
+ "epoch": 0.4205990972507181,
581
+ "grad_norm": 0.1122385635972023,
582
+ "learning_rate": 1.7550438846460734e-05,
583
+ "loss": 0.00217318132519722,
584
+ "step": 4100
585
+ },
586
+ {
587
+ "epoch": 0.42572835453426344,
588
+ "grad_norm": 0.24590978026390076,
589
+ "learning_rate": 1.749344579961245e-05,
590
+ "loss": 0.001975882798433304,
591
+ "step": 4150
592
+ },
593
+ {
594
+ "epoch": 0.4308576118178088,
595
+ "grad_norm": 0.033505771309137344,
596
+ "learning_rate": 1.7436452752764164e-05,
597
+ "loss": 0.002472508400678635,
598
+ "step": 4200
599
+ },
600
+ {
601
+ "epoch": 0.43598686910135415,
602
+ "grad_norm": 0.11213713139295578,
603
+ "learning_rate": 1.737945970591588e-05,
604
+ "loss": 0.002362338155508041,
605
+ "step": 4250
606
+ },
607
+ {
608
+ "epoch": 0.44111612638489944,
609
+ "grad_norm": 0.0998101606965065,
610
+ "learning_rate": 1.7322466659067597e-05,
611
+ "loss": 0.0025117507576942442,
612
+ "step": 4300
613
+ },
614
+ {
615
+ "epoch": 0.4462453836684448,
616
+ "grad_norm": 0.10846878588199615,
617
+ "learning_rate": 1.726547361221931e-05,
618
+ "loss": 0.002475926429033279,
619
+ "step": 4350
620
+ },
621
+ {
622
+ "epoch": 0.45137464095199015,
623
+ "grad_norm": 0.30236443877220154,
624
+ "learning_rate": 1.7208480565371028e-05,
625
+ "loss": 0.002092706859111786,
626
+ "step": 4400
627
+ },
628
+ {
629
+ "epoch": 0.4565038982355355,
630
+ "grad_norm": 0.020106056705117226,
631
+ "learning_rate": 1.7151487518522743e-05,
632
+ "loss": 0.0010956238210201264,
633
+ "step": 4450
634
+ },
635
+ {
636
+ "epoch": 0.46163315551908085,
637
+ "grad_norm": 0.06396861374378204,
638
+ "learning_rate": 1.7094494471674458e-05,
639
+ "loss": 0.0018238377571105956,
640
+ "step": 4500
641
+ },
642
+ {
643
+ "epoch": 0.4667624128026262,
644
+ "grad_norm": 0.4166419506072998,
645
+ "learning_rate": 1.7037501424826173e-05,
646
+ "loss": 0.003854023814201355,
647
+ "step": 4550
648
+ },
649
+ {
650
+ "epoch": 0.4718916700861715,
651
+ "grad_norm": 0.03014219179749489,
652
+ "learning_rate": 1.6980508377977888e-05,
653
+ "loss": 0.0016187562048435211,
654
+ "step": 4600
655
+ },
656
+ {
657
+ "epoch": 0.47702092736971685,
658
+ "grad_norm": 0.25602439045906067,
659
+ "learning_rate": 1.6923515331129603e-05,
660
+ "loss": 0.002320457398891449,
661
+ "step": 4650
662
+ },
663
+ {
664
+ "epoch": 0.4821501846532622,
665
+ "grad_norm": 0.02201703004539013,
666
+ "learning_rate": 1.686652228428132e-05,
667
+ "loss": 0.0012020480632781983,
668
+ "step": 4700
669
+ },
670
+ {
671
+ "epoch": 0.48727944193680756,
672
+ "grad_norm": 0.05641191452741623,
673
+ "learning_rate": 1.6809529237433037e-05,
674
+ "loss": 0.002663995623588562,
675
+ "step": 4750
676
+ },
677
+ {
678
+ "epoch": 0.4924086992203529,
679
+ "grad_norm": 0.06632937490940094,
680
+ "learning_rate": 1.675253619058475e-05,
681
+ "loss": 0.0020142243802547454,
682
+ "step": 4800
683
+ },
684
+ {
685
+ "epoch": 0.49753795650389826,
686
+ "grad_norm": 0.09227260202169418,
687
+ "learning_rate": 1.6695543143736467e-05,
688
+ "loss": 0.002214210033416748,
689
+ "step": 4850
690
+ },
691
+ {
692
+ "epoch": 0.5026672137874436,
693
+ "grad_norm": 0.04919379949569702,
694
+ "learning_rate": 1.6638550096888182e-05,
695
+ "loss": 0.0022323706746101377,
696
+ "step": 4900
697
+ },
698
+ {
699
+ "epoch": 0.5077964710709889,
700
+ "grad_norm": 0.05477536842226982,
701
+ "learning_rate": 1.6581557050039897e-05,
702
+ "loss": 0.0015611539781093598,
703
+ "step": 4950
704
+ },
705
+ {
706
+ "epoch": 0.5129257283545343,
707
+ "grad_norm": 0.07904011756181717,
708
+ "learning_rate": 1.6524564003191613e-05,
709
+ "loss": 0.0026751086115837097,
710
+ "step": 5000
711
+ },
712
+ {
713
+ "epoch": 0.5180549856380796,
714
+ "grad_norm": 0.03175102546811104,
715
+ "learning_rate": 1.6467570956343328e-05,
716
+ "loss": 0.0017083890736103058,
717
+ "step": 5050
718
+ },
719
+ {
720
+ "epoch": 0.5231842429216249,
721
+ "grad_norm": 0.3278403878211975,
722
+ "learning_rate": 1.6410577909495043e-05,
723
+ "loss": 0.00195760115981102,
724
+ "step": 5100
725
+ },
726
+ {
727
+ "epoch": 0.5283135002051703,
728
+ "grad_norm": 0.21054060757160187,
729
+ "learning_rate": 1.6353584862646758e-05,
730
+ "loss": 0.002422951906919479,
731
+ "step": 5150
732
+ },
733
+ {
734
+ "epoch": 0.5334427574887156,
735
+ "grad_norm": 0.13860668241977692,
736
+ "learning_rate": 1.6296591815798476e-05,
737
+ "loss": 0.002446786016225815,
738
+ "step": 5200
739
+ },
740
+ {
741
+ "epoch": 0.538572014772261,
742
+ "grad_norm": 0.3238920569419861,
743
+ "learning_rate": 1.6239598768950188e-05,
744
+ "loss": 0.0024388924241065977,
745
+ "step": 5250
746
+ },
747
+ {
748
+ "epoch": 0.5437012720558063,
749
+ "grad_norm": 0.009936186484992504,
750
+ "learning_rate": 1.6182605722101907e-05,
751
+ "loss": 0.002167973667383194,
752
+ "step": 5300
753
+ },
754
+ {
755
+ "epoch": 0.5488305293393516,
756
+ "grad_norm": 0.10222964733839035,
757
+ "learning_rate": 1.6125612675253622e-05,
758
+ "loss": 0.002880829870700836,
759
+ "step": 5350
760
+ },
761
+ {
762
+ "epoch": 0.553959786622897,
763
+ "grad_norm": 0.006579564418643713,
764
+ "learning_rate": 1.6068619628405337e-05,
765
+ "loss": 0.0015730999410152435,
766
+ "step": 5400
767
+ },
768
+ {
769
+ "epoch": 0.5590890439064423,
770
+ "grad_norm": 0.013894481584429741,
771
+ "learning_rate": 1.6011626581557052e-05,
772
+ "loss": 0.0010276009142398835,
773
+ "step": 5450
774
+ },
775
+ {
776
+ "epoch": 0.5642183011899877,
777
+ "grad_norm": 0.1254359483718872,
778
+ "learning_rate": 1.5954633534708767e-05,
779
+ "loss": 0.0017169848084449767,
780
+ "step": 5500
781
+ },
782
+ {
783
+ "epoch": 0.569347558473533,
784
+ "grad_norm": 0.12541364133358002,
785
+ "learning_rate": 1.5897640487860482e-05,
786
+ "loss": 0.0028713226318359377,
787
+ "step": 5550
788
+ },
789
+ {
790
+ "epoch": 0.5744768157570784,
791
+ "grad_norm": 0.08445322513580322,
792
+ "learning_rate": 1.5840647441012197e-05,
793
+ "loss": 0.0016673532128334044,
794
+ "step": 5600
795
+ },
796
+ {
797
+ "epoch": 0.5796060730406237,
798
+ "grad_norm": 0.04391612112522125,
799
+ "learning_rate": 1.5783654394163916e-05,
800
+ "loss": 0.002128530740737915,
801
+ "step": 5650
802
+ },
803
+ {
804
+ "epoch": 0.584735330324169,
805
+ "grad_norm": 0.5561177134513855,
806
+ "learning_rate": 1.5726661347315628e-05,
807
+ "loss": 0.0019441001117229463,
808
+ "step": 5700
809
+ },
810
+ {
811
+ "epoch": 0.5898645876077144,
812
+ "grad_norm": 0.09118201583623886,
813
+ "learning_rate": 1.5669668300467346e-05,
814
+ "loss": 0.0021015436947345732,
815
+ "step": 5750
816
+ },
817
+ {
818
+ "epoch": 0.5949938448912597,
819
+ "grad_norm": 0.022824030369520187,
820
+ "learning_rate": 1.561267525361906e-05,
821
+ "loss": 0.002961210608482361,
822
+ "step": 5800
823
+ },
824
+ {
825
+ "epoch": 0.6001231021748051,
826
+ "grad_norm": 0.04558834061026573,
827
+ "learning_rate": 1.5555682206770776e-05,
828
+ "loss": 0.001471487581729889,
829
+ "step": 5850
830
+ },
831
+ {
832
+ "epoch": 0.6052523594583504,
833
+ "grad_norm": 0.031143372878432274,
834
+ "learning_rate": 1.549868915992249e-05,
835
+ "loss": 0.0027905339002609253,
836
+ "step": 5900
837
+ },
838
+ {
839
+ "epoch": 0.6103816167418957,
840
+ "grad_norm": 0.2602635324001312,
841
+ "learning_rate": 1.5441696113074207e-05,
842
+ "loss": 0.002177398204803467,
843
+ "step": 5950
844
+ },
845
+ {
846
+ "epoch": 0.6155108740254411,
847
+ "grad_norm": 0.008853335864841938,
848
+ "learning_rate": 1.538470306622592e-05,
849
+ "loss": 0.0020481890439987185,
850
+ "step": 6000
851
+ },
852
+ {
853
+ "epoch": 0.6206401313089864,
854
+ "grad_norm": 0.024634389206767082,
855
+ "learning_rate": 1.5327710019377637e-05,
856
+ "loss": 0.002121390849351883,
857
+ "step": 6050
858
+ },
859
+ {
860
+ "epoch": 0.6257693885925318,
861
+ "grad_norm": 0.009994241409003735,
862
+ "learning_rate": 1.5270716972529355e-05,
863
+ "loss": 0.001738632023334503,
864
+ "step": 6100
865
+ },
866
+ {
867
+ "epoch": 0.6308986458760771,
868
+ "grad_norm": 0.16715426743030548,
869
+ "learning_rate": 1.5213723925681067e-05,
870
+ "loss": 0.002144707888364792,
871
+ "step": 6150
872
+ },
873
+ {
874
+ "epoch": 0.6360279031596224,
875
+ "grad_norm": 0.05593431368470192,
876
+ "learning_rate": 1.5156730878832784e-05,
877
+ "loss": 0.0014984746277332305,
878
+ "step": 6200
879
+ },
880
+ {
881
+ "epoch": 0.6411571604431678,
882
+ "grad_norm": 0.030156264081597328,
883
+ "learning_rate": 1.50997378319845e-05,
884
+ "loss": 0.0010228657722473145,
885
+ "step": 6250
886
+ },
887
+ {
888
+ "epoch": 0.6462864177267131,
889
+ "grad_norm": 0.045500464737415314,
890
+ "learning_rate": 1.5042744785136214e-05,
891
+ "loss": 0.0017821905016899109,
892
+ "step": 6300
893
+ },
894
+ {
895
+ "epoch": 0.6514156750102585,
896
+ "grad_norm": 0.0190139077603817,
897
+ "learning_rate": 1.4985751738287931e-05,
898
+ "loss": 0.0011478330194950104,
899
+ "step": 6350
900
+ },
901
+ {
902
+ "epoch": 0.6565449322938038,
903
+ "grad_norm": 0.022497054189443588,
904
+ "learning_rate": 1.4928758691439646e-05,
905
+ "loss": 0.003517746925354004,
906
+ "step": 6400
907
+ },
908
+ {
909
+ "epoch": 0.6616741895773492,
910
+ "grad_norm": 0.03552277013659477,
911
+ "learning_rate": 1.4871765644591361e-05,
912
+ "loss": 0.0014983375370502471,
913
+ "step": 6450
914
+ },
915
+ {
916
+ "epoch": 0.6668034468608945,
917
+ "grad_norm": 0.012892907485365868,
918
+ "learning_rate": 1.4814772597743076e-05,
919
+ "loss": 0.0021820993721485137,
920
+ "step": 6500
921
+ },
922
+ {
923
+ "epoch": 0.6719327041444398,
924
+ "grad_norm": 0.0045395540073513985,
925
+ "learning_rate": 1.4757779550894793e-05,
926
+ "loss": 0.001870407611131668,
927
+ "step": 6550
928
+ },
929
+ {
930
+ "epoch": 0.6770619614279852,
931
+ "grad_norm": 0.054160960018634796,
932
+ "learning_rate": 1.4700786504046506e-05,
933
+ "loss": 0.0016691868007183074,
934
+ "step": 6600
935
+ },
936
+ {
937
+ "epoch": 0.6821912187115305,
938
+ "grad_norm": 0.021756375208497047,
939
+ "learning_rate": 1.4643793457198223e-05,
940
+ "loss": 0.0011199010908603667,
941
+ "step": 6650
942
+ },
943
+ {
944
+ "epoch": 0.687320475995076,
945
+ "grad_norm": 0.24180440604686737,
946
+ "learning_rate": 1.458680041034994e-05,
947
+ "loss": 0.002085486799478531,
948
+ "step": 6700
949
+ },
950
+ {
951
+ "epoch": 0.6924497332786212,
952
+ "grad_norm": 0.02364238165318966,
953
+ "learning_rate": 1.4529807363501653e-05,
954
+ "loss": 0.001655416041612625,
955
+ "step": 6750
956
+ },
957
+ {
958
+ "epoch": 0.6975789905621665,
959
+ "grad_norm": 0.013433502987027168,
960
+ "learning_rate": 1.447281431665337e-05,
961
+ "loss": 0.0014493338763713836,
962
+ "step": 6800
963
+ },
964
+ {
965
+ "epoch": 0.702708247845712,
966
+ "grad_norm": 0.09692008048295975,
967
+ "learning_rate": 1.4415821269805085e-05,
968
+ "loss": 0.002443571090698242,
969
+ "step": 6850
970
+ },
971
+ {
972
+ "epoch": 0.7078375051292572,
973
+ "grad_norm": 0.011011690832674503,
974
+ "learning_rate": 1.43588282229568e-05,
975
+ "loss": 0.0011622200906276703,
976
+ "step": 6900
977
+ },
978
+ {
979
+ "epoch": 0.7129667624128027,
980
+ "grad_norm": 0.03306994214653969,
981
+ "learning_rate": 1.4301835176108516e-05,
982
+ "loss": 0.003230692148208618,
983
+ "step": 6950
984
+ },
985
+ {
986
+ "epoch": 0.718096019696348,
987
+ "grad_norm": 0.08337920904159546,
988
+ "learning_rate": 1.4244842129260232e-05,
989
+ "loss": 0.001955211907625198,
990
+ "step": 7000
991
+ },
992
+ {
993
+ "epoch": 0.7232252769798934,
994
+ "grad_norm": 0.08515248447656631,
995
+ "learning_rate": 1.4187849082411946e-05,
996
+ "loss": 0.001256055235862732,
997
+ "step": 7050
998
+ },
999
+ {
1000
+ "epoch": 0.7283545342634387,
1001
+ "grad_norm": 0.11170300841331482,
1002
+ "learning_rate": 1.4130856035563663e-05,
1003
+ "loss": 0.0038022392988204955,
1004
+ "step": 7100
1005
+ },
1006
+ {
1007
+ "epoch": 0.733483791546984,
1008
+ "grad_norm": 0.005572218913584948,
1009
+ "learning_rate": 1.407386298871538e-05,
1010
+ "loss": 0.0017911630868911743,
1011
+ "step": 7150
1012
+ },
1013
+ {
1014
+ "epoch": 0.7386130488305294,
1015
+ "grad_norm": 0.008966299705207348,
1016
+ "learning_rate": 1.4016869941867093e-05,
1017
+ "loss": 0.0018778586387634278,
1018
+ "step": 7200
1019
+ },
1020
+ {
1021
+ "epoch": 0.7437423061140747,
1022
+ "grad_norm": 0.21079233288764954,
1023
+ "learning_rate": 1.395987689501881e-05,
1024
+ "loss": 0.0037654682993888853,
1025
+ "step": 7250
1026
+ },
1027
+ {
1028
+ "epoch": 0.7488715633976201,
1029
+ "grad_norm": 0.0403057225048542,
1030
+ "learning_rate": 1.3902883848170525e-05,
1031
+ "loss": 0.0012686464190483093,
1032
+ "step": 7300
1033
+ },
1034
+ {
1035
+ "epoch": 0.7540008206811654,
1036
+ "grad_norm": 0.09920848160982132,
1037
+ "learning_rate": 1.384589080132224e-05,
1038
+ "loss": 0.002512713074684143,
1039
+ "step": 7350
1040
+ },
1041
+ {
1042
+ "epoch": 0.7591300779647107,
1043
+ "grad_norm": 0.03594447299838066,
1044
+ "learning_rate": 1.3788897754473955e-05,
1045
+ "loss": 0.002175413519144058,
1046
+ "step": 7400
1047
+ },
1048
+ {
1049
+ "epoch": 0.7642593352482561,
1050
+ "grad_norm": 0.012109672650694847,
1051
+ "learning_rate": 1.3731904707625672e-05,
1052
+ "loss": 0.0014021342992782593,
1053
+ "step": 7450
1054
+ },
1055
+ {
1056
+ "epoch": 0.7693885925318014,
1057
+ "grad_norm": 0.055439919233322144,
1058
+ "learning_rate": 1.3674911660777385e-05,
1059
+ "loss": 0.0014539873600006104,
1060
+ "step": 7500
1061
+ },
1062
+ {
1063
+ "epoch": 0.7745178498153468,
1064
+ "grad_norm": 0.03262806683778763,
1065
+ "learning_rate": 1.3617918613929102e-05,
1066
+ "loss": 0.002966693043708801,
1067
+ "step": 7550
1068
+ },
1069
+ {
1070
+ "epoch": 0.7796471070988921,
1071
+ "grad_norm": 0.6407725811004639,
1072
+ "learning_rate": 1.3560925567080819e-05,
1073
+ "loss": 0.0016460046172142029,
1074
+ "step": 7600
1075
+ },
1076
+ {
1077
+ "epoch": 0.7847763643824375,
1078
+ "grad_norm": 0.07301273196935654,
1079
+ "learning_rate": 1.3503932520232532e-05,
1080
+ "loss": 0.001733643114566803,
1081
+ "step": 7650
1082
+ },
1083
+ {
1084
+ "epoch": 0.7899056216659828,
1085
+ "grad_norm": 0.04729823023080826,
1086
+ "learning_rate": 1.344693947338425e-05,
1087
+ "loss": 0.0012972503900527954,
1088
+ "step": 7700
1089
+ },
1090
+ {
1091
+ "epoch": 0.7950348789495281,
1092
+ "grad_norm": 0.055704180151224136,
1093
+ "learning_rate": 1.3389946426535964e-05,
1094
+ "loss": 0.0036036667227745058,
1095
+ "step": 7750
1096
+ },
1097
+ {
1098
+ "epoch": 0.8001641362330735,
1099
+ "grad_norm": 0.04170205816626549,
1100
+ "learning_rate": 1.333295337968768e-05,
1101
+ "loss": 0.002159286290407181,
1102
+ "step": 7800
1103
+ },
1104
+ {
1105
+ "epoch": 0.8052933935166188,
1106
+ "grad_norm": 0.03427044302225113,
1107
+ "learning_rate": 1.3275960332839395e-05,
1108
+ "loss": 0.001466388702392578,
1109
+ "step": 7850
1110
+ },
1111
+ {
1112
+ "epoch": 0.8104226508001642,
1113
+ "grad_norm": 0.1446782499551773,
1114
+ "learning_rate": 1.3218967285991111e-05,
1115
+ "loss": 0.0009994275867938995,
1116
+ "step": 7900
1117
+ },
1118
+ {
1119
+ "epoch": 0.8155519080837095,
1120
+ "grad_norm": 1.960245132446289,
1121
+ "learning_rate": 1.3161974239142825e-05,
1122
+ "loss": 0.002109799236059189,
1123
+ "step": 7950
1124
+ },
1125
+ {
1126
+ "epoch": 0.8206811653672548,
1127
+ "grad_norm": 0.011322901584208012,
1128
+ "learning_rate": 1.3104981192294542e-05,
1129
+ "loss": 0.0019486966729164125,
1130
+ "step": 8000
1131
+ },
1132
+ {
1133
+ "epoch": 0.8258104226508002,
1134
+ "grad_norm": 0.02532426454126835,
1135
+ "learning_rate": 1.3047988145446258e-05,
1136
+ "loss": 0.0010834269225597382,
1137
+ "step": 8050
1138
+ },
1139
+ {
1140
+ "epoch": 0.8309396799343455,
1141
+ "grad_norm": 0.006674936041235924,
1142
+ "learning_rate": 1.2990995098597972e-05,
1143
+ "loss": 0.0012358775734901428,
1144
+ "step": 8100
1145
+ },
1146
+ {
1147
+ "epoch": 0.8360689372178909,
1148
+ "grad_norm": 0.004865461494773626,
1149
+ "learning_rate": 1.2934002051749689e-05,
1150
+ "loss": 0.0026021018624305726,
1151
+ "step": 8150
1152
+ },
1153
+ {
1154
+ "epoch": 0.8411981945014362,
1155
+ "grad_norm": 0.03563657030463219,
1156
+ "learning_rate": 1.2877009004901404e-05,
1157
+ "loss": 0.0010813122987747192,
1158
+ "step": 8200
1159
+ },
1160
+ {
1161
+ "epoch": 0.8463274517849816,
1162
+ "grad_norm": 0.1195644959807396,
1163
+ "learning_rate": 1.2820015958053119e-05,
1164
+ "loss": 0.0018597865104675293,
1165
+ "step": 8250
1166
+ },
1167
+ {
1168
+ "epoch": 0.8514567090685269,
1169
+ "grad_norm": 0.01012786477804184,
1170
+ "learning_rate": 1.2763022911204834e-05,
1171
+ "loss": 0.001704736053943634,
1172
+ "step": 8300
1173
+ },
1174
+ {
1175
+ "epoch": 0.8565859663520722,
1176
+ "grad_norm": 0.02338203229010105,
1177
+ "learning_rate": 1.270602986435655e-05,
1178
+ "loss": 0.0011339907348155976,
1179
+ "step": 8350
1180
+ },
1181
+ {
1182
+ "epoch": 0.8617152236356176,
1183
+ "grad_norm": 0.01370213832706213,
1184
+ "learning_rate": 1.2649036817508264e-05,
1185
+ "loss": 0.0012083581089973449,
1186
+ "step": 8400
1187
+ },
1188
+ {
1189
+ "epoch": 0.8668444809191629,
1190
+ "grad_norm": 0.05253306403756142,
1191
+ "learning_rate": 1.2592043770659981e-05,
1192
+ "loss": 0.0018029493093490601,
1193
+ "step": 8450
1194
+ },
1195
+ {
1196
+ "epoch": 0.8719737382027083,
1197
+ "grad_norm": 0.019181661307811737,
1198
+ "learning_rate": 1.2535050723811696e-05,
1199
+ "loss": 0.0008252602815628052,
1200
+ "step": 8500
1201
+ },
1202
+ {
1203
+ "epoch": 0.8771029954862536,
1204
+ "grad_norm": 0.3479258716106415,
1205
+ "learning_rate": 1.2478057676963411e-05,
1206
+ "loss": 0.0017175833880901338,
1207
+ "step": 8550
1208
+ },
1209
+ {
1210
+ "epoch": 0.8822322527697989,
1211
+ "grad_norm": 0.05874745175242424,
1212
+ "learning_rate": 1.2421064630115126e-05,
1213
+ "loss": 0.002253309488296509,
1214
+ "step": 8600
1215
+ },
1216
+ {
1217
+ "epoch": 0.8873615100533443,
1218
+ "grad_norm": 0.025928650051355362,
1219
+ "learning_rate": 1.2364071583266843e-05,
1220
+ "loss": 0.0010428188741207123,
1221
+ "step": 8650
1222
+ },
1223
+ {
1224
+ "epoch": 0.8924907673368896,
1225
+ "grad_norm": 0.02793911285698414,
1226
+ "learning_rate": 1.2307078536418557e-05,
1227
+ "loss": 0.0010901305079460144,
1228
+ "step": 8700
1229
+ },
1230
+ {
1231
+ "epoch": 0.897620024620435,
1232
+ "grad_norm": 0.026237769052386284,
1233
+ "learning_rate": 1.2250085489570273e-05,
1234
+ "loss": 0.002191057354211807,
1235
+ "step": 8750
1236
+ },
1237
+ {
1238
+ "epoch": 0.9027492819039803,
1239
+ "grad_norm": 0.026652701199054718,
1240
+ "learning_rate": 1.219309244272199e-05,
1241
+ "loss": 0.0022504705190658568,
1242
+ "step": 8800
1243
+ },
1244
+ {
1245
+ "epoch": 0.9078785391875257,
1246
+ "grad_norm": 0.09745952486991882,
1247
+ "learning_rate": 1.2136099395873704e-05,
1248
+ "loss": 0.0013040122389793396,
1249
+ "step": 8850
1250
+ },
1251
+ {
1252
+ "epoch": 0.913007796471071,
1253
+ "grad_norm": 0.3571895956993103,
1254
+ "learning_rate": 1.207910634902542e-05,
1255
+ "loss": 0.0015031418204307556,
1256
+ "step": 8900
1257
+ },
1258
+ {
1259
+ "epoch": 0.9181370537546163,
1260
+ "grad_norm": 0.05399506166577339,
1261
+ "learning_rate": 1.2022113302177136e-05,
1262
+ "loss": 0.003377719819545746,
1263
+ "step": 8950
1264
+ },
1265
+ {
1266
+ "epoch": 0.9232663110381617,
1267
+ "grad_norm": 0.017251811921596527,
1268
+ "learning_rate": 1.196512025532885e-05,
1269
+ "loss": 0.0010861025750637055,
1270
+ "step": 9000
1271
+ },
1272
+ {
1273
+ "epoch": 0.928395568321707,
1274
+ "grad_norm": 0.01030634343624115,
1275
+ "learning_rate": 1.1908127208480566e-05,
1276
+ "loss": 0.0018379358947277068,
1277
+ "step": 9050
1278
+ },
1279
+ {
1280
+ "epoch": 0.9335248256052524,
1281
+ "grad_norm": 0.04392435401678085,
1282
+ "learning_rate": 1.1851134161632283e-05,
1283
+ "loss": 0.0013357341289520265,
1284
+ "step": 9100
1285
+ },
1286
+ {
1287
+ "epoch": 0.9386540828887977,
1288
+ "grad_norm": 0.009581666439771652,
1289
+ "learning_rate": 1.1794141114783996e-05,
1290
+ "loss": 0.0009307318925857544,
1291
+ "step": 9150
1292
+ },
1293
+ {
1294
+ "epoch": 0.943783340172343,
1295
+ "grad_norm": 0.017958320677280426,
1296
+ "learning_rate": 1.1737148067935713e-05,
1297
+ "loss": 0.0008443382382392884,
1298
+ "step": 9200
1299
+ },
1300
+ {
1301
+ "epoch": 0.9489125974558884,
1302
+ "grad_norm": 0.030002426356077194,
1303
+ "learning_rate": 1.168015502108743e-05,
1304
+ "loss": 0.0013130667805671692,
1305
+ "step": 9250
1306
+ },
1307
+ {
1308
+ "epoch": 0.9540418547394337,
1309
+ "grad_norm": 0.05621248111128807,
1310
+ "learning_rate": 1.1623161974239143e-05,
1311
+ "loss": 0.0012183979153633118,
1312
+ "step": 9300
1313
+ },
1314
+ {
1315
+ "epoch": 0.9591711120229791,
1316
+ "grad_norm": 0.015438093803822994,
1317
+ "learning_rate": 1.156616892739086e-05,
1318
+ "loss": 0.0012114503979682922,
1319
+ "step": 9350
1320
+ },
1321
+ {
1322
+ "epoch": 0.9643003693065244,
1323
+ "grad_norm": 0.07112756371498108,
1324
+ "learning_rate": 1.1509175880542575e-05,
1325
+ "loss": 0.0007900720834732055,
1326
+ "step": 9400
1327
+ },
1328
+ {
1329
+ "epoch": 0.9694296265900697,
1330
+ "grad_norm": 0.017932914197444916,
1331
+ "learning_rate": 1.145218283369429e-05,
1332
+ "loss": 0.0010725505650043487,
1333
+ "step": 9450
1334
+ },
1335
+ {
1336
+ "epoch": 0.9745588838736151,
1337
+ "grad_norm": 0.021347526460886,
1338
+ "learning_rate": 1.1395189786846005e-05,
1339
+ "loss": 0.0007509399950504303,
1340
+ "step": 9500
1341
+ },
1342
+ {
1343
+ "epoch": 0.9796881411571604,
1344
+ "grad_norm": 0.06269439309835434,
1345
+ "learning_rate": 1.1338196739997722e-05,
1346
+ "loss": 0.0014818912744522094,
1347
+ "step": 9550
1348
+ },
1349
+ {
1350
+ "epoch": 0.9848173984407058,
1351
+ "grad_norm": 0.030811917036771774,
1352
+ "learning_rate": 1.1281203693149435e-05,
1353
+ "loss": 0.0014112085103988648,
1354
+ "step": 9600
1355
+ },
1356
+ {
1357
+ "epoch": 0.9899466557242511,
1358
+ "grad_norm": 0.01883218064904213,
1359
+ "learning_rate": 1.1224210646301152e-05,
1360
+ "loss": 0.0012676098942756653,
1361
+ "step": 9650
1362
+ },
1363
+ {
1364
+ "epoch": 0.9950759130077965,
1365
+ "grad_norm": 0.011095144785940647,
1366
+ "learning_rate": 1.1167217599452869e-05,
1367
+ "loss": 0.0006470730900764465,
1368
+ "step": 9700
1369
+ },
1370
+ {
1371
+ "epoch": 1.0,
1372
+ "eval_biencoder-val_cosine_accuracy@1": 0.929,
1373
+ "eval_biencoder-val_cosine_accuracy@10": 1.0,
1374
+ "eval_biencoder-val_cosine_accuracy@30": 1.0,
1375
+ "eval_biencoder-val_cosine_accuracy@5": 1.0,
1376
+ "eval_biencoder-val_cosine_map@100": 0.9638333333333332,
1377
+ "eval_biencoder-val_cosine_mrr@10": 0.9638333333333332,
1378
+ "eval_biencoder-val_cosine_ndcg@10": 0.9732722934892877,
1379
+ "eval_biencoder-val_cosine_precision@1": 0.929,
1380
+ "eval_biencoder-val_cosine_precision@10": 0.10000000000000002,
1381
+ "eval_biencoder-val_cosine_precision@3": 0.33333333333333326,
1382
+ "eval_biencoder-val_cosine_precision@5": 0.20000000000000004,
1383
+ "eval_biencoder-val_cosine_recall@1": 0.929,
1384
+ "eval_biencoder-val_cosine_recall@10": 1.0,
1385
+ "eval_biencoder-val_cosine_recall@3": 1.0,
1386
+ "eval_biencoder-val_cosine_recall@5": 1.0,
1387
+ "eval_runtime": 0.8723,
1388
+ "eval_samples_per_second": 0.0,
1389
+ "eval_steps_per_second": 0.0,
1390
+ "step": 9748
1391
+ },
1392
+ {
1393
+ "epoch": 1.0002051702913417,
1394
+ "grad_norm": 0.020323621109128,
1395
+ "learning_rate": 1.1110224552604583e-05,
1396
+ "loss": 0.0013780112564563752,
1397
+ "step": 9750
1398
+ },
1399
+ {
1400
+ "epoch": 1.0053344275748872,
1401
+ "grad_norm": 0.004307797644287348,
1402
+ "learning_rate": 1.10532315057563e-05,
1403
+ "loss": 0.0006412163376808167,
1404
+ "step": 9800
1405
+ },
1406
+ {
1407
+ "epoch": 1.0104636848584325,
1408
+ "grad_norm": 0.009799239225685596,
1409
+ "learning_rate": 1.0996238458908014e-05,
1410
+ "loss": 0.0004977862536907196,
1411
+ "step": 9850
1412
+ },
1413
+ {
1414
+ "epoch": 1.0155929421419778,
1415
+ "grad_norm": 0.022461585700511932,
1416
+ "learning_rate": 1.093924541205973e-05,
1417
+ "loss": 0.0010745452344417571,
1418
+ "step": 9900
1419
+ },
1420
+ {
1421
+ "epoch": 1.0207221994255231,
1422
+ "grad_norm": 0.07712484151124954,
1423
+ "learning_rate": 1.0882252365211445e-05,
1424
+ "loss": 0.0010736149549484253,
1425
+ "step": 9950
1426
+ },
1427
+ {
1428
+ "epoch": 1.0258514567090686,
1429
+ "grad_norm": 0.5892838835716248,
1430
+ "learning_rate": 1.0825259318363162e-05,
1431
+ "loss": 0.0008927598595619201,
1432
+ "step": 10000
1433
+ },
1434
+ {
1435
+ "epoch": 1.030980713992614,
1436
+ "grad_norm": 0.017941532656550407,
1437
+ "learning_rate": 1.0768266271514875e-05,
1438
+ "loss": 0.0012636129558086396,
1439
+ "step": 10050
1440
+ },
1441
+ {
1442
+ "epoch": 1.0361099712761592,
1443
+ "grad_norm": 0.01072506420314312,
1444
+ "learning_rate": 1.0711273224666592e-05,
1445
+ "loss": 0.0007530584931373596,
1446
+ "step": 10100
1447
+ },
1448
+ {
1449
+ "epoch": 1.0412392285597045,
1450
+ "grad_norm": 0.0135742062702775,
1451
+ "learning_rate": 1.0654280177818309e-05,
1452
+ "loss": 0.0008298644423484803,
1453
+ "step": 10150
1454
+ },
1455
+ {
1456
+ "epoch": 1.0463684858432498,
1457
+ "grad_norm": 0.004919547121971846,
1458
+ "learning_rate": 1.0597287130970022e-05,
1459
+ "loss": 0.0006191575527191162,
1460
+ "step": 10200
1461
+ },
1462
+ {
1463
+ "epoch": 1.0514977431267953,
1464
+ "grad_norm": 0.006896316539496183,
1465
+ "learning_rate": 1.0540294084121739e-05,
1466
+ "loss": 0.0005970624089241028,
1467
+ "step": 10250
1468
+ },
1469
+ {
1470
+ "epoch": 1.0566270004103406,
1471
+ "grad_norm": 0.017362670972943306,
1472
+ "learning_rate": 1.0483301037273454e-05,
1473
+ "loss": 0.0010388505458831787,
1474
+ "step": 10300
1475
+ },
1476
+ {
1477
+ "epoch": 1.061756257693886,
1478
+ "grad_norm": 0.009948116727173328,
1479
+ "learning_rate": 1.0426307990425169e-05,
1480
+ "loss": 0.002600117623806,
1481
+ "step": 10350
1482
+ },
1483
+ {
1484
+ "epoch": 1.0668855149774312,
1485
+ "grad_norm": 0.13733653724193573,
1486
+ "learning_rate": 1.0369314943576884e-05,
1487
+ "loss": 0.0006903569400310516,
1488
+ "step": 10400
1489
+ },
1490
+ {
1491
+ "epoch": 1.0720147722609765,
1492
+ "grad_norm": 0.021996526047587395,
1493
+ "learning_rate": 1.0312321896728601e-05,
1494
+ "loss": 0.0006774544715881347,
1495
+ "step": 10450
1496
+ },
1497
+ {
1498
+ "epoch": 1.077144029544522,
1499
+ "grad_norm": 0.019396133720874786,
1500
+ "learning_rate": 1.0255328849880314e-05,
1501
+ "loss": 0.001027439534664154,
1502
+ "step": 10500
1503
+ },
1504
+ {
1505
+ "epoch": 1.0822732868280673,
1506
+ "grad_norm": 0.051219504326581955,
1507
+ "learning_rate": 1.0198335803032031e-05,
1508
+ "loss": 0.0016321080923080444,
1509
+ "step": 10550
1510
+ },
1511
+ {
1512
+ "epoch": 1.0874025441116126,
1513
+ "grad_norm": 0.042537640780210495,
1514
+ "learning_rate": 1.0141342756183748e-05,
1515
+ "loss": 0.001430421769618988,
1516
+ "step": 10600
1517
+ },
1518
+ {
1519
+ "epoch": 1.092531801395158,
1520
+ "grad_norm": 0.007988890632987022,
1521
+ "learning_rate": 1.0084349709335461e-05,
1522
+ "loss": 0.0007320943474769592,
1523
+ "step": 10650
1524
+ },
1525
+ {
1526
+ "epoch": 1.0976610586787032,
1527
+ "grad_norm": 0.04716062545776367,
1528
+ "learning_rate": 1.0027356662487178e-05,
1529
+ "loss": 0.0006467428803443909,
1530
+ "step": 10700
1531
+ },
1532
+ {
1533
+ "epoch": 1.1027903159622487,
1534
+ "grad_norm": 0.04211856424808502,
1535
+ "learning_rate": 9.970363615638893e-06,
1536
+ "loss": 0.0020744732022285463,
1537
+ "step": 10750
1538
+ },
1539
+ {
1540
+ "epoch": 1.107919573245794,
1541
+ "grad_norm": 0.022933784872293472,
1542
+ "learning_rate": 9.913370568790608e-06,
1543
+ "loss": 0.002727853357791901,
1544
+ "step": 10800
1545
+ },
1546
+ {
1547
+ "epoch": 1.1130488305293393,
1548
+ "grad_norm": 0.056059807538986206,
1549
+ "learning_rate": 9.856377521942324e-06,
1550
+ "loss": 0.0006137263774871827,
1551
+ "step": 10850
1552
+ },
1553
+ {
1554
+ "epoch": 1.1181780878128846,
1555
+ "grad_norm": 0.010968386195600033,
1556
+ "learning_rate": 9.799384475094039e-06,
1557
+ "loss": 0.0006363588571548462,
1558
+ "step": 10900
1559
+ },
1560
+ {
1561
+ "epoch": 1.1233073450964302,
1562
+ "grad_norm": 0.41986656188964844,
1563
+ "learning_rate": 9.742391428245754e-06,
1564
+ "loss": 0.0012197810411453246,
1565
+ "step": 10950
1566
+ },
1567
+ {
1568
+ "epoch": 1.1284366023799755,
1569
+ "grad_norm": 0.03437654301524162,
1570
+ "learning_rate": 9.68539838139747e-06,
1571
+ "loss": 0.0016349181532859802,
1572
+ "step": 11000
1573
+ },
1574
+ {
1575
+ "epoch": 1.1335658596635207,
1576
+ "grad_norm": 0.04348124563694,
1577
+ "learning_rate": 9.628405334549186e-06,
1578
+ "loss": 0.001471232771873474,
1579
+ "step": 11050
1580
+ },
1581
+ {
1582
+ "epoch": 1.138695116947066,
1583
+ "grad_norm": 0.03468090295791626,
1584
+ "learning_rate": 9.571412287700901e-06,
1585
+ "loss": 0.0025710177421569822,
1586
+ "step": 11100
1587
+ },
1588
+ {
1589
+ "epoch": 1.1438243742306113,
1590
+ "grad_norm": 0.007106569595634937,
1591
+ "learning_rate": 9.514419240852618e-06,
1592
+ "loss": 0.0008309808373451233,
1593
+ "step": 11150
1594
+ },
1595
+ {
1596
+ "epoch": 1.1489536315141566,
1597
+ "grad_norm": 0.040379106998443604,
1598
+ "learning_rate": 9.457426194004333e-06,
1599
+ "loss": 0.0007265037298202514,
1600
+ "step": 11200
1601
+ },
1602
+ {
1603
+ "epoch": 1.1540828887977022,
1604
+ "grad_norm": 0.07387026399374008,
1605
+ "learning_rate": 9.400433147156048e-06,
1606
+ "loss": 0.0007269151508808136,
1607
+ "step": 11250
1608
+ },
1609
+ {
1610
+ "epoch": 1.1592121460812475,
1611
+ "grad_norm": 0.006104862317442894,
1612
+ "learning_rate": 9.343440100307763e-06,
1613
+ "loss": 0.0012055251002311706,
1614
+ "step": 11300
1615
+ },
1616
+ {
1617
+ "epoch": 1.1643414033647927,
1618
+ "grad_norm": 0.026950648054480553,
1619
+ "learning_rate": 9.286447053459478e-06,
1620
+ "loss": 0.001047862470149994,
1621
+ "step": 11350
1622
+ },
1623
+ {
1624
+ "epoch": 1.169470660648338,
1625
+ "grad_norm": 0.19440318644046783,
1626
+ "learning_rate": 9.229454006611193e-06,
1627
+ "loss": 0.0005311770737171173,
1628
+ "step": 11400
1629
+ },
1630
+ {
1631
+ "epoch": 1.1745999179318836,
1632
+ "grad_norm": 0.06631787121295929,
1633
+ "learning_rate": 9.17246095976291e-06,
1634
+ "loss": 0.0007610043883323669,
1635
+ "step": 11450
1636
+ },
1637
+ {
1638
+ "epoch": 1.1797291752154289,
1639
+ "grad_norm": 0.027273444458842278,
1640
+ "learning_rate": 9.115467912914625e-06,
1641
+ "loss": 0.000621786117553711,
1642
+ "step": 11500
1643
+ },
1644
+ {
1645
+ "epoch": 1.1848584324989742,
1646
+ "grad_norm": 0.0168417040258646,
1647
+ "learning_rate": 9.05847486606634e-06,
1648
+ "loss": 0.0005212992429733277,
1649
+ "step": 11550
1650
+ },
1651
+ {
1652
+ "epoch": 1.1899876897825195,
1653
+ "grad_norm": 0.12111318111419678,
1654
+ "learning_rate": 9.001481819218057e-06,
1655
+ "loss": 0.0016236424446105957,
1656
+ "step": 11600
1657
+ },
1658
+ {
1659
+ "epoch": 1.1951169470660647,
1660
+ "grad_norm": 0.01940443180501461,
1661
+ "learning_rate": 8.944488772369772e-06,
1662
+ "loss": 0.0008924838900566101,
1663
+ "step": 11650
1664
+ },
1665
+ {
1666
+ "epoch": 1.2002462043496103,
1667
+ "grad_norm": 0.009674574248492718,
1668
+ "learning_rate": 8.887495725521487e-06,
1669
+ "loss": 0.0011235535144805907,
1670
+ "step": 11700
1671
+ },
1672
+ {
1673
+ "epoch": 1.2053754616331556,
1674
+ "grad_norm": 0.005509492009878159,
1675
+ "learning_rate": 8.830502678673202e-06,
1676
+ "loss": 0.0006038323044776917,
1677
+ "step": 11750
1678
+ },
1679
+ {
1680
+ "epoch": 1.2105047189167009,
1681
+ "grad_norm": 0.23156745731830597,
1682
+ "learning_rate": 8.773509631824918e-06,
1683
+ "loss": 0.0013854414224624633,
1684
+ "step": 11800
1685
+ },
1686
+ {
1687
+ "epoch": 1.2156339762002462,
1688
+ "grad_norm": 0.10829994827508926,
1689
+ "learning_rate": 8.716516584976633e-06,
1690
+ "loss": 0.000596238374710083,
1691
+ "step": 11850
1692
+ },
1693
+ {
1694
+ "epoch": 1.2207632334837915,
1695
+ "grad_norm": 0.004275528248399496,
1696
+ "learning_rate": 8.65952353812835e-06,
1697
+ "loss": 0.0006009680032730102,
1698
+ "step": 11900
1699
+ },
1700
+ {
1701
+ "epoch": 1.225892490767337,
1702
+ "grad_norm": 0.009944693185389042,
1703
+ "learning_rate": 8.602530491280065e-06,
1704
+ "loss": 0.0014707162976264954,
1705
+ "step": 11950
1706
+ },
1707
+ {
1708
+ "epoch": 1.2310217480508823,
1709
+ "grad_norm": 0.013600296340882778,
1710
+ "learning_rate": 8.54553744443178e-06,
1711
+ "loss": 0.0005389291048049926,
1712
+ "step": 12000
1713
+ },
1714
+ {
1715
+ "epoch": 1.2361510053344276,
1716
+ "grad_norm": 0.0223905798047781,
1717
+ "learning_rate": 8.488544397583497e-06,
1718
+ "loss": 0.003673248589038849,
1719
+ "step": 12050
1720
+ },
1721
+ {
1722
+ "epoch": 1.2412802626179729,
1723
+ "grad_norm": 0.012790598906576633,
1724
+ "learning_rate": 8.431551350735212e-06,
1725
+ "loss": 0.000642637312412262,
1726
+ "step": 12100
1727
+ },
1728
+ {
1729
+ "epoch": 1.2464095199015182,
1730
+ "grad_norm": 0.007352329324930906,
1731
+ "learning_rate": 8.374558303886927e-06,
1732
+ "loss": 0.0004784885048866272,
1733
+ "step": 12150
1734
+ },
1735
+ {
1736
+ "epoch": 1.2515387771850637,
1737
+ "grad_norm": 0.44836658239364624,
1738
+ "learning_rate": 8.317565257038642e-06,
1739
+ "loss": 0.0007071253657341003,
1740
+ "step": 12200
1741
+ },
1742
+ {
1743
+ "epoch": 1.256668034468609,
1744
+ "grad_norm": 0.06353507936000824,
1745
+ "learning_rate": 8.260572210190357e-06,
1746
+ "loss": 0.0015534579753875733,
1747
+ "step": 12250
1748
+ },
1749
+ {
1750
+ "epoch": 1.2617972917521543,
1751
+ "grad_norm": 0.07285618782043457,
1752
+ "learning_rate": 8.203579163342072e-06,
1753
+ "loss": 0.0005829083919525146,
1754
+ "step": 12300
1755
+ },
1756
+ {
1757
+ "epoch": 1.2669265490356996,
1758
+ "grad_norm": 0.004738403484225273,
1759
+ "learning_rate": 8.146586116493789e-06,
1760
+ "loss": 0.0015484358370304108,
1761
+ "step": 12350
1762
+ },
1763
+ {
1764
+ "epoch": 1.272055806319245,
1765
+ "grad_norm": 0.01686130091547966,
1766
+ "learning_rate": 8.089593069645504e-06,
1767
+ "loss": 0.0004922723770141601,
1768
+ "step": 12400
1769
+ },
1770
+ {
1771
+ "epoch": 1.2771850636027904,
1772
+ "grad_norm": 0.017023002728819847,
1773
+ "learning_rate": 8.03260002279722e-06,
1774
+ "loss": 0.000672076940536499,
1775
+ "step": 12450
1776
+ },
1777
+ {
1778
+ "epoch": 1.2823143208863357,
1779
+ "grad_norm": 0.011692719534039497,
1780
+ "learning_rate": 7.975606975948936e-06,
1781
+ "loss": 0.0006229496002197266,
1782
+ "step": 12500
1783
+ },
1784
+ {
1785
+ "epoch": 1.287443578169881,
1786
+ "grad_norm": 0.009096607565879822,
1787
+ "learning_rate": 7.918613929100651e-06,
1788
+ "loss": 0.0009301286935806274,
1789
+ "step": 12550
1790
+ },
1791
+ {
1792
+ "epoch": 1.2925728354534263,
1793
+ "grad_norm": 0.055945273488759995,
1794
+ "learning_rate": 7.861620882252366e-06,
1795
+ "loss": 0.0004671996831893921,
1796
+ "step": 12600
1797
+ },
1798
+ {
1799
+ "epoch": 1.2977020927369716,
1800
+ "grad_norm": 0.01874576508998871,
1801
+ "learning_rate": 7.804627835404081e-06,
1802
+ "loss": 0.00038488924503326415,
1803
+ "step": 12650
1804
+ },
1805
+ {
1806
+ "epoch": 1.302831350020517,
1807
+ "grad_norm": 0.02037273906171322,
1808
+ "learning_rate": 7.747634788555796e-06,
1809
+ "loss": 0.001074599027633667,
1810
+ "step": 12700
1811
+ },
1812
+ {
1813
+ "epoch": 1.3079606073040624,
1814
+ "grad_norm": 0.028113173320889473,
1815
+ "learning_rate": 7.690641741707512e-06,
1816
+ "loss": 0.0006131212413311005,
1817
+ "step": 12750
1818
+ },
1819
+ {
1820
+ "epoch": 1.3130898645876077,
1821
+ "grad_norm": 0.007892386987805367,
1822
+ "learning_rate": 7.633648694859228e-06,
1823
+ "loss": 0.000581781268119812,
1824
+ "step": 12800
1825
+ },
1826
+ {
1827
+ "epoch": 1.318219121871153,
1828
+ "grad_norm": 0.04557061940431595,
1829
+ "learning_rate": 7.5766556480109435e-06,
1830
+ "loss": 0.0005579075217247009,
1831
+ "step": 12850
1832
+ },
1833
+ {
1834
+ "epoch": 1.3233483791546985,
1835
+ "grad_norm": 0.004758005030453205,
1836
+ "learning_rate": 7.519662601162659e-06,
1837
+ "loss": 0.00044855713844299316,
1838
+ "step": 12900
1839
+ },
1840
+ {
1841
+ "epoch": 1.3284776364382438,
1842
+ "grad_norm": 0.053557779639959335,
1843
+ "learning_rate": 7.462669554314375e-06,
1844
+ "loss": 0.0014822837710380555,
1845
+ "step": 12950
1846
+ },
1847
+ {
1848
+ "epoch": 1.333606893721789,
1849
+ "grad_norm": 0.013253672979772091,
1850
+ "learning_rate": 7.40567650746609e-06,
1851
+ "loss": 0.0013494199514389037,
1852
+ "step": 13000
1853
+ },
1854
+ {
1855
+ "epoch": 1.3387361510053344,
1856
+ "grad_norm": 0.007712292950600386,
1857
+ "learning_rate": 7.348683460617805e-06,
1858
+ "loss": 0.0008460769057273865,
1859
+ "step": 13050
1860
+ },
1861
+ {
1862
+ "epoch": 1.3438654082888797,
1863
+ "grad_norm": 0.028559701517224312,
1864
+ "learning_rate": 7.291690413769521e-06,
1865
+ "loss": 0.0009650763869285584,
1866
+ "step": 13100
1867
+ },
1868
+ {
1869
+ "epoch": 1.348994665572425,
1870
+ "grad_norm": 0.03476807847619057,
1871
+ "learning_rate": 7.234697366921236e-06,
1872
+ "loss": 0.0006557583808898925,
1873
+ "step": 13150
1874
+ },
1875
+ {
1876
+ "epoch": 1.3541239228559705,
1877
+ "grad_norm": 0.03715438395738602,
1878
+ "learning_rate": 7.177704320072951e-06,
1879
+ "loss": 0.001336461305618286,
1880
+ "step": 13200
1881
+ },
1882
+ {
1883
+ "epoch": 1.3592531801395158,
1884
+ "grad_norm": 0.006591553334146738,
1885
+ "learning_rate": 7.120711273224668e-06,
1886
+ "loss": 0.0005404025316238404,
1887
+ "step": 13250
1888
+ },
1889
+ {
1890
+ "epoch": 1.364382437423061,
1891
+ "grad_norm": 0.013354040682315826,
1892
+ "learning_rate": 7.063718226376383e-06,
1893
+ "loss": 0.0016759490966796874,
1894
+ "step": 13300
1895
+ },
1896
+ {
1897
+ "epoch": 1.3695116947066066,
1898
+ "grad_norm": 0.004464422352612019,
1899
+ "learning_rate": 7.006725179528098e-06,
1900
+ "loss": 0.0020402561128139494,
1901
+ "step": 13350
1902
+ },
1903
+ {
1904
+ "epoch": 1.374640951990152,
1905
+ "grad_norm": 0.0071783666498959064,
1906
+ "learning_rate": 6.949732132679814e-06,
1907
+ "loss": 0.00044939160346984863,
1908
+ "step": 13400
1909
+ },
1910
+ {
1911
+ "epoch": 1.3797702092736972,
1912
+ "grad_norm": 0.04091225564479828,
1913
+ "learning_rate": 6.892739085831529e-06,
1914
+ "loss": 0.00044884562492370606,
1915
+ "step": 13450
1916
+ },
1917
+ {
1918
+ "epoch": 1.3848994665572425,
1919
+ "grad_norm": 0.008949765004217625,
1920
+ "learning_rate": 6.835746038983244e-06,
1921
+ "loss": 0.0009319144487380982,
1922
+ "step": 13500
1923
+ },
1924
+ {
1925
+ "epoch": 1.3900287238407878,
1926
+ "grad_norm": 0.016390034928917885,
1927
+ "learning_rate": 6.77875299213496e-06,
1928
+ "loss": 0.0004179143905639648,
1929
+ "step": 13550
1930
+ },
1931
+ {
1932
+ "epoch": 1.395157981124333,
1933
+ "grad_norm": 0.044730667024850845,
1934
+ "learning_rate": 6.721759945286675e-06,
1935
+ "loss": 0.0008553892374038696,
1936
+ "step": 13600
1937
+ },
1938
+ {
1939
+ "epoch": 1.4002872384078786,
1940
+ "grad_norm": 0.03032684326171875,
1941
+ "learning_rate": 6.6647668984383905e-06,
1942
+ "loss": 0.00040693581104278564,
1943
+ "step": 13650
1944
+ },
1945
+ {
1946
+ "epoch": 1.405416495691424,
1947
+ "grad_norm": 0.00720440037548542,
1948
+ "learning_rate": 6.6077738515901064e-06,
1949
+ "loss": 0.0007417532801628113,
1950
+ "step": 13700
1951
+ },
1952
+ {
1953
+ "epoch": 1.4105457529749692,
1954
+ "grad_norm": 0.043598294258117676,
1955
+ "learning_rate": 6.5507808047418216e-06,
1956
+ "loss": 0.001252269446849823,
1957
+ "step": 13750
1958
+ },
1959
+ {
1960
+ "epoch": 1.4156750102585145,
1961
+ "grad_norm": 0.2630448043346405,
1962
+ "learning_rate": 6.493787757893537e-06,
1963
+ "loss": 0.0006510725617408752,
1964
+ "step": 13800
1965
+ },
1966
+ {
1967
+ "epoch": 1.42080426754206,
1968
+ "grad_norm": 0.12175612151622772,
1969
+ "learning_rate": 6.4367947110452535e-06,
1970
+ "loss": 0.0009742790460586548,
1971
+ "step": 13850
1972
+ },
1973
+ {
1974
+ "epoch": 1.4259335248256053,
1975
+ "grad_norm": 0.008543884381651878,
1976
+ "learning_rate": 6.379801664196969e-06,
1977
+ "loss": 0.0010201582312583924,
1978
+ "step": 13900
1979
+ },
1980
+ {
1981
+ "epoch": 1.4310627821091506,
1982
+ "grad_norm": 0.011405929923057556,
1983
+ "learning_rate": 6.322808617348684e-06,
1984
+ "loss": 0.0008232490718364715,
1985
+ "step": 13950
1986
+ },
1987
+ {
1988
+ "epoch": 1.436192039392696,
1989
+ "grad_norm": 0.006674911826848984,
1990
+ "learning_rate": 6.2658155705004e-06,
1991
+ "loss": 0.0009403946995735168,
1992
+ "step": 14000
1993
+ },
1994
+ {
1995
+ "epoch": 1.4413212966762412,
1996
+ "grad_norm": 0.02636873908340931,
1997
+ "learning_rate": 6.208822523652115e-06,
1998
+ "loss": 0.0015288326144218445,
1999
+ "step": 14050
2000
+ },
2001
+ {
2002
+ "epoch": 1.4464505539597865,
2003
+ "grad_norm": 0.010799755342304707,
2004
+ "learning_rate": 6.15182947680383e-06,
2005
+ "loss": 0.0008086912333965301,
2006
+ "step": 14100
2007
+ },
2008
+ {
2009
+ "epoch": 1.451579811243332,
2010
+ "grad_norm": 0.00788508728146553,
2011
+ "learning_rate": 6.094836429955546e-06,
2012
+ "loss": 0.000679481029510498,
2013
+ "step": 14150
2014
+ },
2015
+ {
2016
+ "epoch": 1.4567090685268773,
2017
+ "grad_norm": 0.1568962037563324,
2018
+ "learning_rate": 6.037843383107261e-06,
2019
+ "loss": 0.0006565728783607483,
2020
+ "step": 14200
2021
+ },
2022
+ {
2023
+ "epoch": 1.4618383258104226,
2024
+ "grad_norm": 0.025997241958975792,
2025
+ "learning_rate": 5.980850336258976e-06,
2026
+ "loss": 0.0010558784008026123,
2027
+ "step": 14250
2028
+ },
2029
+ {
2030
+ "epoch": 1.4669675830939681,
2031
+ "grad_norm": 0.013356027193367481,
2032
+ "learning_rate": 5.923857289410693e-06,
2033
+ "loss": 0.0022724229097366333,
2034
+ "step": 14300
2035
+ },
2036
+ {
2037
+ "epoch": 1.4720968403775134,
2038
+ "grad_norm": 0.025233469903469086,
2039
+ "learning_rate": 5.866864242562408e-06,
2040
+ "loss": 0.0011099863052368164,
2041
+ "step": 14350
2042
+ },
2043
+ {
2044
+ "epoch": 1.4772260976610587,
2045
+ "grad_norm": 0.00507698580622673,
2046
+ "learning_rate": 5.809871195714123e-06,
2047
+ "loss": 0.0004247647523880005,
2048
+ "step": 14400
2049
+ },
2050
+ {
2051
+ "epoch": 1.482355354944604,
2052
+ "grad_norm": 0.005569028202444315,
2053
+ "learning_rate": 5.752878148865839e-06,
2054
+ "loss": 0.0008500492572784423,
2055
+ "step": 14450
2056
+ },
2057
+ {
2058
+ "epoch": 1.4874846122281493,
2059
+ "grad_norm": 0.06364163011312485,
2060
+ "learning_rate": 5.695885102017554e-06,
2061
+ "loss": 0.0005384421348571777,
2062
+ "step": 14500
2063
+ },
2064
+ {
2065
+ "epoch": 1.4926138695116946,
2066
+ "grad_norm": 0.0061021894216537476,
2067
+ "learning_rate": 5.638892055169269e-06,
2068
+ "loss": 0.0009390240907669067,
2069
+ "step": 14550
2070
+ },
2071
+ {
2072
+ "epoch": 1.4977431267952401,
2073
+ "grad_norm": 0.06408653408288956,
2074
+ "learning_rate": 5.581899008320985e-06,
2075
+ "loss": 0.0003880223631858826,
2076
+ "step": 14600
2077
+ },
2078
+ {
2079
+ "epoch": 1.5028723840787854,
2080
+ "grad_norm": 0.006361554842442274,
2081
+ "learning_rate": 5.5249059614727005e-06,
2082
+ "loss": 0.00038288965821266176,
2083
+ "step": 14650
2084
+ },
2085
+ {
2086
+ "epoch": 1.5080016413623307,
2087
+ "grad_norm": 0.012359695509076118,
2088
+ "learning_rate": 5.4679129146244156e-06,
2089
+ "loss": 0.00042018115520477296,
2090
+ "step": 14700
2091
+ },
2092
+ {
2093
+ "epoch": 1.5131308986458762,
2094
+ "grad_norm": 0.09342452883720398,
2095
+ "learning_rate": 5.410919867776132e-06,
2096
+ "loss": 0.0010700696706771851,
2097
+ "step": 14750
2098
+ },
2099
+ {
2100
+ "epoch": 1.5182601559294215,
2101
+ "grad_norm": 0.011165376752614975,
2102
+ "learning_rate": 5.3539268209278475e-06,
2103
+ "loss": 0.002023402750492096,
2104
+ "step": 14800
2105
+ },
2106
+ {
2107
+ "epoch": 1.5233894132129668,
2108
+ "grad_norm": 0.04357055202126503,
2109
+ "learning_rate": 5.296933774079563e-06,
2110
+ "loss": 0.0007374601066112518,
2111
+ "step": 14850
2112
+ },
2113
+ {
2114
+ "epoch": 1.5285186704965121,
2115
+ "grad_norm": 0.004575764294713736,
2116
+ "learning_rate": 5.239940727231279e-06,
2117
+ "loss": 0.0004954004287719727,
2118
+ "step": 14900
2119
+ },
2120
+ {
2121
+ "epoch": 1.5336479277800574,
2122
+ "grad_norm": 0.01638239435851574,
2123
+ "learning_rate": 5.182947680382994e-06,
2124
+ "loss": 0.0004453620314598084,
2125
+ "step": 14950
2126
+ },
2127
+ {
2128
+ "epoch": 1.5387771850636027,
2129
+ "grad_norm": 0.020702429115772247,
2130
+ "learning_rate": 5.125954633534709e-06,
2131
+ "loss": 0.0004130262136459351,
2132
+ "step": 15000
2133
+ },
2134
+ {
2135
+ "epoch": 1.543906442347148,
2136
+ "grad_norm": 0.036988165229558945,
2137
+ "learning_rate": 5.068961586686425e-06,
2138
+ "loss": 0.0008672660589218139,
2139
+ "step": 15050
2140
+ },
2141
+ {
2142
+ "epoch": 1.5490356996306933,
2143
+ "grad_norm": 0.006969288922846317,
2144
+ "learning_rate": 5.01196853983814e-06,
2145
+ "loss": 0.0005355362594127656,
2146
+ "step": 15100
2147
+ },
2148
+ {
2149
+ "epoch": 1.5541649569142388,
2150
+ "grad_norm": 0.0261154156178236,
2151
+ "learning_rate": 4.954975492989856e-06,
2152
+ "loss": 0.0006160007417201996,
2153
+ "step": 15150
2154
+ },
2155
+ {
2156
+ "epoch": 1.5592942141977841,
2157
+ "grad_norm": 0.003943952731788158,
2158
+ "learning_rate": 4.897982446141571e-06,
2159
+ "loss": 0.0008477187156677246,
2160
+ "step": 15200
2161
+ },
2162
+ {
2163
+ "epoch": 1.5644234714813297,
2164
+ "grad_norm": 0.01097947172820568,
2165
+ "learning_rate": 4.840989399293287e-06,
2166
+ "loss": 0.000611473023891449,
2167
+ "step": 15250
2168
+ },
2169
+ {
2170
+ "epoch": 1.569552728764875,
2171
+ "grad_norm": 0.2929246425628662,
2172
+ "learning_rate": 4.783996352445002e-06,
2173
+ "loss": 0.0022622349858283995,
2174
+ "step": 15300
2175
+ },
2176
+ {
2177
+ "epoch": 1.5746819860484202,
2178
+ "grad_norm": 0.020579788833856583,
2179
+ "learning_rate": 4.727003305596717e-06,
2180
+ "loss": 0.000628172755241394,
2181
+ "step": 15350
2182
+ },
2183
+ {
2184
+ "epoch": 1.5798112433319655,
2185
+ "grad_norm": 0.04158257693052292,
2186
+ "learning_rate": 4.670010258748433e-06,
2187
+ "loss": 0.0005809402465820312,
2188
+ "step": 15400
2189
+ },
2190
+ {
2191
+ "epoch": 1.5849405006155108,
2192
+ "grad_norm": 0.030321037396788597,
2193
+ "learning_rate": 4.613017211900149e-06,
2194
+ "loss": 0.0012504719197750092,
2195
+ "step": 15450
2196
+ },
2197
+ {
2198
+ "epoch": 1.5900697578990561,
2199
+ "grad_norm": 0.003760402789339423,
2200
+ "learning_rate": 4.556024165051864e-06,
2201
+ "loss": 0.0005912356078624726,
2202
+ "step": 15500
2203
+ },
2204
+ {
2205
+ "epoch": 1.5951990151826014,
2206
+ "grad_norm": 0.007559386547654867,
2207
+ "learning_rate": 4.499031118203579e-06,
2208
+ "loss": 0.00040479302406311036,
2209
+ "step": 15550
2210
+ },
2211
+ {
2212
+ "epoch": 1.600328272466147,
2213
+ "grad_norm": 0.03453294560313225,
2214
+ "learning_rate": 4.442038071355295e-06,
2215
+ "loss": 0.0020462825894355776,
2216
+ "step": 15600
2217
+ },
2218
+ {
2219
+ "epoch": 1.6054575297496922,
2220
+ "grad_norm": 0.020316729322075844,
2221
+ "learning_rate": 4.38504502450701e-06,
2222
+ "loss": 0.0016433340311050415,
2223
+ "step": 15650
2224
+ },
2225
+ {
2226
+ "epoch": 1.6105867870332375,
2227
+ "grad_norm": 0.01812104694545269,
2228
+ "learning_rate": 4.328051977658726e-06,
2229
+ "loss": 0.0004745867848396301,
2230
+ "step": 15700
2231
+ },
2232
+ {
2233
+ "epoch": 1.615716044316783,
2234
+ "grad_norm": 0.018005484715104103,
2235
+ "learning_rate": 4.2710589308104415e-06,
2236
+ "loss": 0.0004400739073753357,
2237
+ "step": 15750
2238
+ },
2239
+ {
2240
+ "epoch": 1.6208453016003284,
2241
+ "grad_norm": 0.1476868987083435,
2242
+ "learning_rate": 4.214065883962157e-06,
2243
+ "loss": 0.0010610869526863098,
2244
+ "step": 15800
2245
+ },
2246
+ {
2247
+ "epoch": 1.6259745588838737,
2248
+ "grad_norm": 0.2994936406612396,
2249
+ "learning_rate": 4.157072837113873e-06,
2250
+ "loss": 0.001360025405883789,
2251
+ "step": 15850
2252
+ },
2253
+ {
2254
+ "epoch": 1.631103816167419,
2255
+ "grad_norm": 0.002983258804306388,
2256
+ "learning_rate": 4.100079790265588e-06,
2257
+ "loss": 0.0005485677719116211,
2258
+ "step": 15900
2259
+ },
2260
+ {
2261
+ "epoch": 1.6362330734509642,
2262
+ "grad_norm": 0.005654474720358849,
2263
+ "learning_rate": 4.043086743417303e-06,
2264
+ "loss": 0.001110573410987854,
2265
+ "step": 15950
2266
+ },
2267
+ {
2268
+ "epoch": 1.6413623307345095,
2269
+ "grad_norm": 0.019883785396814346,
2270
+ "learning_rate": 3.986093696569019e-06,
2271
+ "loss": 0.0007011586427688599,
2272
+ "step": 16000
2273
+ },
2274
+ {
2275
+ "epoch": 1.6464915880180548,
2276
+ "grad_norm": 0.04697249457240105,
2277
+ "learning_rate": 3.929100649720735e-06,
2278
+ "loss": 0.0005320942401885986,
2279
+ "step": 16050
2280
+ },
2281
+ {
2282
+ "epoch": 1.6516208453016004,
2283
+ "grad_norm": 0.004929785616695881,
2284
+ "learning_rate": 3.87210760287245e-06,
2285
+ "loss": 0.0008407318592071533,
2286
+ "step": 16100
2287
+ },
2288
+ {
2289
+ "epoch": 1.6567501025851457,
2290
+ "grad_norm": 0.03038635477423668,
2291
+ "learning_rate": 3.815114556024165e-06,
2292
+ "loss": 0.00030867159366607664,
2293
+ "step": 16150
2294
+ },
2295
+ {
2296
+ "epoch": 1.6618793598686912,
2297
+ "grad_norm": 0.06552842259407043,
2298
+ "learning_rate": 3.758121509175881e-06,
2299
+ "loss": 0.0011920902132987977,
2300
+ "step": 16200
2301
+ },
2302
+ {
2303
+ "epoch": 1.6670086171522365,
2304
+ "grad_norm": 0.013011803850531578,
2305
+ "learning_rate": 3.701128462327596e-06,
2306
+ "loss": 0.000447293221950531,
2307
+ "step": 16250
2308
+ },
2309
+ {
2310
+ "epoch": 1.6721378744357818,
2311
+ "grad_norm": 0.00991652812808752,
2312
+ "learning_rate": 3.644135415479312e-06,
2313
+ "loss": 0.00033876419067382815,
2314
+ "step": 16300
2315
+ },
2316
+ {
2317
+ "epoch": 1.677267131719327,
2318
+ "grad_norm": 0.025500046089291573,
2319
+ "learning_rate": 3.5871423686310276e-06,
2320
+ "loss": 0.0002862626314163208,
2321
+ "step": 16350
2322
+ },
2323
+ {
2324
+ "epoch": 1.6823963890028724,
2325
+ "grad_norm": 0.054780106991529465,
2326
+ "learning_rate": 3.5301493217827427e-06,
2327
+ "loss": 0.0022387760877609253,
2328
+ "step": 16400
2329
+ },
2330
+ {
2331
+ "epoch": 1.6875256462864177,
2332
+ "grad_norm": 0.009139726869761944,
2333
+ "learning_rate": 3.4731562749344582e-06,
2334
+ "loss": 0.0004343575239181519,
2335
+ "step": 16450
2336
+ },
2337
+ {
2338
+ "epoch": 1.692654903569963,
2339
+ "grad_norm": 0.005948680453002453,
2340
+ "learning_rate": 3.4161632280861738e-06,
2341
+ "loss": 0.000540688931941986,
2342
+ "step": 16500
2343
+ },
2344
+ {
2345
+ "epoch": 1.6977841608535083,
2346
+ "grad_norm": 0.028869858011603355,
2347
+ "learning_rate": 3.359170181237889e-06,
2348
+ "loss": 0.00035138726234436036,
2349
+ "step": 16550
2350
+ },
2351
+ {
2352
+ "epoch": 1.7029134181370538,
2353
+ "grad_norm": 0.0448104664683342,
2354
+ "learning_rate": 3.302177134389605e-06,
2355
+ "loss": 0.0005932286381721497,
2356
+ "step": 16600
2357
+ },
2358
+ {
2359
+ "epoch": 1.708042675420599,
2360
+ "grad_norm": 0.09236075729131699,
2361
+ "learning_rate": 3.2451840875413204e-06,
2362
+ "loss": 0.0006601068377494812,
2363
+ "step": 16650
2364
+ },
2365
+ {
2366
+ "epoch": 1.7131719327041446,
2367
+ "grad_norm": 0.014860023744404316,
2368
+ "learning_rate": 3.1881910406930355e-06,
2369
+ "loss": 0.0008951804041862488,
2370
+ "step": 16700
2371
+ },
2372
+ {
2373
+ "epoch": 1.7183011899876899,
2374
+ "grad_norm": 0.019764918833971024,
2375
+ "learning_rate": 3.131197993844751e-06,
2376
+ "loss": 0.00040823638439178465,
2377
+ "step": 16750
2378
+ },
2379
+ {
2380
+ "epoch": 1.7234304472712352,
2381
+ "grad_norm": 0.008326166309416294,
2382
+ "learning_rate": 3.074204946996467e-06,
2383
+ "loss": 0.001432819664478302,
2384
+ "step": 16800
2385
+ },
2386
+ {
2387
+ "epoch": 1.7285597045547805,
2388
+ "grad_norm": 0.0029986989684402943,
2389
+ "learning_rate": 3.017211900148182e-06,
2390
+ "loss": 0.0012194857001304627,
2391
+ "step": 16850
2392
+ },
2393
+ {
2394
+ "epoch": 1.7336889618383258,
2395
+ "grad_norm": 0.09672784060239792,
2396
+ "learning_rate": 2.9602188532998977e-06,
2397
+ "loss": 0.00047722727060317994,
2398
+ "step": 16900
2399
+ },
2400
+ {
2401
+ "epoch": 1.738818219121871,
2402
+ "grad_norm": 0.0037852392997592688,
2403
+ "learning_rate": 2.903225806451613e-06,
2404
+ "loss": 0.0008305265009403229,
2405
+ "step": 16950
2406
+ },
2407
+ {
2408
+ "epoch": 1.7439474764054164,
2409
+ "grad_norm": 0.36351898312568665,
2410
+ "learning_rate": 2.8462327596033283e-06,
2411
+ "loss": 0.0005665953457355499,
2412
+ "step": 17000
2413
+ },
2414
+ {
2415
+ "epoch": 1.7490767336889619,
2416
+ "grad_norm": 0.006676111835986376,
2417
+ "learning_rate": 2.7892397127550443e-06,
2418
+ "loss": 0.0007315759360790253,
2419
+ "step": 17050
2420
+ },
2421
+ {
2422
+ "epoch": 1.7542059909725072,
2423
+ "grad_norm": 0.07323434203863144,
2424
+ "learning_rate": 2.73224666590676e-06,
2425
+ "loss": 0.00037639960646629333,
2426
+ "step": 17100
2427
+ },
2428
+ {
2429
+ "epoch": 1.7593352482560525,
2430
+ "grad_norm": 0.013121088966727257,
2431
+ "learning_rate": 2.675253619058475e-06,
2432
+ "loss": 0.0011937808990478515,
2433
+ "step": 17150
2434
+ },
2435
+ {
2436
+ "epoch": 1.764464505539598,
2437
+ "grad_norm": 0.03192434832453728,
2438
+ "learning_rate": 2.6182605722101905e-06,
2439
+ "loss": 0.0013308241963386537,
2440
+ "step": 17200
2441
+ },
2442
+ {
2443
+ "epoch": 1.7695937628231433,
2444
+ "grad_norm": 0.014523284509778023,
2445
+ "learning_rate": 2.5612675253619065e-06,
2446
+ "loss": 0.0010080689191818237,
2447
+ "step": 17250
2448
+ },
2449
+ {
2450
+ "epoch": 1.7747230201066886,
2451
+ "grad_norm": 0.003345728386193514,
2452
+ "learning_rate": 2.5042744785136216e-06,
2453
+ "loss": 0.0009157347679138183,
2454
+ "step": 17300
2455
+ },
2456
+ {
2457
+ "epoch": 1.7798522773902339,
2458
+ "grad_norm": 0.011479219421744347,
2459
+ "learning_rate": 2.447281431665337e-06,
2460
+ "loss": 0.00039456844329833986,
2461
+ "step": 17350
2462
+ },
2463
+ {
2464
+ "epoch": 1.7849815346737792,
2465
+ "grad_norm": 0.00539002101868391,
2466
+ "learning_rate": 2.3902883848170527e-06,
2467
+ "loss": 0.0003252148628234863,
2468
+ "step": 17400
2469
+ },
2470
+ {
2471
+ "epoch": 1.7901107919573245,
2472
+ "grad_norm": 0.0179832074791193,
2473
+ "learning_rate": 2.333295337968768e-06,
2474
+ "loss": 0.0006710499525070191,
2475
+ "step": 17450
2476
+ },
2477
+ {
2478
+ "epoch": 1.7952400492408698,
2479
+ "grad_norm": 0.019012339413166046,
2480
+ "learning_rate": 2.2763022911204833e-06,
2481
+ "loss": 0.0003579908609390259,
2482
+ "step": 17500
2483
+ },
2484
+ {
2485
+ "epoch": 1.8003693065244153,
2486
+ "grad_norm": 0.021011650562286377,
2487
+ "learning_rate": 2.219309244272199e-06,
2488
+ "loss": 0.0003782680630683899,
2489
+ "step": 17550
2490
+ },
2491
+ {
2492
+ "epoch": 1.8054985638079606,
2493
+ "grad_norm": 0.008447716012597084,
2494
+ "learning_rate": 2.1623161974239144e-06,
2495
+ "loss": 0.0005027109384536743,
2496
+ "step": 17600
2497
+ },
2498
+ {
2499
+ "epoch": 1.810627821091506,
2500
+ "grad_norm": 0.07184159755706787,
2501
+ "learning_rate": 2.10532315057563e-06,
2502
+ "loss": 0.00039920926094055175,
2503
+ "step": 17650
2504
+ },
2505
+ {
2506
+ "epoch": 1.8157570783750514,
2507
+ "grad_norm": 0.009770162403583527,
2508
+ "learning_rate": 2.0483301037273455e-06,
2509
+ "loss": 0.0002949029207229614,
2510
+ "step": 17700
2511
+ },
2512
+ {
2513
+ "epoch": 1.8208863356585967,
2514
+ "grad_norm": 0.02992192842066288,
2515
+ "learning_rate": 1.991337056879061e-06,
2516
+ "loss": 0.0003519010543823242,
2517
+ "step": 17750
2518
+ },
2519
+ {
2520
+ "epoch": 1.826015592942142,
2521
+ "grad_norm": 0.007283203303813934,
2522
+ "learning_rate": 1.9343440100307766e-06,
2523
+ "loss": 0.001447817087173462,
2524
+ "step": 17800
2525
+ },
2526
+ {
2527
+ "epoch": 1.8311448502256873,
2528
+ "grad_norm": 0.0020318494644016027,
2529
+ "learning_rate": 1.8773509631824919e-06,
2530
+ "loss": 0.0003602537512779236,
2531
+ "step": 17850
2532
+ },
2533
+ {
2534
+ "epoch": 1.8362741075092326,
2535
+ "grad_norm": 0.008025232702493668,
2536
+ "learning_rate": 1.8203579163342074e-06,
2537
+ "loss": 0.0007793241739273071,
2538
+ "step": 17900
2539
+ },
2540
+ {
2541
+ "epoch": 1.8414033647927779,
2542
+ "grad_norm": 0.024956921115517616,
2543
+ "learning_rate": 1.763364869485923e-06,
2544
+ "loss": 0.00035044103860855104,
2545
+ "step": 17950
2546
+ },
2547
+ {
2548
+ "epoch": 1.8465326220763232,
2549
+ "grad_norm": 0.005003917030990124,
2550
+ "learning_rate": 1.7063718226376383e-06,
2551
+ "loss": 0.000377134382724762,
2552
+ "step": 18000
2553
+ },
2554
+ {
2555
+ "epoch": 1.8516618793598687,
2556
+ "grad_norm": 0.006638799328356981,
2557
+ "learning_rate": 1.649378775789354e-06,
2558
+ "loss": 0.0006813791394233704,
2559
+ "step": 18050
2560
+ },
2561
+ {
2562
+ "epoch": 1.856791136643414,
2563
+ "grad_norm": 0.023331264033913612,
2564
+ "learning_rate": 1.5923857289410694e-06,
2565
+ "loss": 0.0003337270021438599,
2566
+ "step": 18100
2567
+ },
2568
+ {
2569
+ "epoch": 1.8619203939269595,
2570
+ "grad_norm": 0.01474565640091896,
2571
+ "learning_rate": 1.5353926820927847e-06,
2572
+ "loss": 0.0003927820920944214,
2573
+ "step": 18150
2574
+ },
2575
+ {
2576
+ "epoch": 1.8670496512105048,
2577
+ "grad_norm": 0.004943469073623419,
2578
+ "learning_rate": 1.4783996352445005e-06,
2579
+ "loss": 0.0019324821233749389,
2580
+ "step": 18200
2581
+ },
2582
+ {
2583
+ "epoch": 1.87217890849405,
2584
+ "grad_norm": 0.006063356995582581,
2585
+ "learning_rate": 1.4214065883962158e-06,
2586
+ "loss": 0.0004604470729827881,
2587
+ "step": 18250
2588
+ },
2589
+ {
2590
+ "epoch": 1.8773081657775954,
2591
+ "grad_norm": 0.00663331663236022,
2592
+ "learning_rate": 1.3644135415479311e-06,
2593
+ "loss": 0.0006617265939712525,
2594
+ "step": 18300
2595
+ },
2596
+ {
2597
+ "epoch": 1.8824374230611407,
2598
+ "grad_norm": 0.006572545040398836,
2599
+ "learning_rate": 1.3074204946996469e-06,
2600
+ "loss": 0.00035414040088653563,
2601
+ "step": 18350
2602
+ },
2603
+ {
2604
+ "epoch": 1.887566680344686,
2605
+ "grad_norm": 0.03555886074900627,
2606
+ "learning_rate": 1.2504274478513622e-06,
2607
+ "loss": 0.0013728688657283783,
2608
+ "step": 18400
2609
+ },
2610
+ {
2611
+ "epoch": 1.8926959376282313,
2612
+ "grad_norm": 0.00993818137794733,
2613
+ "learning_rate": 1.1934344010030777e-06,
2614
+ "loss": 0.0005572491884231567,
2615
+ "step": 18450
2616
+ },
2617
+ {
2618
+ "epoch": 1.8978251949117768,
2619
+ "grad_norm": 0.04966617748141289,
2620
+ "learning_rate": 1.1364413541547933e-06,
2621
+ "loss": 0.0003973785042762756,
2622
+ "step": 18500
2623
+ },
2624
+ {
2625
+ "epoch": 1.902954452195322,
2626
+ "grad_norm": 0.0040567717514932156,
2627
+ "learning_rate": 1.0794483073065088e-06,
2628
+ "loss": 0.00035895496606826784,
2629
+ "step": 18550
2630
+ },
2631
+ {
2632
+ "epoch": 1.9080837094788674,
2633
+ "grad_norm": 0.007941263727843761,
2634
+ "learning_rate": 1.0224552604582241e-06,
2635
+ "loss": 0.00032996714115142824,
2636
+ "step": 18600
2637
+ },
2638
+ {
2639
+ "epoch": 1.913212966762413,
2640
+ "grad_norm": 0.04344106465578079,
2641
+ "learning_rate": 9.654622136099397e-07,
2642
+ "loss": 0.0006199817359447479,
2643
+ "step": 18650
2644
+ },
2645
+ {
2646
+ "epoch": 1.9183422240459582,
2647
+ "grad_norm": 0.0051651448011398315,
2648
+ "learning_rate": 9.084691667616552e-07,
2649
+ "loss": 0.0008288529515266418,
2650
+ "step": 18700
2651
+ },
2652
+ {
2653
+ "epoch": 1.9234714813295035,
2654
+ "grad_norm": 0.007254309486597776,
2655
+ "learning_rate": 8.514761199133706e-07,
2656
+ "loss": 0.00048538297414779663,
2657
+ "step": 18750
2658
+ },
2659
+ {
2660
+ "epoch": 1.9286007386130488,
2661
+ "grad_norm": 0.09651651978492737,
2662
+ "learning_rate": 7.944830730650861e-07,
2663
+ "loss": 0.00031837493181228636,
2664
+ "step": 18800
2665
+ },
2666
+ {
2667
+ "epoch": 1.9337299958965941,
2668
+ "grad_norm": 0.0036208152305334806,
2669
+ "learning_rate": 7.374900262168016e-07,
2670
+ "loss": 0.0003172314167022705,
2671
+ "step": 18850
2672
+ },
2673
+ {
2674
+ "epoch": 1.9388592531801394,
2675
+ "grad_norm": 0.010356809943914413,
2676
+ "learning_rate": 6.804969793685171e-07,
2677
+ "loss": 0.000322052538394928,
2678
+ "step": 18900
2679
+ },
2680
+ {
2681
+ "epoch": 1.9439885104636847,
2682
+ "grad_norm": 0.008326847106218338,
2683
+ "learning_rate": 6.235039325202326e-07,
2684
+ "loss": 0.0007110384106636048,
2685
+ "step": 18950
2686
+ },
2687
+ {
2688
+ "epoch": 1.9491177677472302,
2689
+ "grad_norm": 0.02525205910205841,
2690
+ "learning_rate": 5.66510885671948e-07,
2691
+ "loss": 0.0003109830617904663,
2692
+ "step": 19000
2693
+ },
2694
+ {
2695
+ "epoch": 1.9542470250307755,
2696
+ "grad_norm": 0.01355426199734211,
2697
+ "learning_rate": 5.095178388236636e-07,
2698
+ "loss": 0.0008139577507972717,
2699
+ "step": 19050
2700
+ },
2701
+ {
2702
+ "epoch": 1.959376282314321,
2703
+ "grad_norm": 0.008329730480909348,
2704
+ "learning_rate": 4.52524791975379e-07,
2705
+ "loss": 0.0007415449619293213,
2706
+ "step": 19100
2707
+ },
2708
+ {
2709
+ "epoch": 1.9645055395978663,
2710
+ "grad_norm": 0.0014810786815360188,
2711
+ "learning_rate": 3.955317451270945e-07,
2712
+ "loss": 0.0003853049874305725,
2713
+ "step": 19150
2714
+ },
2715
+ {
2716
+ "epoch": 1.9696347968814116,
2717
+ "grad_norm": 0.03657092899084091,
2718
+ "learning_rate": 3.3853869827881005e-07,
2719
+ "loss": 0.0003694988787174225,
2720
+ "step": 19200
2721
+ },
2722
+ {
2723
+ "epoch": 1.974764054164957,
2724
+ "grad_norm": 0.0059157381765544415,
2725
+ "learning_rate": 2.815456514305255e-07,
2726
+ "loss": 0.0013805073499679565,
2727
+ "step": 19250
2728
+ },
2729
+ {
2730
+ "epoch": 1.9798933114485022,
2731
+ "grad_norm": 0.0027979083824902773,
2732
+ "learning_rate": 2.2455260458224098e-07,
2733
+ "loss": 0.0008595210313796998,
2734
+ "step": 19300
2735
+ },
2736
+ {
2737
+ "epoch": 1.9850225687320475,
2738
+ "grad_norm": 0.009878180921077728,
2739
+ "learning_rate": 1.6755955773395646e-07,
2740
+ "loss": 0.0003620834648609161,
2741
+ "step": 19350
2742
+ },
2743
+ {
2744
+ "epoch": 1.9901518260155928,
2745
+ "grad_norm": 0.004368235357105732,
2746
+ "learning_rate": 1.1056651088567195e-07,
2747
+ "loss": 0.0012495499849319459,
2748
+ "step": 19400
2749
+ },
2750
+ {
2751
+ "epoch": 1.9952810832991383,
2752
+ "grad_norm": 0.025456879287958145,
2753
+ "learning_rate": 5.357346403738744e-08,
2754
+ "loss": 0.0009994500875473023,
2755
+ "step": 19450
2756
+ },
2757
+ {
2758
+ "epoch": 2.0,
2759
+ "eval_biencoder-val_cosine_accuracy@1": 0.929,
2760
+ "eval_biencoder-val_cosine_accuracy@10": 1.0,
2761
+ "eval_biencoder-val_cosine_accuracy@30": 1.0,
2762
+ "eval_biencoder-val_cosine_accuracy@5": 1.0,
2763
+ "eval_biencoder-val_cosine_map@100": 0.9638333333333332,
2764
+ "eval_biencoder-val_cosine_mrr@10": 0.9638333333333332,
2765
+ "eval_biencoder-val_cosine_ndcg@10": 0.9732722934892877,
2766
+ "eval_biencoder-val_cosine_precision@1": 0.929,
2767
+ "eval_biencoder-val_cosine_precision@10": 0.10000000000000002,
2768
+ "eval_biencoder-val_cosine_precision@3": 0.33333333333333326,
2769
+ "eval_biencoder-val_cosine_precision@5": 0.20000000000000004,
2770
+ "eval_biencoder-val_cosine_recall@1": 0.929,
2771
+ "eval_biencoder-val_cosine_recall@10": 1.0,
2772
+ "eval_biencoder-val_cosine_recall@3": 1.0,
2773
+ "eval_biencoder-val_cosine_recall@5": 1.0,
2774
+ "eval_runtime": 0.9214,
2775
+ "eval_samples_per_second": 0.0,
2776
+ "eval_steps_per_second": 0.0,
2777
+ "step": 19496
2778
+ }
2779
+ ],
2780
+ "logging_steps": 50,
2781
+ "max_steps": 19496,
2782
+ "num_input_tokens_seen": 0,
2783
+ "num_train_epochs": 2,
2784
+ "save_steps": 500,
2785
+ "stateful_callbacks": {
2786
+ "TrainerControl": {
2787
+ "args": {
2788
+ "should_epoch_stop": false,
2789
+ "should_evaluate": false,
2790
+ "should_log": false,
2791
+ "should_save": true,
2792
+ "should_training_stop": true
2793
+ },
2794
+ "attributes": {}
2795
+ }
2796
+ },
2797
+ "total_flos": 0.0,
2798
+ "train_batch_size": 64,
2799
+ "trial_name": null,
2800
+ "trial_params": null
2801
+ }
checkpoint-19496/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:809cebba7d249e0e07e26de2f1f2406ef5fa05d92e5b996f08767e6c2f60b42b
3
+ size 5521
config.json ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_cross_attention": false,
3
+ "architectures": [
4
+ "BertModel"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": null,
8
+ "classifier_dropout": null,
9
+ "directionality": "bidi",
10
+ "dtype": "float32",
11
+ "eos_token_id": null,
12
+ "gradient_checkpointing": false,
13
+ "hidden_act": "gelu",
14
+ "hidden_dropout_prob": 0.1,
15
+ "hidden_size": 768,
16
+ "initializer_range": 0.02,
17
+ "intermediate_size": 3072,
18
+ "is_decoder": false,
19
+ "layer_norm_eps": 1e-12,
20
+ "max_position_embeddings": 512,
21
+ "model_type": "bert",
22
+ "num_attention_heads": 12,
23
+ "num_hidden_layers": 12,
24
+ "pad_token_id": 0,
25
+ "pooler_fc_size": 768,
26
+ "pooler_num_attention_heads": 12,
27
+ "pooler_num_fc_layers": 3,
28
+ "pooler_size_per_head": 128,
29
+ "pooler_type": "first_token_transform",
30
+ "position_embedding_type": "absolute",
31
+ "tie_word_embeddings": true,
32
+ "transformers_version": "5.17.0",
33
+ "type_vocab_size": 2,
34
+ "use_cache": false,
35
+ "vocab_size": 501153
36
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "pytorch": "2.14.0+cu130",
4
+ "sentence_transformers": "6.1.0",
5
+ "transformers": "5.17.0"
6
+ },
7
+ "default_prompt_name": null,
8
+ "model_type": "SentenceTransformer",
9
+ "prompts": {
10
+ "document": "",
11
+ "query": ""
12
+ },
13
+ "similarity_fn_name": "cosine"
14
+ }
eval/Information-Retrieval_evaluation_biencoder-val_results.csv ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ epoch,steps,cosine-Accuracy@1,cosine-Accuracy@5,cosine-Accuracy@10,cosine-Accuracy@30,cosine-Precision@1,cosine-Recall@1,cosine-Precision@3,cosine-Recall@3,cosine-Precision@5,cosine-Recall@5,cosine-Precision@10,cosine-Recall@10,cosine-MRR@10,cosine-NDCG@10,cosine-MAP@100
2
+ 1.0,9748,0.929,1.0,1.0,1.0,0.929,0.929,0.33333333333333326,1.0,0.20000000000000004,1.0,0.10000000000000002,1.0,0.9638333333333332,0.9732722934892877,0.9638333333333332
3
+ 2.0,19496,0.929,1.0,1.0,1.0,0.929,0.929,0.33333333333333326,1.0,0.20000000000000004,1.0,0.10000000000000002,1.0,0.9638333333333332,0.9732722934892877,0.9638333333333332
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:880ba175dc3dbb5fee1d6fafe965ccf86f9a8e8971a6250d01f650e46c5f008c
3
+ size 1883730160
modules.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.base.modules.transformer.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.sentence_transformer.modules.pooling.Pooling"
13
+ },
14
+ {
15
+ "idx": 2,
16
+ "name": "2",
17
+ "path": "2_Dense",
18
+ "type": "sentence_transformers.base.modules.dense.Dense"
19
+ },
20
+ {
21
+ "idx": 3,
22
+ "name": "3",
23
+ "path": "3_Normalize",
24
+ "type": "sentence_transformers.base.modules.normalize.Normalize"
25
+ }
26
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "transformer_task": "feature-extraction",
3
+ "modality_config": {
4
+ "text": {
5
+ "method": "forward",
6
+ "method_output_name": "last_hidden_state"
7
+ }
8
+ },
9
+ "module_output_name": "token_embeddings"
10
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:92262b29204f8fdc169a63f9005a0e311a16262cef4d96ecfe2a7ed638662ed3
3
+ size 13632172
tokenizer_config.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "backend": "tokenizers",
3
+ "cls_token": "[CLS]",
4
+ "do_basic_tokenize": true,
5
+ "do_lower_case": false,
6
+ "full_tokenizer_file": null,
7
+ "is_local": false,
8
+ "local_files_only": false,
9
+ "mask_token": "[MASK]",
10
+ "model_max_length": 256,
11
+ "never_split": null,
12
+ "pad_token": "[PAD]",
13
+ "sep_token": "[SEP]",
14
+ "strip_accents": null,
15
+ "tokenize_chinese_chars": true,
16
+ "tokenizer_class": "BertTokenizer",
17
+ "unk_token": "[UNK]"
18
+ }