Hugging Face's logo

x2bee
/
ModernBERT-SimCSE-multitask_v03-beta

Sentence Similarity
sentence-transformers
Safetensors
modernbert
feature-extraction
Generated from Trainer
dataset_size:449904
loss:CosineSimilarityLoss
Eval Results
text-embeddings-inference
Model card Files
xet
Community
CocoRoF commited on
Commit
c1e2c46
·
verified ·
1 Parent(s): 986d42d

Training in progress, step 3500, checkpoint

Browse files
Files changed (17) hide show
  1. last-checkpoint/1_Pooling/config.json +10 -0
  2. last-checkpoint/2_Dense/config.json +1 -0
  3. last-checkpoint/2_Dense/model.safetensors +3 -0
  4. last-checkpoint/README.md +806 -0
  5. last-checkpoint/config.json +47 -0
  6. last-checkpoint/config_sentence_transformers.json +10 -0
  7. last-checkpoint/model.safetensors +3 -0
  8. last-checkpoint/modules.json +20 -0
  9. last-checkpoint/optimizer.pt +3 -0
  10. last-checkpoint/rng_state.pth +3 -0
  11. last-checkpoint/scheduler.pt +3 -0
  12. last-checkpoint/sentence_bert_config.json +4 -0
  13. last-checkpoint/special_tokens_map.json +37 -0
  14. last-checkpoint/tokenizer.json +0 -0
  15. last-checkpoint/tokenizer_config.json +952 -0
  16. last-checkpoint/trainer_state.json +2735 -0
  17. last-checkpoint/training_args.bin +3 -0
last-checkpoint/1_Pooling/config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 768,
3
+ "pooling_mode_cls_token": false,
4
+ "pooling_mode_mean_tokens": true,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": false,
9
+ "include_prompt": true
10
+ }
last-checkpoint/2_Dense/config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"in_features": 768, "out_features": 768, "bias": true, "activation_function": "torch.nn.modules.activation.Tanh"}
last-checkpoint/2_Dense/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:729604283e9a51856832604b97a05348cc48597c78f6bf6728a4779709b0e02d
3
+ size 2362528
last-checkpoint/README.md ADDED
@@ -0,0 +1,806 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - sentence-transformers
4
+ - sentence-similarity
5
+ - feature-extraction
6
+ - generated_from_trainer
7
+ - dataset_size:449904
8
+ - loss:CosineSimilarityLoss
9
+ base_model: x2bee/ModernBERT-SimCSE-multitask_v03
10
+ widget:
11
+ - source_sentence: 우리는 움직이는 동행 우주 정지 좌표계에 비례하여 이동하고 있습니다 ... 약 371km / s에서 별자리 leo
12
+ 쪽으로. "
13
+ sentences:
14
+ - 두 마리의 독수리가 가지에 앉는다.
15
+ - 다른 물체와는 관련이 없는 '정지'는 없다.
16
+ - 소녀는 버스의 열린 문 앞에 서 있다.
17
+ - source_sentence: 숲에는 개들이 있다.
18
+ sentences:
19
+ - 양을 보는 아이들.
20
+ - 여왕의 배우자를 "왕"이라고 부르지 않는 것은 아주 좋은 이유가 있다. 왜냐하면 그들은 왕이 아니기 때문이다.
21
+ - 개들은 숲속에 혼자 있다.
22
+ - source_sentence: '첫째, 두 가지 다른 종류의 대시가 있다는 것을 알아야 합니다 : en 대시와 em 대시.'
23
+ sentences:
24
+ - 그들은 그 물건들을 집 주변에 두고 가거나 집의 정리를 해칠 의도가 없다.
25
+ - 세미콜론은 혼자 있을 수 있는 문장에 참여하는데 사용되지만, 그들의 관계를 강조하기 위해 결합됩니다.
26
+ - 그의 남동생이 지켜보는 동안 집 앞에서 트럼펫을 연주하는 금발의 아이.
27
+ - source_sentence: 한 여성이 생선 껍질을 벗기고 있다.
28
+ sentences:
29
+ - 한 남자가 수영장으로 뛰어들었다.
30
+ - 한 여성이 프라이팬에 노란 혼합물을 부어 넣고 있다.
31
+ - 두 마리의 갈색 개가 눈 속에서 서로 놀고 있다.
32
+ - source_sentence: 버스가 바쁜 길을 따라 운전한다.
33
+ sentences:
34
+ - 우리와 같은 태양계가 은하계 밖에서 존재할 수도 있을 것입니다.
35
+ - 그 여자는 데이트하러 가는 중이다.
36
+ - 녹색 버스가 도로를 따라 내려간다.
37
+ datasets:
38
+ - x2bee/misc_sts_pairs_v2_kor_kosimcse
39
+ pipeline_tag: sentence-similarity
40
+ library_name: sentence-transformers
41
+ metrics:
42
+ - pearson_cosine
43
+ - spearman_cosine
44
+ - pearson_euclidean
45
+ - spearman_euclidean
46
+ - pearson_manhattan
47
+ - spearman_manhattan
48
+ - pearson_dot
49
+ - spearman_dot
50
+ - pearson_max
51
+ - spearman_max
52
+ model-index:
53
+ - name: SentenceTransformer based on x2bee/ModernBERT-SimCSE-multitask_v03
54
+ results:
55
+ - task:
56
+ type: semantic-similarity
57
+ name: Semantic Similarity
58
+ dataset:
59
+ name: sts dev
60
+ type: sts_dev
61
+ metrics:
62
+ - type: pearson_cosine
63
+ value: 0.8319192467999278
64
+ name: Pearson Cosine
65
+ - type: spearman_cosine
66
+ value: 0.8396159085327265
67
+ name: Spearman Cosine
68
+ - type: pearson_euclidean
69
+ value: 0.8198226408074469
70
+ name: Pearson Euclidean
71
+ - type: spearman_euclidean
72
+ value: 0.8285927601564604
73
+ name: Spearman Euclidean
74
+ - type: pearson_manhattan
75
+ value: 0.8199114649719743
76
+ name: Pearson Manhattan
77
+ - type: spearman_manhattan
78
+ value: 0.8295556212626334
79
+ name: Spearman Manhattan
80
+ - type: pearson_dot
81
+ value: 0.7234705763545461
82
+ name: Pearson Dot
83
+ - type: spearman_dot
84
+ value: 0.7094397491074207
85
+ name: Spearman Dot
86
+ - type: pearson_max
87
+ value: 0.8319192467999278
88
+ name: Pearson Max
89
+ - type: spearman_max
90
+ value: 0.8396159085327265
91
+ name: Spearman Max
92
+ ---
93
+
94
+ # SentenceTransformer based on x2bee/ModernBERT-SimCSE-multitask_v03
95
+
96
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [x2bee/ModernBERT-SimCSE-multitask_v03](https://huggingface.co/x2bee/ModernBERT-SimCSE-multitask_v03) on the [misc_sts_pairs_v2_kor_kosimcse](https://huggingface.co/datasets/x2bee/misc_sts_pairs_v2_kor_kosimcse) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
97
+
98
+ ## Model Details
99
+
100
+ ### Model Description
101
+ - **Model Type:** Sentence Transformer
102
+ - **Base model:** [x2bee/ModernBERT-SimCSE-multitask_v03](https://huggingface.co/x2bee/ModernBERT-SimCSE-multitask_v03) <!-- at revision 98466830eabc76ee3f71472c4c40ab10f8bf7831 -->
103
+ - **Maximum Sequence Length:** 512 tokens
104
+ - **Output Dimensionality:** 768 dimensions
105
+ - **Similarity Function:** Cosine Similarity
106
+ - **Training Dataset:**
107
+ - [misc_sts_pairs_v2_kor_kosimcse](https://huggingface.co/datasets/x2bee/misc_sts_pairs_v2_kor_kosimcse)
108
+ <!-- - **Language:** Unknown -->
109
+ <!-- - **License:** Unknown -->
110
+
111
+ ### Model Sources
112
+
113
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
114
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
115
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
116
+
117
+ ### Full Model Architecture
118
+
119
+ ```
120
+ SentenceTransformer(
121
+ (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: ModernBertModel
122
+ (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
123
+ (2): Dense({'in_features': 768, 'out_features': 768, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh'})
124
+ )
125
+ ```
126
+
127
+ ## Usage
128
+
129
+ ### Direct Usage (Sentence Transformers)
130
+
131
+ First install the Sentence Transformers library:
132
+
133
+ ```bash
134
+ pip install -U sentence-transformers
135
+ ```
136
+
137
+ Then you can load this model and run inference.
138
+ ```python
139
+ from sentence_transformers import SentenceTransformer
140
+
141
+ # Download from the 🤗 Hub
142
+ model = SentenceTransformer("x2bee/ModernBERT-SimCSE-multitask_v03-beta")
143
+ # Run inference
144
+ sentences = [
145
+ '버스가 바쁜 길을 따라 운전한다.',
146
+ '녹색 버스가 도로를 따라 내려간다.',
147
+ '그 여자는 데이트하러 가는 중이다.',
148
+ ]
149
+ embeddings = model.encode(sentences)
150
+ print(embeddings.shape)
151
+ # [3, 768]
152
+
153
+ # Get the similarity scores for the embeddings
154
+ similarities = model.similarity(embeddings, embeddings)
155
+ print(similarities.shape)
156
+ # [3, 3]
157
+ ```
158
+
159
+ <!--
160
+ ### Direct Usage (Transformers)
161
+
162
+ <details><summary>Click to see the direct usage in Transformers</summary>
163
+
164
+ </details>
165
+ -->
166
+
167
+ <!--
168
+ ### Downstream Usage (Sentence Transformers)
169
+
170
+ You can finetune this model on your own dataset.
171
+
172
+ <details><summary>Click to expand</summary>
173
+
174
+ </details>
175
+ -->
176
+
177
+ <!--
178
+ ### Out-of-Scope Use
179
+
180
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
181
+ -->
182
+
183
+ ## Evaluation
184
+
185
+ ### Metrics
186
+
187
+ #### Semantic Similarity
188
+
189
+ * Dataset: `sts_dev`
190
+ * Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
191
+
192
+ | Metric | Value |
193
+ |:-------------------|:-----------|
194
+ | pearson_cosine | 0.8319 |
195
+ | spearman_cosine | 0.8396 |
196
+ | pearson_euclidean | 0.8198 |
197
+ | spearman_euclidean | 0.8286 |
198
+ | pearson_manhattan | 0.8199 |
199
+ | spearman_manhattan | 0.8296 |
200
+ | pearson_dot | 0.7235 |
201
+ | spearman_dot | 0.7094 |
202
+ | pearson_max | 0.8319 |
203
+ | **spearman_max** | **0.8396** |
204
+
205
+ <!--
206
+ ## Bias, Risks and Limitations
207
+
208
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
209
+ -->
210
+
211
+ <!--
212
+ ### Recommendations
213
+
214
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
215
+ -->
216
+
217
+ ## Training Details
218
+
219
+ ### Training Dataset
220
+
221
+ #### misc_sts_pairs_v2_kor_kosimcse
222
+
223
+ * Dataset: [misc_sts_pairs_v2_kor_kosimcse](https://huggingface.co/datasets/x2bee/misc_sts_pairs_v2_kor_kosimcse) at [e747415](https://huggingface.co/datasets/x2bee/misc_sts_pairs_v2_kor_kosimcse/tree/e747415cfe9ff51d1c1550b8a07e5014c01dea59)
224
+ * Size: 449,904 training samples
225
+ * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
226
+ * Approximate statistics based on the first 1000 samples:
227
+ | | sentence1 | sentence2 | score |
228
+ |:--------|:---------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------|
229
+ | type | string | string | float |
230
+ | details | <ul><li>min: 6 tokens</li><li>mean: 18.3 tokens</li><li>max: 69 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 18.69 tokens</li><li>max: 66 tokens</li></ul> | <ul><li>min: 0.11</li><li>mean: 0.77</li><li>max: 1.0</li></ul> |
231
+ * Samples:
232
+ | sentence1 | sentence2 | score |
233
+ |:-------------------------------------------------|:-------------------------------------------|:--------------------------------|
234
+ | <code>주홍글씨는 언제 출판되었습니까?</code> | <code>《주홍글씨》는 몇 년에 출판되었습니까?</code> | <code>0.8638778924942017</code> |
235
+ | <code>폴란드에서 빨간색과 흰색은 무엇을 의미합니까?</code> | <code>폴란드 국기의 색상은 무엇입니까?</code> | <code>0.6773715019226074</code> |
236
+ | <code>노르만인들은 방어를 위해 모트와 베일리 성을 어떻게 사용했는가?</code> | <code>11세기에는 어떻게 모트와 베일리 성을 만들었습니까?</code> | <code>0.7460665702819824</code> |
237
+ * Loss: [<code>CosineSimilarityLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters:
238
+ ```json
239
+ {
240
+ "loss_fct": "torch.nn.modules.loss.MSELoss"
241
+ }
242
+ ```
243
+
244
+ ### Evaluation Dataset
245
+
246
+ #### Unnamed Dataset
247
+
248
+
249
+ * Size: 1,500 evaluation samples
250
+ * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
251
+ * Approximate statistics based on the first 1000 samples:
252
+ | | sentence1 | sentence2 | score |
253
+ |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------|
254
+ | type | string | string | float |
255
+ | details | <ul><li>min: 7 tokens</li><li>mean: 20.38 tokens</li><li>max: 52 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 20.52 tokens</li><li>max: 54 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.42</li><li>max: 1.0</li></ul> |
256
+ * Samples:
257
+ | sentence1 | sentence2 | score |
258
+ |:-------------------------------------|:------------------------------------|:------------------|
259
+ | <code>안전모를 가진 한 남자가 춤을 추고 있다.</code> | <code>안전모를 쓴 한 남자가 춤을 추고 있다.</code> | <code>1.0</code> |
260
+ | <code>어린아이가 말을 타고 있다.</code> | <code>아이가 말을 타고 있다.</code> | <code>0.95</code> |
261
+ | <code>한 남자가 뱀에게 쥐를 먹이고 있다.</code> | <code>남자가 뱀에게 쥐를 먹이고 있다.</code> | <code>1.0</code> |
262
+ * Loss: [<code>CosineSimilarityLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters:
263
+ ```json
264
+ {
265
+ "loss_fct": "torch.nn.modules.loss.MSELoss"
266
+ }
267
+ ```
268
+
269
+ ### Training Hyperparameters
270
+ #### Non-Default Hyperparameters
271
+
272
+ - `overwrite_output_dir`: True
273
+ - `eval_strategy`: steps
274
+ - `per_device_train_batch_size`: 16
275
+ - `per_device_eval_batch_size`: 16
276
+ - `gradient_accumulation_steps`: 8
277
+ - `learning_rate`: 8e-05
278
+ - `num_train_epochs`: 2.0
279
+ - `warmup_ratio`: 0.2
280
+ - `push_to_hub`: True
281
+ - `hub_model_id`: x2bee/ModernBERT-SimCSE-multitask_v03-beta
282
+ - `hub_strategy`: checkpoint
283
+ - `batch_sampler`: no_duplicates
284
+
285
+ #### All Hyperparameters
286
+ <details><summary>Click to expand</summary>
287
+
288
+ - `overwrite_output_dir`: True
289
+ - `do_predict`: False
290
+ - `eval_strategy`: steps
291
+ - `prediction_loss_only`: True
292
+ - `per_device_train_batch_size`: 16
293
+ - `per_device_eval_batch_size`: 16
294
+ - `per_gpu_train_batch_size`: None
295
+ - `per_gpu_eval_batch_size`: None
296
+ - `gradient_accumulation_steps`: 8
297
+ - `eval_accumulation_steps`: None
298
+ - `torch_empty_cache_steps`: None
299
+ - `learning_rate`: 8e-05
300
+ - `weight_decay`: 0.0
301
+ - `adam_beta1`: 0.9
302
+ - `adam_beta2`: 0.999
303
+ - `adam_epsilon`: 1e-08
304
+ - `max_grad_norm`: 1.0
305
+ - `num_train_epochs`: 2.0
306
+ - `max_steps`: -1
307
+ - `lr_scheduler_type`: linear
308
+ - `lr_scheduler_kwargs`: {}
309
+ - `warmup_ratio`: 0.2
310
+ - `warmup_steps`: 0
311
+ - `log_level`: passive
312
+ - `log_level_replica`: warning
313
+ - `log_on_each_node`: True
314
+ - `logging_nan_inf_filter`: True
315
+ - `save_safetensors`: True
316
+ - `save_on_each_node`: False
317
+ - `save_only_model`: False
318
+ - `restore_callback_states_from_checkpoint`: False
319
+ - `no_cuda`: False
320
+ - `use_cpu`: False
321
+ - `use_mps_device`: False
322
+ - `seed`: 42
323
+ - `data_seed`: None
324
+ - `jit_mode_eval`: False
325
+ - `use_ipex`: False
326
+ - `bf16`: False
327
+ - `fp16`: False
328
+ - `fp16_opt_level`: O1
329
+ - `half_precision_backend`: auto
330
+ - `bf16_full_eval`: False
331
+ - `fp16_full_eval`: False
332
+ - `tf32`: None
333
+ - `local_rank`: 0
334
+ - `ddp_backend`: None
335
+ - `tpu_num_cores`: None
336
+ - `tpu_metrics_debug`: False
337
+ - `debug`: []
338
+ - `dataloader_drop_last`: True
339
+ - `dataloader_num_workers`: 0
340
+ - `dataloader_prefetch_factor`: None
341
+ - `past_index`: -1
342
+ - `disable_tqdm`: False
343
+ - `remove_unused_columns`: True
344
+ - `label_names`: None
345
+ - `load_best_model_at_end`: False
346
+ - `ignore_data_skip`: False
347
+ - `fsdp`: []
348
+ - `fsdp_min_num_params`: 0
349
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
350
+ - `fsdp_transformer_layer_cls_to_wrap`: None
351
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
352
+ - `deepspeed`: None
353
+ - `label_smoothing_factor`: 0.0
354
+ - `optim`: adamw_torch
355
+ - `optim_args`: None
356
+ - `adafactor`: False
357
+ - `group_by_length`: False
358
+ - `length_column_name`: length
359
+ - `ddp_find_unused_parameters`: None
360
+ - `ddp_bucket_cap_mb`: None
361
+ - `ddp_broadcast_buffers`: False
362
+ - `dataloader_pin_memory`: True
363
+ - `dataloader_persistent_workers`: False
364
+ - `skip_memory_metrics`: True
365
+ - `use_legacy_prediction_loop`: False
366
+ - `push_to_hub`: True
367
+ - `resume_from_checkpoint`: None
368
+ - `hub_model_id`: x2bee/ModernBERT-SimCSE-multitask_v03-beta
369
+ - `hub_strategy`: checkpoint
370
+ - `hub_private_repo`: None
371
+ - `hub_always_push`: False
372
+ - `gradient_checkpointing`: False
373
+ - `gradient_checkpointing_kwargs`: None
374
+ - `include_inputs_for_metrics`: False
375
+ - `include_for_metrics`: []
376
+ - `eval_do_concat_batches`: True
377
+ - `fp16_backend`: auto
378
+ - `push_to_hub_model_id`: None
379
+ - `push_to_hub_organization`: None
380
+ - `mp_parameters`:
381
+ - `auto_find_batch_size`: False
382
+ - `full_determinism`: False
383
+ - `torchdynamo`: None
384
+ - `ray_scope`: last
385
+ - `ddp_timeout`: 1800
386
+ - `torch_compile`: False
387
+ - `torch_compile_backend`: None
388
+ - `torch_compile_mode`: None
389
+ - `dispatch_batches`: None
390
+ - `split_batches`: None
391
+ - `include_tokens_per_second`: False
392
+ - `include_num_input_tokens_seen`: False
393
+ - `neftune_noise_alpha`: None
394
+ - `optim_target_modules`: None
395
+ - `batch_eval_metrics`: False
396
+ - `eval_on_start`: False
397
+ - `use_liger_kernel`: False
398
+ - `eval_use_gather_object`: False
399
+ - `average_tokens_across_devices`: False
400
+ - `prompts`: None
401
+ - `batch_sampler`: no_duplicates
402
+ - `multi_dataset_batch_sampler`: proportional
403
+
404
+ </details>
405
+
406
+ ### Training Logs
407
+ <details><summary>Click to expand</summary>
408
+
409
+ | Epoch | Step | Training Loss | Validation Loss | sts_dev_spearman_max |
410
+ |:------:|:----:|:-------------:|:---------------:|:--------------------:|
411
+ | 0.0028 | 10 | 0.0216 | - | - |
412
+ | 0.0057 | 20 | 0.0204 | - | - |
413
+ | 0.0085 | 30 | 0.0194 | - | - |
414
+ | 0.0114 | 40 | 0.0195 | - | - |
415
+ | 0.0142 | 50 | 0.0182 | - | - |
416
+ | 0.0171 | 60 | 0.0161 | - | - |
417
+ | 0.0199 | 70 | 0.015 | - | - |
418
+ | 0.0228 | 80 | 0.0153 | - | - |
419
+ | 0.0256 | 90 | 0.0137 | - | - |
420
+ | 0.0285 | 100 | 0.014 | - | - |
421
+ | 0.0313 | 110 | 0.0122 | - | - |
422
+ | 0.0341 | 120 | 0.0114 | - | - |
423
+ | 0.0370 | 130 | 0.0109 | - | - |
424
+ | 0.0398 | 140 | 0.0097 | - | - |
425
+ | 0.0427 | 150 | 0.0085 | - | - |
426
+ | 0.0455 | 160 | 0.0084 | - | - |
427
+ | 0.0484 | 170 | 0.0083 | - | - |
428
+ | 0.0512 | 180 | 0.0078 | - | - |
429
+ | 0.0541 | 190 | 0.008 | - | - |
430
+ | 0.0569 | 200 | 0.0073 | - | - |
431
+ | 0.0597 | 210 | 0.0079 | - | - |
432
+ | 0.0626 | 220 | 0.0073 | - | - |
433
+ | 0.0654 | 230 | 0.0079 | - | - |
434
+ | 0.0683 | 240 | 0.0068 | - | - |
435
+ | 0.0711 | 250 | 0.0068 | 0.0333 | 0.8229 |
436
+ | 0.0740 | 260 | 0.0073 | - | - |
437
+ | 0.0768 | 270 | 0.0077 | - | - |
438
+ | 0.0797 | 280 | 0.0067 | - | - |
439
+ | 0.0825 | 290 | 0.007 | - | - |
440
+ | 0.0854 | 300 | 0.0065 | - | - |
441
+ | 0.0882 | 310 | 0.0072 | - | - |
442
+ | 0.0910 | 320 | 0.0068 | - | - |
443
+ | 0.0939 | 330 | 0.0064 | - | - |
444
+ | 0.0967 | 340 | 0.0074 | - | - |
445
+ | 0.0996 | 350 | 0.0071 | - | - |
446
+ | 0.1024 | 360 | 0.0065 | - | - |
447
+ | 0.1053 | 370 | 0.0067 | - | - |
448
+ | 0.1081 | 380 | 0.0063 | - | - |
449
+ | 0.1110 | 390 | 0.0062 | - | - |
450
+ | 0.1138 | 400 | 0.0068 | - | - |
451
+ | 0.1166 | 410 | 0.0064 | - | - |
452
+ | 0.1195 | 420 | 0.0064 | - | - |
453
+ | 0.1223 | 430 | 0.0064 | - | - |
454
+ | 0.1252 | 440 | 0.0074 | - | - |
455
+ | 0.1280 | 450 | 0.0069 | - | - |
456
+ | 0.1309 | 460 | 0.0065 | - | - |
457
+ | 0.1337 | 470 | 0.0067 | - | - |
458
+ | 0.1366 | 480 | 0.0068 | - | - |
459
+ | 0.1394 | 490 | 0.0057 | - | - |
460
+ | 0.1423 | 500 | 0.0065 | 0.0343 | 0.8284 |
461
+ | 0.1451 | 510 | 0.0069 | - | - |
462
+ | 0.1479 | 520 | 0.0068 | - | - |
463
+ | 0.1508 | 530 | 0.0065 | - | - |
464
+ | 0.1536 | 540 | 0.0065 | - | - |
465
+ | 0.1565 | 550 | 0.0063 | - | - |
466
+ | 0.1593 | 560 | 0.0058 | - | - |
467
+ | 0.1622 | 570 | 0.0064 | - | - |
468
+ | 0.1650 | 580 | 0.0062 | - | - |
469
+ | 0.1679 | 590 | 0.0061 | - | - |
470
+ | 0.1707 | 600 | 0.0062 | - | - |
471
+ | 0.1735 | 610 | 0.0057 | - | - |
472
+ | 0.1764 | 620 | 0.0066 | - | - |
473
+ | 0.1792 | 630 | 0.0061 | - | - |
474
+ | 0.1821 | 640 | 0.0054 | - | - |
475
+ | 0.1849 | 650 | 0.0066 | - | - |
476
+ | 0.1878 | 660 | 0.0059 | - | - |
477
+ | 0.1906 | 670 | 0.0063 | - | - |
478
+ | 0.1935 | 680 | 0.0065 | - | - |
479
+ | 0.1963 | 690 | 0.0065 | - | - |
480
+ | 0.1992 | 700 | 0.0058 | - | - |
481
+ | 0.2020 | 710 | 0.006 | - | - |
482
+ | 0.2048 | 720 | 0.0062 | - | - |
483
+ | 0.2077 | 730 | 0.0058 | - | - |
484
+ | 0.2105 | 740 | 0.0058 | - | - |
485
+ | 0.2134 | 750 | 0.0056 | 0.0356 | 0.8302 |
486
+ | 0.2162 | 760 | 0.0067 | - | - |
487
+ | 0.2191 | 770 | 0.0063 | - | - |
488
+ | 0.2219 | 780 | 0.0063 | - | - |
489
+ | 0.2248 | 790 | 0.0063 | - | - |
490
+ | 0.2276 | 800 | 0.0056 | - | - |
491
+ | 0.2304 | 810 | 0.0058 | - | - |
492
+ | 0.2333 | 820 | 0.0053 | - | - |
493
+ | 0.2361 | 830 | 0.0057 | - | - |
494
+ | 0.2390 | 840 | 0.0055 | - | - |
495
+ | 0.2418 | 850 | 0.0054 | - | - |
496
+ | 0.2447 | 860 | 0.0065 | - | - |
497
+ | 0.2475 | 870 | 0.0054 | - | - |
498
+ | 0.2504 | 880 | 0.0051 | - | - |
499
+ | 0.2532 | 890 | 0.0057 | - | - |
500
+ | 0.2561 | 900 | 0.0056 | - | - |
501
+ | 0.2589 | 910 | 0.0055 | - | - |
502
+ | 0.2617 | 920 | 0.0051 | - | - |
503
+ | 0.2646 | 930 | 0.0055 | - | - |
504
+ | 0.2674 | 940 | 0.0059 | - | - |
505
+ | 0.2703 | 950 | 0.005 | - | - |
506
+ | 0.2731 | 960 | 0.0058 | - | - |
507
+ | 0.2760 | 970 | 0.005 | - | - |
508
+ | 0.2788 | 980 | 0.0055 | - | - |
509
+ | 0.2817 | 990 | 0.0054 | - | - |
510
+ | 0.2845 | 1000 | 0.0055 | 0.0360 | 0.8319 |
511
+ | 0.2874 | 1010 | 0.0059 | - | - |
512
+ | 0.2902 | 1020 | 0.0049 | - | - |
513
+ | 0.2930 | 1030 | 0.0052 | - | - |
514
+ | 0.2959 | 1040 | 0.0051 | - | - |
515
+ | 0.2987 | 1050 | 0.006 | - | - |
516
+ | 0.3016 | 1060 | 0.0048 | - | - |
517
+ | 0.3044 | 1070 | 0.0055 | - | - |
518
+ | 0.3073 | 1080 | 0.0052 | - | - |
519
+ | 0.3101 | 1090 | 0.0051 | - | - |
520
+ | 0.3130 | 1100 | 0.0051 | - | - |
521
+ | 0.3158 | 1110 | 0.005 | - | - |
522
+ | 0.3186 | 1120 | 0.0054 | - | - |
523
+ | 0.3215 | 1130 | 0.0051 | - | - |
524
+ | 0.3243 | 1140 | 0.0054 | - | - |
525
+ | 0.3272 | 1150 | 0.0056 | - | - |
526
+ | 0.3300 | 1160 | 0.0053 | - | - |
527
+ | 0.3329 | 1170 | 0.0052 | - | - |
528
+ | 0.3357 | 1180 | 0.0051 | - | - |
529
+ | 0.3386 | 1190 | 0.0051 | - | - |
530
+ | 0.3414 | 1200 | 0.0048 | - | - |
531
+ | 0.3443 | 1210 | 0.005 | - | - |
532
+ | 0.3471 | 1220 | 0.0055 | - | - |
533
+ | 0.3499 | 1230 | 0.0049 | - | - |
534
+ | 0.3528 | 1240 | 0.0053 | - | - |
535
+ | 0.3556 | 1250 | 0.0052 | 0.0364 | 0.8330 |
536
+ | 0.3585 | 1260 | 0.0051 | - | - |
537
+ | 0.3613 | 1270 | 0.005 | - | - |
538
+ | 0.3642 | 1280 | 0.005 | - | - |
539
+ | 0.3670 | 1290 | 0.0045 | - | - |
540
+ | 0.3699 | 1300 | 0.0055 | - | - |
541
+ | 0.3727 | 1310 | 0.0049 | - | - |
542
+ | 0.3755 | 1320 | 0.0049 | - | - |
543
+ | 0.3784 | 1330 | 0.0053 | - | - |
544
+ | 0.3812 | 1340 | 0.005 | - | - |
545
+ | 0.3841 | 1350 | 0.0048 | - | - |
546
+ | 0.3869 | 1360 | 0.0049 | - | - |
547
+ | 0.3898 | 1370 | 0.0046 | - | - |
548
+ | 0.3926 | 1380 | 0.0049 | - | - |
549
+ | 0.3955 | 1390 | 0.0052 | - | - |
550
+ | 0.3983 | 1400 | 0.005 | - | - |
551
+ | 0.4012 | 1410 | 0.0052 | - | - |
552
+ | 0.4040 | 1420 | 0.0052 | - | - |
553
+ | 0.4068 | 1430 | 0.0045 | - | - |
554
+ | 0.4097 | 1440 | 0.0046 | - | - |
555
+ | 0.4125 | 1450 | 0.0056 | - | - |
556
+ | 0.4154 | 1460 | 0.0056 | - | - |
557
+ | 0.4182 | 1470 | 0.005 | - | - |
558
+ | 0.4211 | 1480 | 0.0051 | - | - |
559
+ | 0.4239 | 1490 | 0.0049 | - | - |
560
+ | 0.4268 | 1500 | 0.0048 | 0.0374 | 0.8334 |
561
+ | 0.4296 | 1510 | 0.0053 | - | - |
562
+ | 0.4324 | 1520 | 0.0054 | - | - |
563
+ | 0.4353 | 1530 | 0.0048 | - | - |
564
+ | 0.4381 | 1540 | 0.005 | - | - |
565
+ | 0.4410 | 1550 | 0.0045 | - | - |
566
+ | 0.4438 | 1560 | 0.0046 | - | - |
567
+ | 0.4467 | 1570 | 0.0045 | - | - |
568
+ | 0.4495 | 1580 | 0.0049 | - | - |
569
+ | 0.4524 | 1590 | 0.0048 | - | - |
570
+ | 0.4552 | 1600 | 0.005 | - | - |
571
+ | 0.4581 | 1610 | 0.0045 | - | - |
572
+ | 0.4609 | 1620 | 0.0049 | - | - |
573
+ | 0.4637 | 1630 | 0.0044 | - | - |
574
+ | 0.4666 | 1640 | 0.0048 | - | - |
575
+ | 0.4694 | 1650 | 0.0049 | - | - |
576
+ | 0.4723 | 1660 | 0.0048 | - | - |
577
+ | 0.4751 | 1670 | 0.0051 | - | - |
578
+ | 0.4780 | 1680 | 0.0047 | - | - |
579
+ | 0.4808 | 1690 | 0.0048 | - | - |
580
+ | 0.4837 | 1700 | 0.0047 | - | - |
581
+ | 0.4865 | 1710 | 0.0044 | - | - |
582
+ | 0.4893 | 1720 | 0.0049 | - | - |
583
+ | 0.4922 | 1730 | 0.0049 | - | - |
584
+ | 0.4950 | 1740 | 0.0051 | - | - |
585
+ | 0.4979 | 1750 | 0.0043 | 0.0392 | 0.8352 |
586
+ | 0.5007 | 1760 | 0.0043 | - | - |
587
+ | 0.5036 | 1770 | 0.0045 | - | - |
588
+ | 0.5064 | 1780 | 0.0046 | - | - |
589
+ | 0.5093 | 1790 | 0.0042 | - | - |
590
+ | 0.5121 | 1800 | 0.0047 | - | - |
591
+ | 0.5150 | 1810 | 0.0047 | - | - |
592
+ | 0.5178 | 1820 | 0.0046 | - | - |
593
+ | 0.5206 | 1830 | 0.0044 | - | - |
594
+ | 0.5235 | 1840 | 0.0046 | - | - |
595
+ | 0.5263 | 1850 | 0.0047 | - | - |
596
+ | 0.5292 | 1860 | 0.0044 | - | - |
597
+ | 0.5320 | 1870 | 0.0047 | - | - |
598
+ | 0.5349 | 1880 | 0.0049 | - | - |
599
+ | 0.5377 | 1890 | 0.0049 | - | - |
600
+ | 0.5406 | 1900 | 0.0047 | - | - |
601
+ | 0.5434 | 1910 | 0.0045 | - | - |
602
+ | 0.5462 | 1920 | 0.0044 | - | - |
603
+ | 0.5491 | 1930 | 0.0048 | - | - |
604
+ | 0.5519 | 1940 | 0.0041 | - | - |
605
+ | 0.5548 | 1950 | 0.004 | - | - |
606
+ | 0.5576 | 1960 | 0.0048 | - | - |
607
+ | 0.5605 | 1970 | 0.0042 | - | - |
608
+ | 0.5633 | 1980 | 0.0048 | - | - |
609
+ | 0.5662 | 1990 | 0.0045 | - | - |
610
+ | 0.5690 | 2000 | 0.0043 | 0.0375 | 0.8359 |
611
+ | 0.5719 | 2010 | 0.005 | - | - |
612
+ | 0.5747 | 2020 | 0.0049 | - | - |
613
+ | 0.5775 | 2030 | 0.0044 | - | - |
614
+ | 0.5804 | 2040 | 0.0045 | - | - |
615
+ | 0.5832 | 2050 | 0.0043 | - | - |
616
+ | 0.5861 | 2060 | 0.0045 | - | - |
617
+ | 0.5889 | 2070 | 0.004 | - | - |
618
+ | 0.5918 | 2080 | 0.0042 | - | - |
619
+ | 0.5946 | 2090 | 0.0044 | - | - |
620
+ | 0.5975 | 2100 | 0.0043 | - | - |
621
+ | 0.6003 | 2110 | 0.0041 | - | - |
622
+ | 0.6032 | 2120 | 0.0046 | - | - |
623
+ | 0.6060 | 2130 | 0.0048 | - | - |
624
+ | 0.6088 | 2140 | 0.0048 | - | - |
625
+ | 0.6117 | 2150 | 0.0041 | - | - |
626
+ | 0.6145 | 2160 | 0.0044 | - | - |
627
+ | 0.6174 | 2170 | 0.0045 | - | - |
628
+ | 0.6202 | 2180 | 0.0044 | - | - |
629
+ | 0.6231 | 2190 | 0.0044 | - | - |
630
+ | 0.6259 | 2200 | 0.0046 | - | - |
631
+ | 0.6288 | 2210 | 0.0048 | - | - |
632
+ | 0.6316 | 2220 | 0.0045 | - | - |
633
+ | 0.6344 | 2230 | 0.004 | - | - |
634
+ | 0.6373 | 2240 | 0.0041 | - | - |
635
+ | 0.6401 | 2250 | 0.0044 | 0.0391 | 0.8369 |
636
+ | 0.6430 | 2260 | 0.0044 | - | - |
637
+ | 0.6458 | 2270 | 0.0045 | - | - |
638
+ | 0.6487 | 2280 | 0.0041 | - | - |
639
+ | 0.6515 | 2290 | 0.0042 | - | - |
640
+ | 0.6544 | 2300 | 0.0043 | - | - |
641
+ | 0.6572 | 2310 | 0.004 | - | - |
642
+ | 0.6601 | 2320 | 0.0042 | - | - |
643
+ | 0.6629 | 2330 | 0.0041 | - | - |
644
+ | 0.6657 | 2340 | 0.0045 | - | - |
645
+ | 0.6686 | 2350 | 0.0045 | - | - |
646
+ | 0.6714 | 2360 | 0.0042 | - | - |
647
+ | 0.6743 | 2370 | 0.0045 | - | - |
648
+ | 0.6771 | 2380 | 0.0044 | - | - |
649
+ | 0.6800 | 2390 | 0.0044 | - | - |
650
+ | 0.6828 | 2400 | 0.0041 | - | - |
651
+ | 0.6857 | 2410 | 0.0045 | - | - |
652
+ | 0.6885 | 2420 | 0.0046 | - | - |
653
+ | 0.6913 | 2430 | 0.0041 | - | - |
654
+ | 0.6942 | 2440 | 0.0048 | - | - |
655
+ | 0.6970 | 2450 | 0.0041 | - | - |
656
+ | 0.6999 | 2460 | 0.0043 | - | - |
657
+ | 0.7027 | 2470 | 0.0043 | - | - |
658
+ | 0.7056 | 2480 | 0.0037 | - | - |
659
+ | 0.7084 | 2490 | 0.0042 | - | - |
660
+ | 0.7113 | 2500 | 0.0043 | 0.0405 | 0.8365 |
661
+ | 0.7141 | 2510 | 0.0045 | - | - |
662
+ | 0.7170 | 2520 | 0.0044 | - | - |
663
+ | 0.7198 | 2530 | 0.0042 | - | - |
664
+ | 0.7226 | 2540 | 0.0042 | - | - |
665
+ | 0.7255 | 2550 | 0.0041 | - | - |
666
+ | 0.7283 | 2560 | 0.0042 | - | - |
667
+ | 0.7312 | 2570 | 0.0041 | - | - |
668
+ | 0.7340 | 2580 | 0.0042 | - | - |
669
+ | 0.7369 | 2590 | 0.0041 | - | - |
670
+ | 0.7397 | 2600 | 0.0047 | - | - |
671
+ | 0.7426 | 2610 | 0.0038 | - | - |
672
+ | 0.7454 | 2620 | 0.0041 | - | - |
673
+ | 0.7482 | 2630 | 0.0042 | - | - |
674
+ | 0.7511 | 2640 | 0.0042 | - | - |
675
+ | 0.7539 | 2650 | 0.0042 | - | - |
676
+ | 0.7568 | 2660 | 0.0041 | - | - |
677
+ | 0.7596 | 2670 | 0.0042 | - | - |
678
+ | 0.7625 | 2680 | 0.0044 | - | - |
679
+ | 0.7653 | 2690 | 0.0039 | - | - |
680
+ | 0.7682 | 2700 | 0.0037 | - | - |
681
+ | 0.7710 | 2710 | 0.0044 | - | - |
682
+ | 0.7739 | 2720 | 0.0043 | - | - |
683
+ | 0.7767 | 2730 | 0.0042 | - | - |
684
+ | 0.7795 | 2740 | 0.0041 | - | - |
685
+ | 0.7824 | 2750 | 0.0039 | 0.0387 | 0.8376 |
686
+ | 0.7852 | 2760 | 0.0047 | - | - |
687
+ | 0.7881 | 2770 | 0.004 | - | - |
688
+ | 0.7909 | 2780 | 0.0039 | - | - |
689
+ | 0.7938 | 2790 | 0.0039 | - | - |
690
+ | 0.7966 | 2800 | 0.0039 | - | - |
691
+ | 0.7995 | 2810 | 0.0039 | - | - |
692
+ | 0.8023 | 2820 | 0.0039 | - | - |
693
+ | 0.8051 | 2830 | 0.0041 | - | - |
694
+ | 0.8080 | 2840 | 0.0037 | - | - |
695
+ | 0.8108 | 2850 | 0.0044 | - | - |
696
+ | 0.8137 | 2860 | 0.0043 | - | - |
697
+ | 0.8165 | 2870 | 0.0041 | - | - |
698
+ | 0.8194 | 2880 | 0.0043 | - | - |
699
+ | 0.8222 | 2890 | 0.0039 | - | - |
700
+ | 0.8251 | 2900 | 0.0041 | - | - |
701
+ | 0.8279 | 2910 | 0.0044 | - | - |
702
+ | 0.8308 | 2920 | 0.004 | - | - |
703
+ | 0.8336 | 2930 | 0.0042 | - | - |
704
+ | 0.8364 | 2940 | 0.0039 | - | - |
705
+ | 0.8393 | 2950 | 0.004 | - | - |
706
+ | 0.8421 | 2960 | 0.0042 | - | - |
707
+ | 0.8450 | 2970 | 0.004 | - | - |
708
+ | 0.8478 | 2980 | 0.0039 | - | - |
709
+ | 0.8507 | 2990 | 0.0037 | - | - |
710
+ | 0.8535 | 3000 | 0.0039 | 0.0386 | 0.8386 |
711
+ | 0.8564 | 3010 | 0.0041 | - | - |
712
+ | 0.8592 | 3020 | 0.0043 | - | - |
713
+ | 0.8621 | 3030 | 0.0041 | - | - |
714
+ | 0.8649 | 3040 | 0.0041 | - | - |
715
+ | 0.8677 | 3050 | 0.0043 | - | - |
716
+ | 0.8706 | 3060 | 0.0042 | - | - |
717
+ | 0.8734 | 3070 | 0.0039 | - | - |
718
+ | 0.8763 | 3080 | 0.004 | - | - |
719
+ | 0.8791 | 3090 | 0.0039 | - | - |
720
+ | 0.8820 | 3100 | 0.0039 | - | - |
721
+ | 0.8848 | 3110 | 0.004 | - | - |
722
+ | 0.8877 | 3120 | 0.0039 | - | - |
723
+ | 0.8905 | 3130 | 0.0038 | - | - |
724
+ | 0.8933 | 3140 | 0.0036 | - | - |
725
+ | 0.8962 | 3150 | 0.0039 | - | - |
726
+ | 0.8990 | 3160 | 0.0039 | - | - |
727
+ | 0.9019 | 3170 | 0.0038 | - | - |
728
+ | 0.9047 | 3180 | 0.0039 | - | - |
729
+ | 0.9076 | 3190 | 0.0041 | - | - |
730
+ | 0.9104 | 3200 | 0.004 | - | - |
731
+ | 0.9133 | 3210 | 0.0041 | - | - |
732
+ | 0.9161 | 3220 | 0.0042 | - | - |
733
+ | 0.9190 | 3230 | 0.004 | - | - |
734
+ | 0.9218 | 3240 | 0.0041 | - | - |
735
+ | 0.9246 | 3250 | 0.0041 | 0.0420 | 0.8408 |
736
+ | 0.9275 | 3260 | 0.0041 | - | - |
737
+ | 0.9303 | 3270 | 0.004 | - | - |
738
+ | 0.9332 | 3280 | 0.0042 | - | - |
739
+ | 0.9360 | 3290 | 0.004 | - | - |
740
+ | 0.9389 | 3300 | 0.0037 | - | - |
741
+ | 0.9417 | 3310 | 0.0038 | - | - |
742
+ | 0.9446 | 3320 | 0.0039 | - | - |
743
+ | 0.9474 | 3330 | 0.004 | - | - |
744
+ | 0.9502 | 3340 | 0.0037 | - | - |
745
+ | 0.9531 | 3350 | 0.0038 | - | - |
746
+ | 0.9559 | 3360 | 0.0037 | - | - |
747
+ | 0.9588 | 3370 | 0.0042 | - | - |
748
+ | 0.9616 | 3380 | 0.0042 | - | - |
749
+ | 0.9645 | 3390 | 0.0042 | - | - |
750
+ | 0.9673 | 3400 | 0.0037 | - | - |
751
+ | 0.9702 | 3410 | 0.0038 | - | - |
752
+ | 0.9730 | 3420 | 0.0039 | - | - |
753
+ | 0.9759 | 3430 | 0.0038 | - | - |
754
+ | 0.9787 | 3440 | 0.0041 | - | - |
755
+ | 0.9815 | 3450 | 0.004 | - | - |
756
+ | 0.9844 | 3460 | 0.0039 | - | - |
757
+ | 0.9872 | 3470 | 0.0036 | - | - |
758
+ | 0.9901 | 3480 | 0.0037 | - | - |
759
+ | 0.9929 | 3490 | 0.0039 | - | - |
760
+ | 0.9958 | 3500 | 0.0036 | 0.0403 | 0.8396 |
761
+
762
+ </details>
763
+
764
+ ### Framework Versions
765
+ - Python: 3.11.10
766
+ - Sentence Transformers: 3.3.1
767
+ - Transformers: 4.48.0.dev0
768
+ - PyTorch: 2.5.1+cu121
769
+ - Accelerate: 1.1.0
770
+ - Datasets: 3.1.0
771
+ - Tokenizers: 0.21.0
772
+
773
+ ## Citation
774
+
775
+ ### BibTeX
776
+
777
+ #### Sentence Transformers
778
+ ```bibtex
779
+ @inproceedings{reimers-2019-sentence-bert,
780
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
781
+ author = "Reimers, Nils and Gurevych, Iryna",
782
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
783
+ month = "11",
784
+ year = "2019",
785
+ publisher = "Association for Computational Linguistics",
786
+ url = "https://arxiv.org/abs/1908.10084",
787
+ }
788
+ ```
789
+
790
+ <!--
791
+ ## Glossary
792
+
793
+ *Clearly define terms in order to be accessible across audiences.*
794
+ -->
795
+
796
+ <!--
797
+ ## Model Card Authors
798
+
799
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
800
+ -->
801
+
802
+ <!--
803
+ ## Model Card Contact
804
+
805
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
806
+ -->
last-checkpoint/config.json ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "x2bee/ModernBERT-SimCSE-multitask_v03",
3
+ "architectures": [
4
+ "ModernBertModel"
5
+ ],
6
+ "attention_bias": false,
7
+ "attention_dropout": 0.0,
8
+ "bos_token_id": 50281,
9
+ "classifier_activation": "gelu",
10
+ "classifier_bias": false,
11
+ "classifier_dropout": 0.0,
12
+ "classifier_pooling": "mean",
13
+ "cls_token_id": 50281,
14
+ "decoder_bias": true,
15
+ "deterministic_flash_attn": false,
16
+ "embedding_dropout": 0.0,
17
+ "eos_token_id": 50282,
18
+ "global_attn_every_n_layers": 3,
19
+ "global_rope_theta": 160000.0,
20
+ "gradient_checkpointing": false,
21
+ "hidden_activation": "gelu",
22
+ "hidden_size": 768,
23
+ "initializer_cutoff_factor": 2.0,
24
+ "initializer_range": 0.02,
25
+ "intermediate_size": 1152,
26
+ "layer_norm_eps": 1e-05,
27
+ "local_attention": 128,
28
+ "local_rope_theta": 10000.0,
29
+ "max_position_embeddings": 8192,
30
+ "mlp_bias": false,
31
+ "mlp_dropout": 0.0,
32
+ "model_type": "modernbert",
33
+ "norm_bias": false,
34
+ "norm_eps": 1e-05,
35
+ "num_attention_heads": 12,
36
+ "num_hidden_layers": 22,
37
+ "pad_token_id": 50283,
38
+ "position_embedding_type": "absolute",
39
+ "reference_compile": false,
40
+ "repad_logits_with_grad": false,
41
+ "sep_token_id": 50282,
42
+ "sparse_pred_ignore_index": -100,
43
+ "sparse_prediction": false,
44
+ "torch_dtype": "float32",
45
+ "transformers_version": "4.48.0.dev0",
46
+ "vocab_size": 95663
47
+ }
last-checkpoint/config_sentence_transformers.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "3.3.1",
4
+ "transformers": "4.48.0.dev0",
5
+ "pytorch": "2.5.1+cu121"
6
+ },
7
+ "prompts": {},
8
+ "default_prompt_name": null,
9
+ "similarity_fn_name": "cosine"
10
+ }
last-checkpoint/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:48b00787cef5315a1495cd29268addc20d50eba1fb3854c06b8fdf1aac6f3baf
3
+ size 735216376
last-checkpoint/modules.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ },
14
+ {
15
+ "idx": 2,
16
+ "name": "2",
17
+ "path": "2_Dense",
18
+ "type": "sentence_transformers.models.Dense"
19
+ }
20
+ ]
last-checkpoint/optimizer.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a301ee96f1a476c94b8ab6197b86c4489d123886648c1f929b846daf287fbe03
3
+ size 1475248442
last-checkpoint/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fa62fd12355b012a09afe1496b5d8596451c15a706eac24818bf03a9e9fc4738
3
+ size 14244
last-checkpoint/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d54f6f08075be4f195442b6f0693194d0b06b0ee35f1a58d982e2b0ffc75eeab
3
+ size 1000
last-checkpoint/sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 512,
3
+ "do_lower_case": false
4
+ }
last-checkpoint/special_tokens_map.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": {
3
+ "content": "[CLS]",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "mask_token": {
10
+ "content": "[MASK]",
11
+ "lstrip": true,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "[PAD]",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "sep_token": {
24
+ "content": "[SEP]",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "unk_token": {
31
+ "content": "[UNK]",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ }
37
+ }
last-checkpoint/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
last-checkpoint/tokenizer_config.json ADDED
@@ -0,0 +1,952 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "|||IP_ADDRESS|||",
5
+ "lstrip": false,
6
+ "normalized": true,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": false
10
+ },
11
+ "1": {
12
+ "content": "<|padding|>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "50254": {
20
+ "content": " ",
21
+ "lstrip": false,
22
+ "normalized": true,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": false
26
+ },
27
+ "50255": {
28
+ "content": " ",
29
+ "lstrip": false,
30
+ "normalized": true,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": false
34
+ },
35
+ "50256": {
36
+ "content": " ",
37
+ "lstrip": false,
38
+ "normalized": true,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": false
42
+ },
43
+ "50257": {
44
+ "content": " ",
45
+ "lstrip": false,
46
+ "normalized": true,
47
+ "rstrip": false,
48
+ "single_word": false,
49
+ "special": false
50
+ },
51
+ "50258": {
52
+ "content": " ",
53
+ "lstrip": false,
54
+ "normalized": true,
55
+ "rstrip": false,
56
+ "single_word": false,
57
+ "special": false
58
+ },
59
+ "50259": {
60
+ "content": " ",
61
+ "lstrip": false,
62
+ "normalized": true,
63
+ "rstrip": false,
64
+ "single_word": false,
65
+ "special": false
66
+ },
67
+ "50260": {
68
+ "content": " ",
69
+ "lstrip": false,
70
+ "normalized": true,
71
+ "rstrip": false,
72
+ "single_word": false,
73
+ "special": false
74
+ },
75
+ "50261": {
76
+ "content": " ",
77
+ "lstrip": false,
78
+ "normalized": true,
79
+ "rstrip": false,
80
+ "single_word": false,
81
+ "special": false
82
+ },
83
+ "50262": {
84
+ "content": " ",
85
+ "lstrip": false,
86
+ "normalized": true,
87
+ "rstrip": false,
88
+ "single_word": false,
89
+ "special": false
90
+ },
91
+ "50263": {
92
+ "content": " ",
93
+ "lstrip": false,
94
+ "normalized": true,
95
+ "rstrip": false,
96
+ "single_word": false,
97
+ "special": false
98
+ },
99
+ "50264": {
100
+ "content": " ",
101
+ "lstrip": false,
102
+ "normalized": true,
103
+ "rstrip": false,
104
+ "single_word": false,
105
+ "special": false
106
+ },
107
+ "50265": {
108
+ "content": " ",
109
+ "lstrip": false,
110
+ "normalized": true,
111
+ "rstrip": false,
112
+ "single_word": false,
113
+ "special": false
114
+ },
115
+ "50266": {
116
+ "content": " ",
117
+ "lstrip": false,
118
+ "normalized": true,
119
+ "rstrip": false,
120
+ "single_word": false,
121
+ "special": false
122
+ },
123
+ "50267": {
124
+ "content": " ",
125
+ "lstrip": false,
126
+ "normalized": true,
127
+ "rstrip": false,
128
+ "single_word": false,
129
+ "special": false
130
+ },
131
+ "50268": {
132
+ "content": " ",
133
+ "lstrip": false,
134
+ "normalized": true,
135
+ "rstrip": false,
136
+ "single_word": false,
137
+ "special": false
138
+ },
139
+ "50269": {
140
+ "content": " ",
141
+ "lstrip": false,
142
+ "normalized": true,
143
+ "rstrip": false,
144
+ "single_word": false,
145
+ "special": false
146
+ },
147
+ "50270": {
148
+ "content": " ",
149
+ "lstrip": false,
150
+ "normalized": true,
151
+ "rstrip": false,
152
+ "single_word": false,
153
+ "special": false
154
+ },
155
+ "50271": {
156
+ "content": " ",
157
+ "lstrip": false,
158
+ "normalized": true,
159
+ "rstrip": false,
160
+ "single_word": false,
161
+ "special": false
162
+ },
163
+ "50272": {
164
+ "content": " ",
165
+ "lstrip": false,
166
+ "normalized": true,
167
+ "rstrip": false,
168
+ "single_word": false,
169
+ "special": false
170
+ },
171
+ "50273": {
172
+ "content": " ",
173
+ "lstrip": false,
174
+ "normalized": true,
175
+ "rstrip": false,
176
+ "single_word": false,
177
+ "special": false
178
+ },
179
+ "50274": {
180
+ "content": " ",
181
+ "lstrip": false,
182
+ "normalized": true,
183
+ "rstrip": false,
184
+ "single_word": false,
185
+ "special": false
186
+ },
187
+ "50275": {
188
+ "content": " ",
189
+ "lstrip": false,
190
+ "normalized": true,
191
+ "rstrip": false,
192
+ "single_word": false,
193
+ "special": false
194
+ },
195
+ "50276": {
196
+ "content": " ",
197
+ "lstrip": false,
198
+ "normalized": true,
199
+ "rstrip": false,
200
+ "single_word": false,
201
+ "special": false
202
+ },
203
+ "50277": {
204
+ "content": "|||EMAIL_ADDRESS|||",
205
+ "lstrip": false,
206
+ "normalized": true,
207
+ "rstrip": false,
208
+ "single_word": false,
209
+ "special": false
210
+ },
211
+ "50278": {
212
+ "content": "|||PHONE_NUMBER|||",
213
+ "lstrip": false,
214
+ "normalized": true,
215
+ "rstrip": false,
216
+ "single_word": false,
217
+ "special": false
218
+ },
219
+ "50279": {
220
+ "content": "<|endoftext|>",
221
+ "lstrip": false,
222
+ "normalized": false,
223
+ "rstrip": false,
224
+ "single_word": false,
225
+ "special": true
226
+ },
227
+ "50280": {
228
+ "content": "[UNK]",
229
+ "lstrip": false,
230
+ "normalized": false,
231
+ "rstrip": false,
232
+ "single_word": false,
233
+ "special": true
234
+ },
235
+ "50281": {
236
+ "content": "[CLS]",
237
+ "lstrip": false,
238
+ "normalized": false,
239
+ "rstrip": false,
240
+ "single_word": false,
241
+ "special": true
242
+ },
243
+ "50282": {
244
+ "content": "[SEP]",
245
+ "lstrip": false,
246
+ "normalized": false,
247
+ "rstrip": false,
248
+ "single_word": false,
249
+ "special": true
250
+ },
251
+ "50283": {
252
+ "content": "[PAD]",
253
+ "lstrip": false,
254
+ "normalized": false,
255
+ "rstrip": false,
256
+ "single_word": false,
257
+ "special": true
258
+ },
259
+ "50284": {
260
+ "content": "[MASK]",
261
+ "lstrip": true,
262
+ "normalized": false,
263
+ "rstrip": false,
264
+ "single_word": false,
265
+ "special": true
266
+ },
267
+ "95580": {
268
+ "content": "[unused0]",
269
+ "lstrip": false,
270
+ "normalized": true,
271
+ "rstrip": false,
272
+ "single_word": false,
273
+ "special": false
274
+ },
275
+ "95581": {
276
+ "content": "[unused1]",
277
+ "lstrip": false,
278
+ "normalized": true,
279
+ "rstrip": false,
280
+ "single_word": false,
281
+ "special": false
282
+ },
283
+ "95582": {
284
+ "content": "[unused2]",
285
+ "lstrip": false,
286
+ "normalized": true,
287
+ "rstrip": false,
288
+ "single_word": false,
289
+ "special": false
290
+ },
291
+ "95583": {
292
+ "content": "[unused3]",
293
+ "lstrip": false,
294
+ "normalized": true,
295
+ "rstrip": false,
296
+ "single_word": false,
297
+ "special": false
298
+ },
299
+ "95584": {
300
+ "content": "[unused4]",
301
+ "lstrip": false,
302
+ "normalized": true,
303
+ "rstrip": false,
304
+ "single_word": false,
305
+ "special": false
306
+ },
307
+ "95585": {
308
+ "content": "[unused5]",
309
+ "lstrip": false,
310
+ "normalized": true,
311
+ "rstrip": false,
312
+ "single_word": false,
313
+ "special": false
314
+ },
315
+ "95586": {
316
+ "content": "[unused6]",
317
+ "lstrip": false,
318
+ "normalized": true,
319
+ "rstrip": false,
320
+ "single_word": false,
321
+ "special": false
322
+ },
323
+ "95587": {
324
+ "content": "[unused7]",
325
+ "lstrip": false,
326
+ "normalized": true,
327
+ "rstrip": false,
328
+ "single_word": false,
329
+ "special": false
330
+ },
331
+ "95588": {
332
+ "content": "[unused8]",
333
+ "lstrip": false,
334
+ "normalized": true,
335
+ "rstrip": false,
336
+ "single_word": false,
337
+ "special": false
338
+ },
339
+ "95589": {
340
+ "content": "[unused9]",
341
+ "lstrip": false,
342
+ "normalized": true,
343
+ "rstrip": false,
344
+ "single_word": false,
345
+ "special": false
346
+ },
347
+ "95590": {
348
+ "content": "[unused10]",
349
+ "lstrip": false,
350
+ "normalized": true,
351
+ "rstrip": false,
352
+ "single_word": false,
353
+ "special": false
354
+ },
355
+ "95591": {
356
+ "content": "[unused11]",
357
+ "lstrip": false,
358
+ "normalized": true,
359
+ "rstrip": false,
360
+ "single_word": false,
361
+ "special": false
362
+ },
363
+ "95592": {
364
+ "content": "[unused12]",
365
+ "lstrip": false,
366
+ "normalized": true,
367
+ "rstrip": false,
368
+ "single_word": false,
369
+ "special": false
370
+ },
371
+ "95593": {
372
+ "content": "[unused13]",
373
+ "lstrip": false,
374
+ "normalized": true,
375
+ "rstrip": false,
376
+ "single_word": false,
377
+ "special": false
378
+ },
379
+ "95594": {
380
+ "content": "[unused14]",
381
+ "lstrip": false,
382
+ "normalized": true,
383
+ "rstrip": false,
384
+ "single_word": false,
385
+ "special": false
386
+ },
387
+ "95595": {
388
+ "content": "[unused15]",
389
+ "lstrip": false,
390
+ "normalized": true,
391
+ "rstrip": false,
392
+ "single_word": false,
393
+ "special": false
394
+ },
395
+ "95596": {
396
+ "content": "[unused16]",
397
+ "lstrip": false,
398
+ "normalized": true,
399
+ "rstrip": false,
400
+ "single_word": false,
401
+ "special": false
402
+ },
403
+ "95597": {
404
+ "content": "[unused17]",
405
+ "lstrip": false,
406
+ "normalized": true,
407
+ "rstrip": false,
408
+ "single_word": false,
409
+ "special": false
410
+ },
411
+ "95598": {
412
+ "content": "[unused18]",
413
+ "lstrip": false,
414
+ "normalized": true,
415
+ "rstrip": false,
416
+ "single_word": false,
417
+ "special": false
418
+ },
419
+ "95599": {
420
+ "content": "[unused19]",
421
+ "lstrip": false,
422
+ "normalized": true,
423
+ "rstrip": false,
424
+ "single_word": false,
425
+ "special": false
426
+ },
427
+ "95600": {
428
+ "content": "[unused20]",
429
+ "lstrip": false,
430
+ "normalized": true,
431
+ "rstrip": false,
432
+ "single_word": false,
433
+ "special": false
434
+ },
435
+ "95601": {
436
+ "content": "[unused21]",
437
+ "lstrip": false,
438
+ "normalized": true,
439
+ "rstrip": false,
440
+ "single_word": false,
441
+ "special": false
442
+ },
443
+ "95602": {
444
+ "content": "[unused22]",
445
+ "lstrip": false,
446
+ "normalized": true,
447
+ "rstrip": false,
448
+ "single_word": false,
449
+ "special": false
450
+ },
451
+ "95603": {
452
+ "content": "[unused23]",
453
+ "lstrip": false,
454
+ "normalized": true,
455
+ "rstrip": false,
456
+ "single_word": false,
457
+ "special": false
458
+ },
459
+ "95604": {
460
+ "content": "[unused24]",
461
+ "lstrip": false,
462
+ "normalized": true,
463
+ "rstrip": false,
464
+ "single_word": false,
465
+ "special": false
466
+ },
467
+ "95605": {
468
+ "content": "[unused25]",
469
+ "lstrip": false,
470
+ "normalized": true,
471
+ "rstrip": false,
472
+ "single_word": false,
473
+ "special": false
474
+ },
475
+ "95606": {
476
+ "content": "[unused26]",
477
+ "lstrip": false,
478
+ "normalized": true,
479
+ "rstrip": false,
480
+ "single_word": false,
481
+ "special": false
482
+ },
483
+ "95607": {
484
+ "content": "[unused27]",
485
+ "lstrip": false,
486
+ "normalized": true,
487
+ "rstrip": false,
488
+ "single_word": false,
489
+ "special": false
490
+ },
491
+ "95608": {
492
+ "content": "[unused28]",
493
+ "lstrip": false,
494
+ "normalized": true,
495
+ "rstrip": false,
496
+ "single_word": false,
497
+ "special": false
498
+ },
499
+ "95609": {
500
+ "content": "[unused29]",
501
+ "lstrip": false,
502
+ "normalized": true,
503
+ "rstrip": false,
504
+ "single_word": false,
505
+ "special": false
506
+ },
507
+ "95610": {
508
+ "content": "[unused30]",
509
+ "lstrip": false,
510
+ "normalized": true,
511
+ "rstrip": false,
512
+ "single_word": false,
513
+ "special": false
514
+ },
515
+ "95611": {
516
+ "content": "[unused31]",
517
+ "lstrip": false,
518
+ "normalized": true,
519
+ "rstrip": false,
520
+ "single_word": false,
521
+ "special": false
522
+ },
523
+ "95612": {
524
+ "content": "[unused32]",
525
+ "lstrip": false,
526
+ "normalized": true,
527
+ "rstrip": false,
528
+ "single_word": false,
529
+ "special": false
530
+ },
531
+ "95613": {
532
+ "content": "[unused33]",
533
+ "lstrip": false,
534
+ "normalized": true,
535
+ "rstrip": false,
536
+ "single_word": false,
537
+ "special": false
538
+ },
539
+ "95614": {
540
+ "content": "[unused34]",
541
+ "lstrip": false,
542
+ "normalized": true,
543
+ "rstrip": false,
544
+ "single_word": false,
545
+ "special": false
546
+ },
547
+ "95615": {
548
+ "content": "[unused35]",
549
+ "lstrip": false,
550
+ "normalized": true,
551
+ "rstrip": false,
552
+ "single_word": false,
553
+ "special": false
554
+ },
555
+ "95616": {
556
+ "content": "[unused36]",
557
+ "lstrip": false,
558
+ "normalized": true,
559
+ "rstrip": false,
560
+ "single_word": false,
561
+ "special": false
562
+ },
563
+ "95617": {
564
+ "content": "[unused37]",
565
+ "lstrip": false,
566
+ "normalized": true,
567
+ "rstrip": false,
568
+ "single_word": false,
569
+ "special": false
570
+ },
571
+ "95618": {
572
+ "content": "[unused38]",
573
+ "lstrip": false,
574
+ "normalized": true,
575
+ "rstrip": false,
576
+ "single_word": false,
577
+ "special": false
578
+ },
579
+ "95619": {
580
+ "content": "[unused39]",
581
+ "lstrip": false,
582
+ "normalized": true,
583
+ "rstrip": false,
584
+ "single_word": false,
585
+ "special": false
586
+ },
587
+ "95620": {
588
+ "content": "[unused40]",
589
+ "lstrip": false,
590
+ "normalized": true,
591
+ "rstrip": false,
592
+ "single_word": false,
593
+ "special": false
594
+ },
595
+ "95621": {
596
+ "content": "[unused41]",
597
+ "lstrip": false,
598
+ "normalized": true,
599
+ "rstrip": false,
600
+ "single_word": false,
601
+ "special": false
602
+ },
603
+ "95622": {
604
+ "content": "[unused42]",
605
+ "lstrip": false,
606
+ "normalized": true,
607
+ "rstrip": false,
608
+ "single_word": false,
609
+ "special": false
610
+ },
611
+ "95623": {
612
+ "content": "[unused43]",
613
+ "lstrip": false,
614
+ "normalized": true,
615
+ "rstrip": false,
616
+ "single_word": false,
617
+ "special": false
618
+ },
619
+ "95624": {
620
+ "content": "[unused44]",
621
+ "lstrip": false,
622
+ "normalized": true,
623
+ "rstrip": false,
624
+ "single_word": false,
625
+ "special": false
626
+ },
627
+ "95625": {
628
+ "content": "[unused45]",
629
+ "lstrip": false,
630
+ "normalized": true,
631
+ "rstrip": false,
632
+ "single_word": false,
633
+ "special": false
634
+ },
635
+ "95626": {
636
+ "content": "[unused46]",
637
+ "lstrip": false,
638
+ "normalized": true,
639
+ "rstrip": false,
640
+ "single_word": false,
641
+ "special": false
642
+ },
643
+ "95627": {
644
+ "content": "[unused47]",
645
+ "lstrip": false,
646
+ "normalized": true,
647
+ "rstrip": false,
648
+ "single_word": false,
649
+ "special": false
650
+ },
651
+ "95628": {
652
+ "content": "[unused48]",
653
+ "lstrip": false,
654
+ "normalized": true,
655
+ "rstrip": false,
656
+ "single_word": false,
657
+ "special": false
658
+ },
659
+ "95629": {
660
+ "content": "[unused49]",
661
+ "lstrip": false,
662
+ "normalized": true,
663
+ "rstrip": false,
664
+ "single_word": false,
665
+ "special": false
666
+ },
667
+ "95630": {
668
+ "content": "[unused50]",
669
+ "lstrip": false,
670
+ "normalized": true,
671
+ "rstrip": false,
672
+ "single_word": false,
673
+ "special": false
674
+ },
675
+ "95631": {
676
+ "content": "[unused51]",
677
+ "lstrip": false,
678
+ "normalized": true,
679
+ "rstrip": false,
680
+ "single_word": false,
681
+ "special": false
682
+ },
683
+ "95632": {
684
+ "content": "[unused52]",
685
+ "lstrip": false,
686
+ "normalized": true,
687
+ "rstrip": false,
688
+ "single_word": false,
689
+ "special": false
690
+ },
691
+ "95633": {
692
+ "content": "[unused53]",
693
+ "lstrip": false,
694
+ "normalized": true,
695
+ "rstrip": false,
696
+ "single_word": false,
697
+ "special": false
698
+ },
699
+ "95634": {
700
+ "content": "[unused54]",
701
+ "lstrip": false,
702
+ "normalized": true,
703
+ "rstrip": false,
704
+ "single_word": false,
705
+ "special": false
706
+ },
707
+ "95635": {
708
+ "content": "[unused55]",
709
+ "lstrip": false,
710
+ "normalized": true,
711
+ "rstrip": false,
712
+ "single_word": false,
713
+ "special": false
714
+ },
715
+ "95636": {
716
+ "content": "[unused56]",
717
+ "lstrip": false,
718
+ "normalized": true,
719
+ "rstrip": false,
720
+ "single_word": false,
721
+ "special": false
722
+ },
723
+ "95637": {
724
+ "content": "[unused57]",
725
+ "lstrip": false,
726
+ "normalized": true,
727
+ "rstrip": false,
728
+ "single_word": false,
729
+ "special": false
730
+ },
731
+ "95638": {
732
+ "content": "[unused58]",
733
+ "lstrip": false,
734
+ "normalized": true,
735
+ "rstrip": false,
736
+ "single_word": false,
737
+ "special": false
738
+ },
739
+ "95639": {
740
+ "content": "[unused59]",
741
+ "lstrip": false,
742
+ "normalized": true,
743
+ "rstrip": false,
744
+ "single_word": false,
745
+ "special": false
746
+ },
747
+ "95640": {
748
+ "content": "[unused60]",
749
+ "lstrip": false,
750
+ "normalized": true,
751
+ "rstrip": false,
752
+ "single_word": false,
753
+ "special": false
754
+ },
755
+ "95641": {
756
+ "content": "[unused61]",
757
+ "lstrip": false,
758
+ "normalized": true,
759
+ "rstrip": false,
760
+ "single_word": false,
761
+ "special": false
762
+ },
763
+ "95642": {
764
+ "content": "[unused62]",
765
+ "lstrip": false,
766
+ "normalized": true,
767
+ "rstrip": false,
768
+ "single_word": false,
769
+ "special": false
770
+ },
771
+ "95643": {
772
+ "content": "[unused63]",
773
+ "lstrip": false,
774
+ "normalized": true,
775
+ "rstrip": false,
776
+ "single_word": false,
777
+ "special": false
778
+ },
779
+ "95644": {
780
+ "content": "[unused64]",
781
+ "lstrip": false,
782
+ "normalized": true,
783
+ "rstrip": false,
784
+ "single_word": false,
785
+ "special": false
786
+ },
787
+ "95645": {
788
+ "content": "[unused65]",
789
+ "lstrip": false,
790
+ "normalized": true,
791
+ "rstrip": false,
792
+ "single_word": false,
793
+ "special": false
794
+ },
795
+ "95646": {
796
+ "content": "[unused66]",
797
+ "lstrip": false,
798
+ "normalized": true,
799
+ "rstrip": false,
800
+ "single_word": false,
801
+ "special": false
802
+ },
803
+ "95647": {
804
+ "content": "[unused67]",
805
+ "lstrip": false,
806
+ "normalized": true,
807
+ "rstrip": false,
808
+ "single_word": false,
809
+ "special": false
810
+ },
811
+ "95648": {
812
+ "content": "[unused68]",
813
+ "lstrip": false,
814
+ "normalized": true,
815
+ "rstrip": false,
816
+ "single_word": false,
817
+ "special": false
818
+ },
819
+ "95649": {
820
+ "content": "[unused69]",
821
+ "lstrip": false,
822
+ "normalized": true,
823
+ "rstrip": false,
824
+ "single_word": false,
825
+ "special": false
826
+ },
827
+ "95650": {
828
+ "content": "[unused70]",
829
+ "lstrip": false,
830
+ "normalized": true,
831
+ "rstrip": false,
832
+ "single_word": false,
833
+ "special": false
834
+ },
835
+ "95651": {
836
+ "content": "[unused71]",
837
+ "lstrip": false,
838
+ "normalized": true,
839
+ "rstrip": false,
840
+ "single_word": false,
841
+ "special": false
842
+ },
843
+ "95652": {
844
+ "content": "[unused72]",
845
+ "lstrip": false,
846
+ "normalized": true,
847
+ "rstrip": false,
848
+ "single_word": false,
849
+ "special": false
850
+ },
851
+ "95653": {
852
+ "content": "[unused73]",
853
+ "lstrip": false,
854
+ "normalized": true,
855
+ "rstrip": false,
856
+ "single_word": false,
857
+ "special": false
858
+ },
859
+ "95654": {
860
+ "content": "[unused74]",
861
+ "lstrip": false,
862
+ "normalized": true,
863
+ "rstrip": false,
864
+ "single_word": false,
865
+ "special": false
866
+ },
867
+ "95655": {
868
+ "content": "[unused75]",
869
+ "lstrip": false,
870
+ "normalized": true,
871
+ "rstrip": false,
872
+ "single_word": false,
873
+ "special": false
874
+ },
875
+ "95656": {
876
+ "content": "[unused76]",
877
+ "lstrip": false,
878
+ "normalized": true,
879
+ "rstrip": false,
880
+ "single_word": false,
881
+ "special": false
882
+ },
883
+ "95657": {
884
+ "content": "[unused77]",
885
+ "lstrip": false,
886
+ "normalized": true,
887
+ "rstrip": false,
888
+ "single_word": false,
889
+ "special": false
890
+ },
891
+ "95658": {
892
+ "content": "[unused78]",
893
+ "lstrip": false,
894
+ "normalized": true,
895
+ "rstrip": false,
896
+ "single_word": false,
897
+ "special": false
898
+ },
899
+ "95659": {
900
+ "content": "[unused79]",
901
+ "lstrip": false,
902
+ "normalized": true,
903
+ "rstrip": false,
904
+ "single_word": false,
905
+ "special": false
906
+ },
907
+ "95660": {
908
+ "content": "[unused80]",
909
+ "lstrip": false,
910
+ "normalized": true,
911
+ "rstrip": false,
912
+ "single_word": false,
913
+ "special": false
914
+ },
915
+ "95661": {
916
+ "content": "[unused81]",
917
+ "lstrip": false,
918
+ "normalized": true,
919
+ "rstrip": false,
920
+ "single_word": false,
921
+ "special": false
922
+ },
923
+ "95662": {
924
+ "content": "[unused82]",
925
+ "lstrip": false,
926
+ "normalized": true,
927
+ "rstrip": false,
928
+ "single_word": false,
929
+ "special": false
930
+ }
931
+ },
932
+ "clean_up_tokenization_spaces": true,
933
+ "cls_token": "[CLS]",
934
+ "extra_special_tokens": {},
935
+ "mask_token": "[MASK]",
936
+ "max_length": 512,
937
+ "model_input_names": [
938
+ "input_ids",
939
+ "attention_mask"
940
+ ],
941
+ "model_max_length": 512,
942
+ "pad_to_multiple_of": null,
943
+ "pad_token": "[PAD]",
944
+ "pad_token_type_id": 0,
945
+ "padding_side": "right",
946
+ "sep_token": "[SEP]",
947
+ "stride": 0,
948
+ "tokenizer_class": "PreTrainedTokenizerFast",
949
+ "truncation_side": "right",
950
+ "truncation_strategy": "longest_first",
951
+ "unk_token": "[UNK]"
952
+ }
last-checkpoint/trainer_state.json ADDED
@@ -0,0 +1,2735 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 0.9957679860592482,
5
+ "eval_steps": 250,
6
+ "global_step": 3500,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "epoch": 0.0028450513887407092,
13
+ "grad_norm": 0.30044543743133545,
14
+ "learning_rate": 7.113007913221304e-08,
15
+ "loss": 0.0216,
16
+ "step": 10
17
+ },
18
+ {
19
+ "epoch": 0.0056901027774814185,
20
+ "grad_norm": 0.22410444915294647,
21
+ "learning_rate": 1.4226015826442608e-07,
22
+ "loss": 0.0204,
23
+ "step": 20
24
+ },
25
+ {
26
+ "epoch": 0.008535154166222128,
27
+ "grad_norm": 0.29121559858322144,
28
+ "learning_rate": 2.1339023739663913e-07,
29
+ "loss": 0.0194,
30
+ "step": 30
31
+ },
32
+ {
33
+ "epoch": 0.011380205554962837,
34
+ "grad_norm": 0.3065185248851776,
35
+ "learning_rate": 2.8452031652885216e-07,
36
+ "loss": 0.0195,
37
+ "step": 40
38
+ },
39
+ {
40
+ "epoch": 0.014225256943703546,
41
+ "grad_norm": 0.26153531670570374,
42
+ "learning_rate": 3.5565039566106516e-07,
43
+ "loss": 0.0182,
44
+ "step": 50
45
+ },
46
+ {
47
+ "epoch": 0.017070308332444255,
48
+ "grad_norm": 0.268179327249527,
49
+ "learning_rate": 4.2678047479327826e-07,
50
+ "loss": 0.0161,
51
+ "step": 60
52
+ },
53
+ {
54
+ "epoch": 0.019915359721184963,
55
+ "grad_norm": 0.26602137088775635,
56
+ "learning_rate": 4.979105539254913e-07,
57
+ "loss": 0.015,
58
+ "step": 70
59
+ },
60
+ {
61
+ "epoch": 0.022760411109925674,
62
+ "grad_norm": 0.20688457787036896,
63
+ "learning_rate": 5.690406330577043e-07,
64
+ "loss": 0.0153,
65
+ "step": 80
66
+ },
67
+ {
68
+ "epoch": 0.02560546249866638,
69
+ "grad_norm": 0.19044898450374603,
70
+ "learning_rate": 6.401707121899174e-07,
71
+ "loss": 0.0137,
72
+ "step": 90
73
+ },
74
+ {
75
+ "epoch": 0.028450513887407092,
76
+ "grad_norm": 0.22615332901477814,
77
+ "learning_rate": 7.113007913221303e-07,
78
+ "loss": 0.014,
79
+ "step": 100
80
+ },
81
+ {
82
+ "epoch": 0.0312955652761478,
83
+ "grad_norm": 0.20191101729869843,
84
+ "learning_rate": 7.824308704543435e-07,
85
+ "loss": 0.0122,
86
+ "step": 110
87
+ },
88
+ {
89
+ "epoch": 0.03414061666488851,
90
+ "grad_norm": 0.19718502461910248,
91
+ "learning_rate": 8.535609495865565e-07,
92
+ "loss": 0.0114,
93
+ "step": 120
94
+ },
95
+ {
96
+ "epoch": 0.03698566805362922,
97
+ "grad_norm": 0.1824241578578949,
98
+ "learning_rate": 9.246910287187695e-07,
99
+ "loss": 0.0109,
100
+ "step": 130
101
+ },
102
+ {
103
+ "epoch": 0.039830719442369926,
104
+ "grad_norm": 0.13791899383068085,
105
+ "learning_rate": 9.958211078509825e-07,
106
+ "loss": 0.0097,
107
+ "step": 140
108
+ },
109
+ {
110
+ "epoch": 0.04267577083111064,
111
+ "grad_norm": 0.11571934819221497,
112
+ "learning_rate": 1.0669511869831955e-06,
113
+ "loss": 0.0085,
114
+ "step": 150
115
+ },
116
+ {
117
+ "epoch": 0.04552082221985135,
118
+ "grad_norm": 0.13633102178573608,
119
+ "learning_rate": 1.1380812661154086e-06,
120
+ "loss": 0.0084,
121
+ "step": 160
122
+ },
123
+ {
124
+ "epoch": 0.048365873608592055,
125
+ "grad_norm": 0.08016248047351837,
126
+ "learning_rate": 1.2092113452476218e-06,
127
+ "loss": 0.0083,
128
+ "step": 170
129
+ },
130
+ {
131
+ "epoch": 0.05121092499733276,
132
+ "grad_norm": 0.10402435064315796,
133
+ "learning_rate": 1.2803414243798347e-06,
134
+ "loss": 0.0078,
135
+ "step": 180
136
+ },
137
+ {
138
+ "epoch": 0.05405597638607347,
139
+ "grad_norm": 0.10400915145874023,
140
+ "learning_rate": 1.3514715035120477e-06,
141
+ "loss": 0.008,
142
+ "step": 190
143
+ },
144
+ {
145
+ "epoch": 0.056901027774814185,
146
+ "grad_norm": 0.05126596987247467,
147
+ "learning_rate": 1.4226015826442606e-06,
148
+ "loss": 0.0073,
149
+ "step": 200
150
+ },
151
+ {
152
+ "epoch": 0.05974607916355489,
153
+ "grad_norm": 0.09815330803394318,
154
+ "learning_rate": 1.4937316617764738e-06,
155
+ "loss": 0.0079,
156
+ "step": 210
157
+ },
158
+ {
159
+ "epoch": 0.0625911305522956,
160
+ "grad_norm": 0.07512035965919495,
161
+ "learning_rate": 1.564861740908687e-06,
162
+ "loss": 0.0073,
163
+ "step": 220
164
+ },
165
+ {
166
+ "epoch": 0.06543618194103631,
167
+ "grad_norm": 0.0874195247888565,
168
+ "learning_rate": 1.6359918200409001e-06,
169
+ "loss": 0.0079,
170
+ "step": 230
171
+ },
172
+ {
173
+ "epoch": 0.06828123332977702,
174
+ "grad_norm": 0.05660032480955124,
175
+ "learning_rate": 1.707121899173113e-06,
176
+ "loss": 0.0068,
177
+ "step": 240
178
+ },
179
+ {
180
+ "epoch": 0.07112628471851773,
181
+ "grad_norm": 0.06961371004581451,
182
+ "learning_rate": 1.778251978305326e-06,
183
+ "loss": 0.0068,
184
+ "step": 250
185
+ },
186
+ {
187
+ "epoch": 0.07112628471851773,
188
+ "eval_loss": 0.03334415331482887,
189
+ "eval_runtime": 9.3985,
190
+ "eval_samples_per_second": 159.599,
191
+ "eval_steps_per_second": 10.002,
192
+ "eval_sts_dev_pearson_cosine": 0.8139279991578542,
193
+ "eval_sts_dev_pearson_dot": 0.750640851965511,
194
+ "eval_sts_dev_pearson_euclidean": 0.7813827425904325,
195
+ "eval_sts_dev_pearson_manhattan": 0.7825586268743923,
196
+ "eval_sts_dev_pearson_max": 0.8139279991578542,
197
+ "eval_sts_dev_spearman_cosine": 0.8228542044736794,
198
+ "eval_sts_dev_spearman_dot": 0.7341843393391922,
199
+ "eval_sts_dev_spearman_euclidean": 0.7859707213660079,
200
+ "eval_sts_dev_spearman_manhattan": 0.7886657735697056,
201
+ "eval_sts_dev_spearman_max": 0.8228542044736794,
202
+ "step": 250
203
+ },
204
+ {
205
+ "epoch": 0.07397133610725844,
206
+ "grad_norm": 0.05404730886220932,
207
+ "learning_rate": 1.849382057437539e-06,
208
+ "loss": 0.0073,
209
+ "step": 260
210
+ },
211
+ {
212
+ "epoch": 0.07681638749599914,
213
+ "grad_norm": 0.07932556420564651,
214
+ "learning_rate": 1.920512136569752e-06,
215
+ "loss": 0.0077,
216
+ "step": 270
217
+ },
218
+ {
219
+ "epoch": 0.07966143888473985,
220
+ "grad_norm": 0.055235493928194046,
221
+ "learning_rate": 1.991642215701965e-06,
222
+ "loss": 0.0067,
223
+ "step": 280
224
+ },
225
+ {
226
+ "epoch": 0.08250649027348056,
227
+ "grad_norm": 0.06954500079154968,
228
+ "learning_rate": 2.0627722948341784e-06,
229
+ "loss": 0.007,
230
+ "step": 290
231
+ },
232
+ {
233
+ "epoch": 0.08535154166222128,
234
+ "grad_norm": 0.08223138749599457,
235
+ "learning_rate": 2.133902373966391e-06,
236
+ "loss": 0.0065,
237
+ "step": 300
238
+ },
239
+ {
240
+ "epoch": 0.08819659305096199,
241
+ "grad_norm": 0.06235595792531967,
242
+ "learning_rate": 2.2050324530986043e-06,
243
+ "loss": 0.0072,
244
+ "step": 310
245
+ },
246
+ {
247
+ "epoch": 0.0910416444397027,
248
+ "grad_norm": 0.06472361832857132,
249
+ "learning_rate": 2.2761625322308173e-06,
250
+ "loss": 0.0068,
251
+ "step": 320
252
+ },
253
+ {
254
+ "epoch": 0.0938866958284434,
255
+ "grad_norm": 0.06870505958795547,
256
+ "learning_rate": 2.3472926113630302e-06,
257
+ "loss": 0.0064,
258
+ "step": 330
259
+ },
260
+ {
261
+ "epoch": 0.09673174721718411,
262
+ "grad_norm": 0.07072167843580246,
263
+ "learning_rate": 2.4184226904952436e-06,
264
+ "loss": 0.0074,
265
+ "step": 340
266
+ },
267
+ {
268
+ "epoch": 0.09957679860592482,
269
+ "grad_norm": 0.065520741045475,
270
+ "learning_rate": 2.489552769627456e-06,
271
+ "loss": 0.0071,
272
+ "step": 350
273
+ },
274
+ {
275
+ "epoch": 0.10242184999466553,
276
+ "grad_norm": 0.06320654600858688,
277
+ "learning_rate": 2.5606828487596695e-06,
278
+ "loss": 0.0065,
279
+ "step": 360
280
+ },
281
+ {
282
+ "epoch": 0.10526690138340623,
283
+ "grad_norm": 0.08099081367254257,
284
+ "learning_rate": 2.6318129278918824e-06,
285
+ "loss": 0.0067,
286
+ "step": 370
287
+ },
288
+ {
289
+ "epoch": 0.10811195277214694,
290
+ "grad_norm": 0.06201442331075668,
291
+ "learning_rate": 2.7029430070240954e-06,
292
+ "loss": 0.0063,
293
+ "step": 380
294
+ },
295
+ {
296
+ "epoch": 0.11095700416088766,
297
+ "grad_norm": 0.06379993259906769,
298
+ "learning_rate": 2.7740730861563087e-06,
299
+ "loss": 0.0062,
300
+ "step": 390
301
+ },
302
+ {
303
+ "epoch": 0.11380205554962837,
304
+ "grad_norm": 0.05820363759994507,
305
+ "learning_rate": 2.8452031652885213e-06,
306
+ "loss": 0.0068,
307
+ "step": 400
308
+ },
309
+ {
310
+ "epoch": 0.11664710693836908,
311
+ "grad_norm": 0.05578601360321045,
312
+ "learning_rate": 2.9163332444207346e-06,
313
+ "loss": 0.0064,
314
+ "step": 410
315
+ },
316
+ {
317
+ "epoch": 0.11949215832710978,
318
+ "grad_norm": 0.06488487124443054,
319
+ "learning_rate": 2.9874633235529476e-06,
320
+ "loss": 0.0064,
321
+ "step": 420
322
+ },
323
+ {
324
+ "epoch": 0.12233720971585049,
325
+ "grad_norm": 0.06347406655550003,
326
+ "learning_rate": 3.0585934026851605e-06,
327
+ "loss": 0.0064,
328
+ "step": 430
329
+ },
330
+ {
331
+ "epoch": 0.1251822611045912,
332
+ "grad_norm": 0.07820002734661102,
333
+ "learning_rate": 3.129723481817374e-06,
334
+ "loss": 0.0074,
335
+ "step": 440
336
+ },
337
+ {
338
+ "epoch": 0.1280273124933319,
339
+ "grad_norm": 0.052174266427755356,
340
+ "learning_rate": 3.200853560949587e-06,
341
+ "loss": 0.0069,
342
+ "step": 450
343
+ },
344
+ {
345
+ "epoch": 0.13087236388207263,
346
+ "grad_norm": 0.0655951127409935,
347
+ "learning_rate": 3.2719836400818002e-06,
348
+ "loss": 0.0065,
349
+ "step": 460
350
+ },
351
+ {
352
+ "epoch": 0.13371741527081332,
353
+ "grad_norm": 0.07082219421863556,
354
+ "learning_rate": 3.3431137192140127e-06,
355
+ "loss": 0.0067,
356
+ "step": 470
357
+ },
358
+ {
359
+ "epoch": 0.13656246665955404,
360
+ "grad_norm": 0.05722203105688095,
361
+ "learning_rate": 3.414243798346226e-06,
362
+ "loss": 0.0068,
363
+ "step": 480
364
+ },
365
+ {
366
+ "epoch": 0.13940751804829474,
367
+ "grad_norm": 0.042512740939855576,
368
+ "learning_rate": 3.485373877478439e-06,
369
+ "loss": 0.0057,
370
+ "step": 490
371
+ },
372
+ {
373
+ "epoch": 0.14225256943703546,
374
+ "grad_norm": 0.07777195423841476,
375
+ "learning_rate": 3.556503956610652e-06,
376
+ "loss": 0.0065,
377
+ "step": 500
378
+ },
379
+ {
380
+ "epoch": 0.14225256943703546,
381
+ "eval_loss": 0.034268368035554886,
382
+ "eval_runtime": 9.4571,
383
+ "eval_samples_per_second": 158.611,
384
+ "eval_steps_per_second": 9.94,
385
+ "eval_sts_dev_pearson_cosine": 0.8181914244558982,
386
+ "eval_sts_dev_pearson_dot": 0.7449626401674933,
387
+ "eval_sts_dev_pearson_euclidean": 0.7912044446283607,
388
+ "eval_sts_dev_pearson_manhattan": 0.7922858740750306,
389
+ "eval_sts_dev_pearson_max": 0.8181914244558982,
390
+ "eval_sts_dev_spearman_cosine": 0.828420839653431,
391
+ "eval_sts_dev_spearman_dot": 0.7263741357918354,
392
+ "eval_sts_dev_spearman_euclidean": 0.7977480260329146,
393
+ "eval_sts_dev_spearman_manhattan": 0.800148774872662,
394
+ "eval_sts_dev_spearman_max": 0.828420839653431,
395
+ "step": 500
396
+ },
397
+ {
398
+ "epoch": 0.14509762082577615,
399
+ "grad_norm": 0.06219501420855522,
400
+ "learning_rate": 3.6276340357428654e-06,
401
+ "loss": 0.0069,
402
+ "step": 510
403
+ },
404
+ {
405
+ "epoch": 0.14794267221451687,
406
+ "grad_norm": 0.06601779162883759,
407
+ "learning_rate": 3.698764114875078e-06,
408
+ "loss": 0.0068,
409
+ "step": 520
410
+ },
411
+ {
412
+ "epoch": 0.1507877236032576,
413
+ "grad_norm": 0.06005557253956795,
414
+ "learning_rate": 3.7698941940072913e-06,
415
+ "loss": 0.0065,
416
+ "step": 530
417
+ },
418
+ {
419
+ "epoch": 0.1536327749919983,
420
+ "grad_norm": 0.05748499929904938,
421
+ "learning_rate": 3.841024273139504e-06,
422
+ "loss": 0.0065,
423
+ "step": 540
424
+ },
425
+ {
426
+ "epoch": 0.156477826380739,
427
+ "grad_norm": 0.06991992145776749,
428
+ "learning_rate": 3.912154352271717e-06,
429
+ "loss": 0.0063,
430
+ "step": 550
431
+ },
432
+ {
433
+ "epoch": 0.1593228777694797,
434
+ "grad_norm": 0.04834827408194542,
435
+ "learning_rate": 3.98328443140393e-06,
436
+ "loss": 0.0058,
437
+ "step": 560
438
+ },
439
+ {
440
+ "epoch": 0.16216792915822043,
441
+ "grad_norm": 0.05604851618409157,
442
+ "learning_rate": 4.054414510536143e-06,
443
+ "loss": 0.0064,
444
+ "step": 570
445
+ },
446
+ {
447
+ "epoch": 0.16501298054696112,
448
+ "grad_norm": 0.05394286289811134,
449
+ "learning_rate": 4.125544589668357e-06,
450
+ "loss": 0.0062,
451
+ "step": 580
452
+ },
453
+ {
454
+ "epoch": 0.16785803193570184,
455
+ "grad_norm": 0.0512845516204834,
456
+ "learning_rate": 4.19667466880057e-06,
457
+ "loss": 0.0061,
458
+ "step": 590
459
+ },
460
+ {
461
+ "epoch": 0.17070308332444256,
462
+ "grad_norm": 0.0574568510055542,
463
+ "learning_rate": 4.267804747932782e-06,
464
+ "loss": 0.0062,
465
+ "step": 600
466
+ },
467
+ {
468
+ "epoch": 0.17354813471318326,
469
+ "grad_norm": 0.06107944995164871,
470
+ "learning_rate": 4.338934827064996e-06,
471
+ "loss": 0.0057,
472
+ "step": 610
473
+ },
474
+ {
475
+ "epoch": 0.17639318610192398,
476
+ "grad_norm": 0.06508038192987442,
477
+ "learning_rate": 4.410064906197209e-06,
478
+ "loss": 0.0066,
479
+ "step": 620
480
+ },
481
+ {
482
+ "epoch": 0.17923823749066467,
483
+ "grad_norm": 0.0429425947368145,
484
+ "learning_rate": 4.481194985329422e-06,
485
+ "loss": 0.0061,
486
+ "step": 630
487
+ },
488
+ {
489
+ "epoch": 0.1820832888794054,
490
+ "grad_norm": 0.07746722549200058,
491
+ "learning_rate": 4.5523250644616345e-06,
492
+ "loss": 0.0054,
493
+ "step": 640
494
+ },
495
+ {
496
+ "epoch": 0.18492834026814609,
497
+ "grad_norm": 0.05604277178645134,
498
+ "learning_rate": 4.6234551435938475e-06,
499
+ "loss": 0.0066,
500
+ "step": 650
501
+ },
502
+ {
503
+ "epoch": 0.1877733916568868,
504
+ "grad_norm": 0.04747707024216652,
505
+ "learning_rate": 4.6945852227260604e-06,
506
+ "loss": 0.0059,
507
+ "step": 660
508
+ },
509
+ {
510
+ "epoch": 0.1906184430456275,
511
+ "grad_norm": 0.049124643206596375,
512
+ "learning_rate": 4.765715301858273e-06,
513
+ "loss": 0.0063,
514
+ "step": 670
515
+ },
516
+ {
517
+ "epoch": 0.19346349443436822,
518
+ "grad_norm": 0.05432770773768425,
519
+ "learning_rate": 4.836845380990487e-06,
520
+ "loss": 0.0065,
521
+ "step": 680
522
+ },
523
+ {
524
+ "epoch": 0.19630854582310894,
525
+ "grad_norm": 0.04070006310939789,
526
+ "learning_rate": 4.9079754601227e-06,
527
+ "loss": 0.0065,
528
+ "step": 690
529
+ },
530
+ {
531
+ "epoch": 0.19915359721184964,
532
+ "grad_norm": 0.05693609267473221,
533
+ "learning_rate": 4.979105539254912e-06,
534
+ "loss": 0.0058,
535
+ "step": 700
536
+ },
537
+ {
538
+ "epoch": 0.20199864860059036,
539
+ "grad_norm": 0.051310617476701736,
540
+ "learning_rate": 5.050235618387126e-06,
541
+ "loss": 0.006,
542
+ "step": 710
543
+ },
544
+ {
545
+ "epoch": 0.20484369998933105,
546
+ "grad_norm": 0.0512520931661129,
547
+ "learning_rate": 5.121365697519339e-06,
548
+ "loss": 0.0062,
549
+ "step": 720
550
+ },
551
+ {
552
+ "epoch": 0.20768875137807177,
553
+ "grad_norm": 0.06032096594572067,
554
+ "learning_rate": 5.192495776651552e-06,
555
+ "loss": 0.0058,
556
+ "step": 730
557
+ },
558
+ {
559
+ "epoch": 0.21053380276681247,
560
+ "grad_norm": 0.05407226085662842,
561
+ "learning_rate": 5.263625855783765e-06,
562
+ "loss": 0.0058,
563
+ "step": 740
564
+ },
565
+ {
566
+ "epoch": 0.2133788541555532,
567
+ "grad_norm": 0.04472426697611809,
568
+ "learning_rate": 5.334755934915979e-06,
569
+ "loss": 0.0056,
570
+ "step": 750
571
+ },
572
+ {
573
+ "epoch": 0.2133788541555532,
574
+ "eval_loss": 0.035597361624240875,
575
+ "eval_runtime": 9.4023,
576
+ "eval_samples_per_second": 159.536,
577
+ "eval_steps_per_second": 9.998,
578
+ "eval_sts_dev_pearson_cosine": 0.8192166641007746,
579
+ "eval_sts_dev_pearson_dot": 0.7403343789875012,
580
+ "eval_sts_dev_pearson_euclidean": 0.7962870902346526,
581
+ "eval_sts_dev_pearson_manhattan": 0.7970902993298804,
582
+ "eval_sts_dev_pearson_max": 0.8192166641007746,
583
+ "eval_sts_dev_spearman_cosine": 0.8301874075212713,
584
+ "eval_sts_dev_spearman_dot": 0.7214281718521472,
585
+ "eval_sts_dev_spearman_euclidean": 0.8028538594956929,
586
+ "eval_sts_dev_spearman_manhattan": 0.805293771786166,
587
+ "eval_sts_dev_spearman_max": 0.8301874075212713,
588
+ "step": 750
589
+ },
590
+ {
591
+ "epoch": 0.21622390554429388,
592
+ "grad_norm": 0.06494403630495071,
593
+ "learning_rate": 5.405886014048191e-06,
594
+ "loss": 0.0067,
595
+ "step": 760
596
+ },
597
+ {
598
+ "epoch": 0.2190689569330346,
599
+ "grad_norm": 0.06772973388433456,
600
+ "learning_rate": 5.477016093180404e-06,
601
+ "loss": 0.0063,
602
+ "step": 770
603
+ },
604
+ {
605
+ "epoch": 0.22191400832177532,
606
+ "grad_norm": 0.051205262541770935,
607
+ "learning_rate": 5.5481461723126175e-06,
608
+ "loss": 0.0063,
609
+ "step": 780
610
+ },
611
+ {
612
+ "epoch": 0.22475905971051602,
613
+ "grad_norm": 0.057035986334085464,
614
+ "learning_rate": 5.6192762514448304e-06,
615
+ "loss": 0.0063,
616
+ "step": 790
617
+ },
618
+ {
619
+ "epoch": 0.22760411109925674,
620
+ "grad_norm": 0.048164695501327515,
621
+ "learning_rate": 5.6904063305770425e-06,
622
+ "loss": 0.0056,
623
+ "step": 800
624
+ },
625
+ {
626
+ "epoch": 0.23044916248799743,
627
+ "grad_norm": 0.05380985513329506,
628
+ "learning_rate": 5.761536409709256e-06,
629
+ "loss": 0.0058,
630
+ "step": 810
631
+ },
632
+ {
633
+ "epoch": 0.23329421387673815,
634
+ "grad_norm": 0.03878633305430412,
635
+ "learning_rate": 5.832666488841469e-06,
636
+ "loss": 0.0053,
637
+ "step": 820
638
+ },
639
+ {
640
+ "epoch": 0.23613926526547885,
641
+ "grad_norm": 0.04493873938918114,
642
+ "learning_rate": 5.903796567973682e-06,
643
+ "loss": 0.0057,
644
+ "step": 830
645
+ },
646
+ {
647
+ "epoch": 0.23898431665421957,
648
+ "grad_norm": 0.03924809396266937,
649
+ "learning_rate": 5.974926647105895e-06,
650
+ "loss": 0.0055,
651
+ "step": 840
652
+ },
653
+ {
654
+ "epoch": 0.24182936804296026,
655
+ "grad_norm": 0.058627255260944366,
656
+ "learning_rate": 6.046056726238109e-06,
657
+ "loss": 0.0054,
658
+ "step": 850
659
+ },
660
+ {
661
+ "epoch": 0.24467441943170098,
662
+ "grad_norm": 0.068621426820755,
663
+ "learning_rate": 6.117186805370321e-06,
664
+ "loss": 0.0065,
665
+ "step": 860
666
+ },
667
+ {
668
+ "epoch": 0.2475194708204417,
669
+ "grad_norm": 0.05066554993391037,
670
+ "learning_rate": 6.188316884502534e-06,
671
+ "loss": 0.0054,
672
+ "step": 870
673
+ },
674
+ {
675
+ "epoch": 0.2503645222091824,
676
+ "grad_norm": 0.05321214348077774,
677
+ "learning_rate": 6.259446963634748e-06,
678
+ "loss": 0.0051,
679
+ "step": 880
680
+ },
681
+ {
682
+ "epoch": 0.2532095735979231,
683
+ "grad_norm": 0.054955266416072845,
684
+ "learning_rate": 6.330577042766961e-06,
685
+ "loss": 0.0057,
686
+ "step": 890
687
+ },
688
+ {
689
+ "epoch": 0.2560546249866638,
690
+ "grad_norm": 0.05507129430770874,
691
+ "learning_rate": 6.401707121899174e-06,
692
+ "loss": 0.0056,
693
+ "step": 900
694
+ },
695
+ {
696
+ "epoch": 0.2588996763754045,
697
+ "grad_norm": 0.05375712364912033,
698
+ "learning_rate": 6.472837201031387e-06,
699
+ "loss": 0.0055,
700
+ "step": 910
701
+ },
702
+ {
703
+ "epoch": 0.26174472776414526,
704
+ "grad_norm": 0.04651631414890289,
705
+ "learning_rate": 6.5439672801636004e-06,
706
+ "loss": 0.0051,
707
+ "step": 920
708
+ },
709
+ {
710
+ "epoch": 0.26458977915288595,
711
+ "grad_norm": 0.042346760630607605,
712
+ "learning_rate": 6.6150973592958125e-06,
713
+ "loss": 0.0055,
714
+ "step": 930
715
+ },
716
+ {
717
+ "epoch": 0.26743483054162664,
718
+ "grad_norm": 0.04895606264472008,
719
+ "learning_rate": 6.6862274384280255e-06,
720
+ "loss": 0.0059,
721
+ "step": 940
722
+ },
723
+ {
724
+ "epoch": 0.2702798819303674,
725
+ "grad_norm": 0.04482842609286308,
726
+ "learning_rate": 6.757357517560239e-06,
727
+ "loss": 0.005,
728
+ "step": 950
729
+ },
730
+ {
731
+ "epoch": 0.2731249333191081,
732
+ "grad_norm": 0.041253965348005295,
733
+ "learning_rate": 6.828487596692452e-06,
734
+ "loss": 0.0058,
735
+ "step": 960
736
+ },
737
+ {
738
+ "epoch": 0.2759699847078488,
739
+ "grad_norm": 0.034503597766160965,
740
+ "learning_rate": 6.899617675824664e-06,
741
+ "loss": 0.005,
742
+ "step": 970
743
+ },
744
+ {
745
+ "epoch": 0.2788150360965895,
746
+ "grad_norm": 0.06278059631586075,
747
+ "learning_rate": 6.970747754956878e-06,
748
+ "loss": 0.0055,
749
+ "step": 980
750
+ },
751
+ {
752
+ "epoch": 0.2816600874853302,
753
+ "grad_norm": 0.04181290045380592,
754
+ "learning_rate": 7.041877834089091e-06,
755
+ "loss": 0.0054,
756
+ "step": 990
757
+ },
758
+ {
759
+ "epoch": 0.2845051388740709,
760
+ "grad_norm": 0.051747649908065796,
761
+ "learning_rate": 7.113007913221304e-06,
762
+ "loss": 0.0055,
763
+ "step": 1000
764
+ },
765
+ {
766
+ "epoch": 0.2845051388740709,
767
+ "eval_loss": 0.03596724942326546,
768
+ "eval_runtime": 9.3729,
769
+ "eval_samples_per_second": 160.036,
770
+ "eval_steps_per_second": 10.029,
771
+ "eval_sts_dev_pearson_cosine": 0.82172548616974,
772
+ "eval_sts_dev_pearson_dot": 0.7364892222173365,
773
+ "eval_sts_dev_pearson_euclidean": 0.8020381417699484,
774
+ "eval_sts_dev_pearson_manhattan": 0.8025412567249582,
775
+ "eval_sts_dev_pearson_max": 0.82172548616974,
776
+ "eval_sts_dev_spearman_cosine": 0.8319127997746368,
777
+ "eval_sts_dev_spearman_dot": 0.7174649408827799,
778
+ "eval_sts_dev_spearman_euclidean": 0.8094012861546513,
779
+ "eval_sts_dev_spearman_manhattan": 0.8112845636960319,
780
+ "eval_sts_dev_spearman_max": 0.8319127997746368,
781
+ "step": 1000
782
+ },
783
+ {
784
+ "epoch": 0.2873501902628116,
785
+ "grad_norm": 0.041287198662757874,
786
+ "learning_rate": 7.184137992353517e-06,
787
+ "loss": 0.0059,
788
+ "step": 1010
789
+ },
790
+ {
791
+ "epoch": 0.2901952416515523,
792
+ "grad_norm": 0.04565191641449928,
793
+ "learning_rate": 7.255268071485731e-06,
794
+ "loss": 0.0049,
795
+ "step": 1020
796
+ },
797
+ {
798
+ "epoch": 0.29304029304029305,
799
+ "grad_norm": 0.049692653119564056,
800
+ "learning_rate": 7.326398150617943e-06,
801
+ "loss": 0.0052,
802
+ "step": 1030
803
+ },
804
+ {
805
+ "epoch": 0.29588534442903375,
806
+ "grad_norm": 0.04820411652326584,
807
+ "learning_rate": 7.397528229750156e-06,
808
+ "loss": 0.0051,
809
+ "step": 1040
810
+ },
811
+ {
812
+ "epoch": 0.29873039581777444,
813
+ "grad_norm": 0.05799231678247452,
814
+ "learning_rate": 7.46865830888237e-06,
815
+ "loss": 0.006,
816
+ "step": 1050
817
+ },
818
+ {
819
+ "epoch": 0.3015754472065152,
820
+ "grad_norm": 0.039241332560777664,
821
+ "learning_rate": 7.5397883880145825e-06,
822
+ "loss": 0.0048,
823
+ "step": 1060
824
+ },
825
+ {
826
+ "epoch": 0.3044204985952559,
827
+ "grad_norm": 0.04567272588610649,
828
+ "learning_rate": 7.610918467146795e-06,
829
+ "loss": 0.0055,
830
+ "step": 1070
831
+ },
832
+ {
833
+ "epoch": 0.3072655499839966,
834
+ "grad_norm": 0.05523727089166641,
835
+ "learning_rate": 7.682048546279008e-06,
836
+ "loss": 0.0052,
837
+ "step": 1080
838
+ },
839
+ {
840
+ "epoch": 0.31011060137273727,
841
+ "grad_norm": 0.046325985342264175,
842
+ "learning_rate": 7.753178625411222e-06,
843
+ "loss": 0.0051,
844
+ "step": 1090
845
+ },
846
+ {
847
+ "epoch": 0.312955652761478,
848
+ "grad_norm": 0.048398926854133606,
849
+ "learning_rate": 7.824308704543434e-06,
850
+ "loss": 0.0051,
851
+ "step": 1100
852
+ },
853
+ {
854
+ "epoch": 0.3158007041502187,
855
+ "grad_norm": 0.04315317049622536,
856
+ "learning_rate": 7.895438783675648e-06,
857
+ "loss": 0.005,
858
+ "step": 1110
859
+ },
860
+ {
861
+ "epoch": 0.3186457555389594,
862
+ "grad_norm": 0.04845454543828964,
863
+ "learning_rate": 7.96656886280786e-06,
864
+ "loss": 0.0054,
865
+ "step": 1120
866
+ },
867
+ {
868
+ "epoch": 0.32149080692770016,
869
+ "grad_norm": 0.049591273069381714,
870
+ "learning_rate": 8.037698941940072e-06,
871
+ "loss": 0.0051,
872
+ "step": 1130
873
+ },
874
+ {
875
+ "epoch": 0.32433585831644085,
876
+ "grad_norm": 0.036491356790065765,
877
+ "learning_rate": 8.108829021072286e-06,
878
+ "loss": 0.0054,
879
+ "step": 1140
880
+ },
881
+ {
882
+ "epoch": 0.32718090970518154,
883
+ "grad_norm": 0.032174497842788696,
884
+ "learning_rate": 8.1799591002045e-06,
885
+ "loss": 0.0056,
886
+ "step": 1150
887
+ },
888
+ {
889
+ "epoch": 0.33002596109392224,
890
+ "grad_norm": 0.050426822155714035,
891
+ "learning_rate": 8.251089179336714e-06,
892
+ "loss": 0.0053,
893
+ "step": 1160
894
+ },
895
+ {
896
+ "epoch": 0.332871012482663,
897
+ "grad_norm": 0.03973342478275299,
898
+ "learning_rate": 8.322219258468926e-06,
899
+ "loss": 0.0052,
900
+ "step": 1170
901
+ },
902
+ {
903
+ "epoch": 0.3357160638714037,
904
+ "grad_norm": 0.051343441009521484,
905
+ "learning_rate": 8.39334933760114e-06,
906
+ "loss": 0.0051,
907
+ "step": 1180
908
+ },
909
+ {
910
+ "epoch": 0.3385611152601444,
911
+ "grad_norm": 0.04074539989233017,
912
+ "learning_rate": 8.464479416733352e-06,
913
+ "loss": 0.0051,
914
+ "step": 1190
915
+ },
916
+ {
917
+ "epoch": 0.3414061666488851,
918
+ "grad_norm": 0.040065567940473557,
919
+ "learning_rate": 8.535609495865564e-06,
920
+ "loss": 0.0048,
921
+ "step": 1200
922
+ },
923
+ {
924
+ "epoch": 0.3442512180376258,
925
+ "grad_norm": 0.054492607712745667,
926
+ "learning_rate": 8.606739574997778e-06,
927
+ "loss": 0.005,
928
+ "step": 1210
929
+ },
930
+ {
931
+ "epoch": 0.3470962694263665,
932
+ "grad_norm": 0.048747118562459946,
933
+ "learning_rate": 8.677869654129991e-06,
934
+ "loss": 0.0055,
935
+ "step": 1220
936
+ },
937
+ {
938
+ "epoch": 0.3499413208151072,
939
+ "grad_norm": 0.03102809563279152,
940
+ "learning_rate": 8.748999733262203e-06,
941
+ "loss": 0.0049,
942
+ "step": 1230
943
+ },
944
+ {
945
+ "epoch": 0.35278637220384795,
946
+ "grad_norm": 0.046874821186065674,
947
+ "learning_rate": 8.820129812394417e-06,
948
+ "loss": 0.0053,
949
+ "step": 1240
950
+ },
951
+ {
952
+ "epoch": 0.35563142359258865,
953
+ "grad_norm": 0.04343179240822792,
954
+ "learning_rate": 8.89125989152663e-06,
955
+ "loss": 0.0052,
956
+ "step": 1250
957
+ },
958
+ {
959
+ "epoch": 0.35563142359258865,
960
+ "eval_loss": 0.036384228616952896,
961
+ "eval_runtime": 9.2709,
962
+ "eval_samples_per_second": 161.796,
963
+ "eval_steps_per_second": 10.139,
964
+ "eval_sts_dev_pearson_cosine": 0.8227235713076437,
965
+ "eval_sts_dev_pearson_dot": 0.7352421859428611,
966
+ "eval_sts_dev_pearson_euclidean": 0.8053003746237307,
967
+ "eval_sts_dev_pearson_manhattan": 0.805677081921409,
968
+ "eval_sts_dev_pearson_max": 0.8227235713076437,
969
+ "eval_sts_dev_spearman_cosine": 0.8330328864427068,
970
+ "eval_sts_dev_spearman_dot": 0.715812823875112,
971
+ "eval_sts_dev_spearman_euclidean": 0.8129674308880657,
972
+ "eval_sts_dev_spearman_manhattan": 0.814635019437101,
973
+ "eval_sts_dev_spearman_max": 0.8330328864427068,
974
+ "step": 1250
975
+ },
976
+ {
977
+ "epoch": 0.35847647498132934,
978
+ "grad_norm": 0.043001964688301086,
979
+ "learning_rate": 8.962389970658843e-06,
980
+ "loss": 0.0051,
981
+ "step": 1260
982
+ },
983
+ {
984
+ "epoch": 0.36132152637007003,
985
+ "grad_norm": 0.03615862876176834,
986
+ "learning_rate": 9.033520049791055e-06,
987
+ "loss": 0.005,
988
+ "step": 1270
989
+ },
990
+ {
991
+ "epoch": 0.3641665777588108,
992
+ "grad_norm": 0.039267782121896744,
993
+ "learning_rate": 9.104650128923269e-06,
994
+ "loss": 0.005,
995
+ "step": 1280
996
+ },
997
+ {
998
+ "epoch": 0.3670116291475515,
999
+ "grad_norm": 0.04287976399064064,
1000
+ "learning_rate": 9.175780208055483e-06,
1001
+ "loss": 0.0045,
1002
+ "step": 1290
1003
+ },
1004
+ {
1005
+ "epoch": 0.36985668053629217,
1006
+ "grad_norm": 0.04853058233857155,
1007
+ "learning_rate": 9.246910287187695e-06,
1008
+ "loss": 0.0055,
1009
+ "step": 1300
1010
+ },
1011
+ {
1012
+ "epoch": 0.3727017319250329,
1013
+ "grad_norm": 0.044756922870874405,
1014
+ "learning_rate": 9.318040366319909e-06,
1015
+ "loss": 0.0049,
1016
+ "step": 1310
1017
+ },
1018
+ {
1019
+ "epoch": 0.3755467833137736,
1020
+ "grad_norm": 0.04728604108095169,
1021
+ "learning_rate": 9.389170445452121e-06,
1022
+ "loss": 0.0049,
1023
+ "step": 1320
1024
+ },
1025
+ {
1026
+ "epoch": 0.3783918347025143,
1027
+ "grad_norm": 0.05135470628738403,
1028
+ "learning_rate": 9.460300524584335e-06,
1029
+ "loss": 0.0053,
1030
+ "step": 1330
1031
+ },
1032
+ {
1033
+ "epoch": 0.381236886091255,
1034
+ "grad_norm": 0.05084382742643356,
1035
+ "learning_rate": 9.531430603716547e-06,
1036
+ "loss": 0.005,
1037
+ "step": 1340
1038
+ },
1039
+ {
1040
+ "epoch": 0.38408193747999575,
1041
+ "grad_norm": 0.04461762309074402,
1042
+ "learning_rate": 9.60256068284876e-06,
1043
+ "loss": 0.0048,
1044
+ "step": 1350
1045
+ },
1046
+ {
1047
+ "epoch": 0.38692698886873644,
1048
+ "grad_norm": 0.04691236838698387,
1049
+ "learning_rate": 9.673690761980974e-06,
1050
+ "loss": 0.0049,
1051
+ "step": 1360
1052
+ },
1053
+ {
1054
+ "epoch": 0.38977204025747714,
1055
+ "grad_norm": 0.05038540065288544,
1056
+ "learning_rate": 9.744820841113186e-06,
1057
+ "loss": 0.0046,
1058
+ "step": 1370
1059
+ },
1060
+ {
1061
+ "epoch": 0.3926170916462179,
1062
+ "grad_norm": 0.037911634892225266,
1063
+ "learning_rate": 9.8159509202454e-06,
1064
+ "loss": 0.0049,
1065
+ "step": 1380
1066
+ },
1067
+ {
1068
+ "epoch": 0.3954621430349586,
1069
+ "grad_norm": 0.03995656967163086,
1070
+ "learning_rate": 9.887080999377612e-06,
1071
+ "loss": 0.0052,
1072
+ "step": 1390
1073
+ },
1074
+ {
1075
+ "epoch": 0.3983071944236993,
1076
+ "grad_norm": 0.03694682940840721,
1077
+ "learning_rate": 9.958211078509824e-06,
1078
+ "loss": 0.005,
1079
+ "step": 1400
1080
+ },
1081
+ {
1082
+ "epoch": 0.40115224581243997,
1083
+ "grad_norm": 0.04984055832028389,
1084
+ "learning_rate": 1.0029341157642038e-05,
1085
+ "loss": 0.0052,
1086
+ "step": 1410
1087
+ },
1088
+ {
1089
+ "epoch": 0.4039972972011807,
1090
+ "grad_norm": 0.045273568481206894,
1091
+ "learning_rate": 1.0100471236774252e-05,
1092
+ "loss": 0.0052,
1093
+ "step": 1420
1094
+ },
1095
+ {
1096
+ "epoch": 0.4068423485899214,
1097
+ "grad_norm": 0.04406927153468132,
1098
+ "learning_rate": 1.0171601315906466e-05,
1099
+ "loss": 0.0045,
1100
+ "step": 1430
1101
+ },
1102
+ {
1103
+ "epoch": 0.4096873999786621,
1104
+ "grad_norm": 0.04544726386666298,
1105
+ "learning_rate": 1.0242731395038678e-05,
1106
+ "loss": 0.0046,
1107
+ "step": 1440
1108
+ },
1109
+ {
1110
+ "epoch": 0.4125324513674028,
1111
+ "grad_norm": 0.05660213530063629,
1112
+ "learning_rate": 1.031386147417089e-05,
1113
+ "loss": 0.0056,
1114
+ "step": 1450
1115
+ },
1116
+ {
1117
+ "epoch": 0.41537750275614355,
1118
+ "grad_norm": 0.03915410116314888,
1119
+ "learning_rate": 1.0384991553303104e-05,
1120
+ "loss": 0.0056,
1121
+ "step": 1460
1122
+ },
1123
+ {
1124
+ "epoch": 0.41822255414488424,
1125
+ "grad_norm": 0.04316354915499687,
1126
+ "learning_rate": 1.0456121632435316e-05,
1127
+ "loss": 0.005,
1128
+ "step": 1470
1129
+ },
1130
+ {
1131
+ "epoch": 0.42106760553362493,
1132
+ "grad_norm": 0.03962288424372673,
1133
+ "learning_rate": 1.052725171156753e-05,
1134
+ "loss": 0.0051,
1135
+ "step": 1480
1136
+ },
1137
+ {
1138
+ "epoch": 0.4239126569223657,
1139
+ "grad_norm": 0.05906355381011963,
1140
+ "learning_rate": 1.0598381790699743e-05,
1141
+ "loss": 0.0049,
1142
+ "step": 1490
1143
+ },
1144
+ {
1145
+ "epoch": 0.4267577083111064,
1146
+ "grad_norm": 0.048688966780900955,
1147
+ "learning_rate": 1.0669511869831957e-05,
1148
+ "loss": 0.0048,
1149
+ "step": 1500
1150
+ },
1151
+ {
1152
+ "epoch": 0.4267577083111064,
1153
+ "eval_loss": 0.037407051771879196,
1154
+ "eval_runtime": 9.34,
1155
+ "eval_samples_per_second": 160.6,
1156
+ "eval_steps_per_second": 10.064,
1157
+ "eval_sts_dev_pearson_cosine": 0.8235580009492447,
1158
+ "eval_sts_dev_pearson_dot": 0.7304409388102885,
1159
+ "eval_sts_dev_pearson_euclidean": 0.80850209692844,
1160
+ "eval_sts_dev_pearson_manhattan": 0.8086421345574696,
1161
+ "eval_sts_dev_pearson_max": 0.8235580009492447,
1162
+ "eval_sts_dev_spearman_cosine": 0.8334282731331559,
1163
+ "eval_sts_dev_spearman_dot": 0.7108658935799598,
1164
+ "eval_sts_dev_spearman_euclidean": 0.8172139025761608,
1165
+ "eval_sts_dev_spearman_manhattan": 0.8184702685018853,
1166
+ "eval_sts_dev_spearman_max": 0.8334282731331559,
1167
+ "step": 1500
1168
+ },
1169
+ {
1170
+ "epoch": 0.42960275969984707,
1171
+ "grad_norm": 0.06282392889261246,
1172
+ "learning_rate": 1.074064194896417e-05,
1173
+ "loss": 0.0053,
1174
+ "step": 1510
1175
+ },
1176
+ {
1177
+ "epoch": 0.43244781108858776,
1178
+ "grad_norm": 0.04287637770175934,
1179
+ "learning_rate": 1.0811772028096381e-05,
1180
+ "loss": 0.0054,
1181
+ "step": 1520
1182
+ },
1183
+ {
1184
+ "epoch": 0.4352928624773285,
1185
+ "grad_norm": 0.03431074693799019,
1186
+ "learning_rate": 1.0882902107228595e-05,
1187
+ "loss": 0.0048,
1188
+ "step": 1530
1189
+ },
1190
+ {
1191
+ "epoch": 0.4381379138660692,
1192
+ "grad_norm": 0.04143288731575012,
1193
+ "learning_rate": 1.0954032186360807e-05,
1194
+ "loss": 0.005,
1195
+ "step": 1540
1196
+ },
1197
+ {
1198
+ "epoch": 0.4409829652548099,
1199
+ "grad_norm": 0.03329918906092644,
1200
+ "learning_rate": 1.1025162265493021e-05,
1201
+ "loss": 0.0045,
1202
+ "step": 1550
1203
+ },
1204
+ {
1205
+ "epoch": 0.44382801664355065,
1206
+ "grad_norm": 0.040052179247140884,
1207
+ "learning_rate": 1.1096292344625235e-05,
1208
+ "loss": 0.0046,
1209
+ "step": 1560
1210
+ },
1211
+ {
1212
+ "epoch": 0.44667306803229134,
1213
+ "grad_norm": 0.03860851004719734,
1214
+ "learning_rate": 1.1167422423757449e-05,
1215
+ "loss": 0.0045,
1216
+ "step": 1570
1217
+ },
1218
+ {
1219
+ "epoch": 0.44951811942103204,
1220
+ "grad_norm": 0.04196810722351074,
1221
+ "learning_rate": 1.1238552502889661e-05,
1222
+ "loss": 0.0049,
1223
+ "step": 1580
1224
+ },
1225
+ {
1226
+ "epoch": 0.45236317080977273,
1227
+ "grad_norm": 0.039289627224206924,
1228
+ "learning_rate": 1.1309682582021873e-05,
1229
+ "loss": 0.0048,
1230
+ "step": 1590
1231
+ },
1232
+ {
1233
+ "epoch": 0.4552082221985135,
1234
+ "grad_norm": 0.0410202331840992,
1235
+ "learning_rate": 1.1380812661154085e-05,
1236
+ "loss": 0.005,
1237
+ "step": 1600
1238
+ },
1239
+ {
1240
+ "epoch": 0.4580532735872542,
1241
+ "grad_norm": 0.042857419699430466,
1242
+ "learning_rate": 1.1451942740286299e-05,
1243
+ "loss": 0.0045,
1244
+ "step": 1610
1245
+ },
1246
+ {
1247
+ "epoch": 0.46089832497599487,
1248
+ "grad_norm": 0.05660340189933777,
1249
+ "learning_rate": 1.1523072819418513e-05,
1250
+ "loss": 0.0049,
1251
+ "step": 1620
1252
+ },
1253
+ {
1254
+ "epoch": 0.4637433763647356,
1255
+ "grad_norm": 0.03311100974678993,
1256
+ "learning_rate": 1.1594202898550726e-05,
1257
+ "loss": 0.0044,
1258
+ "step": 1630
1259
+ },
1260
+ {
1261
+ "epoch": 0.4665884277534763,
1262
+ "grad_norm": 0.040902819484472275,
1263
+ "learning_rate": 1.1665332977682939e-05,
1264
+ "loss": 0.0048,
1265
+ "step": 1640
1266
+ },
1267
+ {
1268
+ "epoch": 0.469433479142217,
1269
+ "grad_norm": 0.03882967308163643,
1270
+ "learning_rate": 1.1736463056815152e-05,
1271
+ "loss": 0.0049,
1272
+ "step": 1650
1273
+ },
1274
+ {
1275
+ "epoch": 0.4722785305309577,
1276
+ "grad_norm": 0.03872678056359291,
1277
+ "learning_rate": 1.1807593135947364e-05,
1278
+ "loss": 0.0048,
1279
+ "step": 1660
1280
+ },
1281
+ {
1282
+ "epoch": 0.47512358191969845,
1283
+ "grad_norm": 0.04378141462802887,
1284
+ "learning_rate": 1.1878723215079577e-05,
1285
+ "loss": 0.0051,
1286
+ "step": 1670
1287
+ },
1288
+ {
1289
+ "epoch": 0.47796863330843914,
1290
+ "grad_norm": 0.03466705232858658,
1291
+ "learning_rate": 1.194985329421179e-05,
1292
+ "loss": 0.0047,
1293
+ "step": 1680
1294
+ },
1295
+ {
1296
+ "epoch": 0.48081368469717983,
1297
+ "grad_norm": 0.03464537858963013,
1298
+ "learning_rate": 1.2020983373344004e-05,
1299
+ "loss": 0.0048,
1300
+ "step": 1690
1301
+ },
1302
+ {
1303
+ "epoch": 0.4836587360859205,
1304
+ "grad_norm": 0.03331427648663521,
1305
+ "learning_rate": 1.2092113452476218e-05,
1306
+ "loss": 0.0047,
1307
+ "step": 1700
1308
+ },
1309
+ {
1310
+ "epoch": 0.4865037874746613,
1311
+ "grad_norm": 0.03817320987582207,
1312
+ "learning_rate": 1.216324353160843e-05,
1313
+ "loss": 0.0044,
1314
+ "step": 1710
1315
+ },
1316
+ {
1317
+ "epoch": 0.48934883886340197,
1318
+ "grad_norm": 0.03969097137451172,
1319
+ "learning_rate": 1.2234373610740642e-05,
1320
+ "loss": 0.0049,
1321
+ "step": 1720
1322
+ },
1323
+ {
1324
+ "epoch": 0.49219389025214266,
1325
+ "grad_norm": 0.035453908145427704,
1326
+ "learning_rate": 1.2305503689872856e-05,
1327
+ "loss": 0.0049,
1328
+ "step": 1730
1329
+ },
1330
+ {
1331
+ "epoch": 0.4950389416408834,
1332
+ "grad_norm": 0.05088822916150093,
1333
+ "learning_rate": 1.2376633769005068e-05,
1334
+ "loss": 0.0051,
1335
+ "step": 1740
1336
+ },
1337
+ {
1338
+ "epoch": 0.4978839930296241,
1339
+ "grad_norm": 0.0713745728135109,
1340
+ "learning_rate": 1.2447763848137282e-05,
1341
+ "loss": 0.0043,
1342
+ "step": 1750
1343
+ },
1344
+ {
1345
+ "epoch": 0.4978839930296241,
1346
+ "eval_loss": 0.039244912564754486,
1347
+ "eval_runtime": 9.3246,
1348
+ "eval_samples_per_second": 160.865,
1349
+ "eval_steps_per_second": 10.081,
1350
+ "eval_sts_dev_pearson_cosine": 0.8238186946290151,
1351
+ "eval_sts_dev_pearson_dot": 0.7239411623613566,
1352
+ "eval_sts_dev_pearson_euclidean": 0.8121715578027333,
1353
+ "eval_sts_dev_pearson_manhattan": 0.8120907547694739,
1354
+ "eval_sts_dev_pearson_max": 0.8238186946290151,
1355
+ "eval_sts_dev_spearman_cosine": 0.835172641675273,
1356
+ "eval_sts_dev_spearman_dot": 0.7062934262157234,
1357
+ "eval_sts_dev_spearman_euclidean": 0.8213874307893159,
1358
+ "eval_sts_dev_spearman_manhattan": 0.8223449132950557,
1359
+ "eval_sts_dev_spearman_max": 0.835172641675273,
1360
+ "step": 1750
1361
+ },
1362
+ {
1363
+ "epoch": 0.5007290444183649,
1364
+ "grad_norm": 0.04352276772260666,
1365
+ "learning_rate": 1.2518893927269496e-05,
1366
+ "loss": 0.0043,
1367
+ "step": 1760
1368
+ },
1369
+ {
1370
+ "epoch": 0.5035740958071055,
1371
+ "grad_norm": 0.03687438368797302,
1372
+ "learning_rate": 1.259002400640171e-05,
1373
+ "loss": 0.0045,
1374
+ "step": 1770
1375
+ },
1376
+ {
1377
+ "epoch": 0.5064191471958462,
1378
+ "grad_norm": 0.03907119855284691,
1379
+ "learning_rate": 1.2661154085533921e-05,
1380
+ "loss": 0.0046,
1381
+ "step": 1780
1382
+ },
1383
+ {
1384
+ "epoch": 0.5092641985845869,
1385
+ "grad_norm": 0.04670177772641182,
1386
+ "learning_rate": 1.2732284164666134e-05,
1387
+ "loss": 0.0042,
1388
+ "step": 1790
1389
+ },
1390
+ {
1391
+ "epoch": 0.5121092499733276,
1392
+ "grad_norm": 0.042154308408498764,
1393
+ "learning_rate": 1.2803414243798347e-05,
1394
+ "loss": 0.0047,
1395
+ "step": 1800
1396
+ },
1397
+ {
1398
+ "epoch": 0.5149543013620683,
1399
+ "grad_norm": 0.049788545817136765,
1400
+ "learning_rate": 1.287454432293056e-05,
1401
+ "loss": 0.0047,
1402
+ "step": 1810
1403
+ },
1404
+ {
1405
+ "epoch": 0.517799352750809,
1406
+ "grad_norm": 0.030743401497602463,
1407
+ "learning_rate": 1.2945674402062773e-05,
1408
+ "loss": 0.0046,
1409
+ "step": 1820
1410
+ },
1411
+ {
1412
+ "epoch": 0.5206444041395498,
1413
+ "grad_norm": 0.03050127439200878,
1414
+ "learning_rate": 1.3016804481194987e-05,
1415
+ "loss": 0.0044,
1416
+ "step": 1830
1417
+ },
1418
+ {
1419
+ "epoch": 0.5234894555282905,
1420
+ "grad_norm": 0.04552179202437401,
1421
+ "learning_rate": 1.3087934560327201e-05,
1422
+ "loss": 0.0046,
1423
+ "step": 1840
1424
+ },
1425
+ {
1426
+ "epoch": 0.5263345069170312,
1427
+ "grad_norm": 0.035957857966423035,
1428
+ "learning_rate": 1.3159064639459413e-05,
1429
+ "loss": 0.0047,
1430
+ "step": 1850
1431
+ },
1432
+ {
1433
+ "epoch": 0.5291795583057719,
1434
+ "grad_norm": 0.042550280690193176,
1435
+ "learning_rate": 1.3230194718591625e-05,
1436
+ "loss": 0.0044,
1437
+ "step": 1860
1438
+ },
1439
+ {
1440
+ "epoch": 0.5320246096945126,
1441
+ "grad_norm": 0.046532485634088516,
1442
+ "learning_rate": 1.3301324797723837e-05,
1443
+ "loss": 0.0047,
1444
+ "step": 1870
1445
+ },
1446
+ {
1447
+ "epoch": 0.5348696610832533,
1448
+ "grad_norm": 0.06353285163640976,
1449
+ "learning_rate": 1.3372454876856051e-05,
1450
+ "loss": 0.0049,
1451
+ "step": 1880
1452
+ },
1453
+ {
1454
+ "epoch": 0.537714712471994,
1455
+ "grad_norm": 0.06265376508235931,
1456
+ "learning_rate": 1.3443584955988265e-05,
1457
+ "loss": 0.0049,
1458
+ "step": 1890
1459
+ },
1460
+ {
1461
+ "epoch": 0.5405597638607348,
1462
+ "grad_norm": 0.03437663987278938,
1463
+ "learning_rate": 1.3514715035120479e-05,
1464
+ "loss": 0.0047,
1465
+ "step": 1900
1466
+ },
1467
+ {
1468
+ "epoch": 0.5434048152494755,
1469
+ "grad_norm": 0.0506511814892292,
1470
+ "learning_rate": 1.358584511425269e-05,
1471
+ "loss": 0.0045,
1472
+ "step": 1910
1473
+ },
1474
+ {
1475
+ "epoch": 0.5462498666382162,
1476
+ "grad_norm": 0.041182588785886765,
1477
+ "learning_rate": 1.3656975193384904e-05,
1478
+ "loss": 0.0044,
1479
+ "step": 1920
1480
+ },
1481
+ {
1482
+ "epoch": 0.5490949180269569,
1483
+ "grad_norm": 0.0319492369890213,
1484
+ "learning_rate": 1.3728105272517117e-05,
1485
+ "loss": 0.0048,
1486
+ "step": 1930
1487
+ },
1488
+ {
1489
+ "epoch": 0.5519399694156976,
1490
+ "grad_norm": 0.03551415726542473,
1491
+ "learning_rate": 1.3799235351649329e-05,
1492
+ "loss": 0.0041,
1493
+ "step": 1940
1494
+ },
1495
+ {
1496
+ "epoch": 0.5547850208044383,
1497
+ "grad_norm": 0.0385081022977829,
1498
+ "learning_rate": 1.3870365430781542e-05,
1499
+ "loss": 0.004,
1500
+ "step": 1950
1501
+ },
1502
+ {
1503
+ "epoch": 0.557630072193179,
1504
+ "grad_norm": 0.03801606595516205,
1505
+ "learning_rate": 1.3941495509913756e-05,
1506
+ "loss": 0.0048,
1507
+ "step": 1960
1508
+ },
1509
+ {
1510
+ "epoch": 0.5604751235819196,
1511
+ "grad_norm": 0.0409582257270813,
1512
+ "learning_rate": 1.401262558904597e-05,
1513
+ "loss": 0.0042,
1514
+ "step": 1970
1515
+ },
1516
+ {
1517
+ "epoch": 0.5633201749706604,
1518
+ "grad_norm": 0.040966786444187164,
1519
+ "learning_rate": 1.4083755668178182e-05,
1520
+ "loss": 0.0048,
1521
+ "step": 1980
1522
+ },
1523
+ {
1524
+ "epoch": 0.5661652263594011,
1525
+ "grad_norm": 0.045747336000204086,
1526
+ "learning_rate": 1.4154885747310394e-05,
1527
+ "loss": 0.0045,
1528
+ "step": 1990
1529
+ },
1530
+ {
1531
+ "epoch": 0.5690102777481418,
1532
+ "grad_norm": 0.04319002479314804,
1533
+ "learning_rate": 1.4226015826442608e-05,
1534
+ "loss": 0.0043,
1535
+ "step": 2000
1536
+ },
1537
+ {
1538
+ "epoch": 0.5690102777481418,
1539
+ "eval_loss": 0.03749217092990875,
1540
+ "eval_runtime": 9.2357,
1541
+ "eval_samples_per_second": 162.414,
1542
+ "eval_steps_per_second": 10.178,
1543
+ "eval_sts_dev_pearson_cosine": 0.8261085791313695,
1544
+ "eval_sts_dev_pearson_dot": 0.7313352904564399,
1545
+ "eval_sts_dev_pearson_euclidean": 0.8139794096198416,
1546
+ "eval_sts_dev_pearson_manhattan": 0.8137749296469377,
1547
+ "eval_sts_dev_pearson_max": 0.8261085791313695,
1548
+ "eval_sts_dev_spearman_cosine": 0.835855895733968,
1549
+ "eval_sts_dev_spearman_dot": 0.7134011062715361,
1550
+ "eval_sts_dev_spearman_euclidean": 0.8230230911277879,
1551
+ "eval_sts_dev_spearman_manhattan": 0.8237999277944462,
1552
+ "eval_sts_dev_spearman_max": 0.835855895733968,
1553
+ "step": 2000
1554
+ },
1555
+ {
1556
+ "epoch": 0.5718553291368825,
1557
+ "grad_norm": 0.04026263579726219,
1558
+ "learning_rate": 1.429714590557482e-05,
1559
+ "loss": 0.005,
1560
+ "step": 2010
1561
+ },
1562
+ {
1563
+ "epoch": 0.5747003805256232,
1564
+ "grad_norm": 0.035753291100263596,
1565
+ "learning_rate": 1.4368275984707034e-05,
1566
+ "loss": 0.0049,
1567
+ "step": 2020
1568
+ },
1569
+ {
1570
+ "epoch": 0.5775454319143639,
1571
+ "grad_norm": 0.047706492245197296,
1572
+ "learning_rate": 1.4439406063839248e-05,
1573
+ "loss": 0.0044,
1574
+ "step": 2030
1575
+ },
1576
+ {
1577
+ "epoch": 0.5803904833031046,
1578
+ "grad_norm": 0.0450417585670948,
1579
+ "learning_rate": 1.4510536142971462e-05,
1580
+ "loss": 0.0045,
1581
+ "step": 2040
1582
+ },
1583
+ {
1584
+ "epoch": 0.5832355346918454,
1585
+ "grad_norm": 0.040127743035554886,
1586
+ "learning_rate": 1.4581666222103674e-05,
1587
+ "loss": 0.0043,
1588
+ "step": 2050
1589
+ },
1590
+ {
1591
+ "epoch": 0.5860805860805861,
1592
+ "grad_norm": 0.03849497064948082,
1593
+ "learning_rate": 1.4652796301235886e-05,
1594
+ "loss": 0.0045,
1595
+ "step": 2060
1596
+ },
1597
+ {
1598
+ "epoch": 0.5889256374693268,
1599
+ "grad_norm": 0.03406572341918945,
1600
+ "learning_rate": 1.47239263803681e-05,
1601
+ "loss": 0.004,
1602
+ "step": 2070
1603
+ },
1604
+ {
1605
+ "epoch": 0.5917706888580675,
1606
+ "grad_norm": 0.03106602467596531,
1607
+ "learning_rate": 1.4795056459500312e-05,
1608
+ "loss": 0.0042,
1609
+ "step": 2080
1610
+ },
1611
+ {
1612
+ "epoch": 0.5946157402468082,
1613
+ "grad_norm": 0.0420919694006443,
1614
+ "learning_rate": 1.4866186538632525e-05,
1615
+ "loss": 0.0044,
1616
+ "step": 2090
1617
+ },
1618
+ {
1619
+ "epoch": 0.5974607916355489,
1620
+ "grad_norm": 0.03017972595989704,
1621
+ "learning_rate": 1.493731661776474e-05,
1622
+ "loss": 0.0043,
1623
+ "step": 2100
1624
+ },
1625
+ {
1626
+ "epoch": 0.6003058430242896,
1627
+ "grad_norm": 0.036625951528549194,
1628
+ "learning_rate": 1.5008446696896953e-05,
1629
+ "loss": 0.0041,
1630
+ "step": 2110
1631
+ },
1632
+ {
1633
+ "epoch": 0.6031508944130304,
1634
+ "grad_norm": 0.043762583285570145,
1635
+ "learning_rate": 1.5079576776029165e-05,
1636
+ "loss": 0.0046,
1637
+ "step": 2120
1638
+ },
1639
+ {
1640
+ "epoch": 0.6059959458017711,
1641
+ "grad_norm": 0.04011479392647743,
1642
+ "learning_rate": 1.5150706855161377e-05,
1643
+ "loss": 0.0048,
1644
+ "step": 2130
1645
+ },
1646
+ {
1647
+ "epoch": 0.6088409971905118,
1648
+ "grad_norm": 0.038334403187036514,
1649
+ "learning_rate": 1.522183693429359e-05,
1650
+ "loss": 0.0048,
1651
+ "step": 2140
1652
+ },
1653
+ {
1654
+ "epoch": 0.6116860485792525,
1655
+ "grad_norm": 0.031101884320378304,
1656
+ "learning_rate": 1.5292967013425805e-05,
1657
+ "loss": 0.0041,
1658
+ "step": 2150
1659
+ },
1660
+ {
1661
+ "epoch": 0.6145310999679932,
1662
+ "grad_norm": 0.03326570242643356,
1663
+ "learning_rate": 1.5364097092558017e-05,
1664
+ "loss": 0.0044,
1665
+ "step": 2160
1666
+ },
1667
+ {
1668
+ "epoch": 0.6173761513567338,
1669
+ "grad_norm": 0.03964686393737793,
1670
+ "learning_rate": 1.543522717169023e-05,
1671
+ "loss": 0.0045,
1672
+ "step": 2170
1673
+ },
1674
+ {
1675
+ "epoch": 0.6202212027454745,
1676
+ "grad_norm": 0.03298981860280037,
1677
+ "learning_rate": 1.5506357250822444e-05,
1678
+ "loss": 0.0044,
1679
+ "step": 2180
1680
+ },
1681
+ {
1682
+ "epoch": 0.6230662541342153,
1683
+ "grad_norm": 0.03743202984333038,
1684
+ "learning_rate": 1.5577487329954657e-05,
1685
+ "loss": 0.0044,
1686
+ "step": 2190
1687
+ },
1688
+ {
1689
+ "epoch": 0.625911305522956,
1690
+ "grad_norm": 0.04800928011536598,
1691
+ "learning_rate": 1.564861740908687e-05,
1692
+ "loss": 0.0046,
1693
+ "step": 2200
1694
+ },
1695
+ {
1696
+ "epoch": 0.6287563569116967,
1697
+ "grad_norm": 0.03906644880771637,
1698
+ "learning_rate": 1.571974748821908e-05,
1699
+ "loss": 0.0048,
1700
+ "step": 2210
1701
+ },
1702
+ {
1703
+ "epoch": 0.6316014083004374,
1704
+ "grad_norm": 0.037462957203388214,
1705
+ "learning_rate": 1.5790877567351296e-05,
1706
+ "loss": 0.0045,
1707
+ "step": 2220
1708
+ },
1709
+ {
1710
+ "epoch": 0.6344464596891781,
1711
+ "grad_norm": 0.037155114114284515,
1712
+ "learning_rate": 1.586200764648351e-05,
1713
+ "loss": 0.004,
1714
+ "step": 2230
1715
+ },
1716
+ {
1717
+ "epoch": 0.6372915110779188,
1718
+ "grad_norm": 0.034092146903276443,
1719
+ "learning_rate": 1.593313772561572e-05,
1720
+ "loss": 0.0041,
1721
+ "step": 2240
1722
+ },
1723
+ {
1724
+ "epoch": 0.6401365624666595,
1725
+ "grad_norm": 0.03896261751651764,
1726
+ "learning_rate": 1.6004267804747936e-05,
1727
+ "loss": 0.0044,
1728
+ "step": 2250
1729
+ },
1730
+ {
1731
+ "epoch": 0.6401365624666595,
1732
+ "eval_loss": 0.039099473506212234,
1733
+ "eval_runtime": 9.2514,
1734
+ "eval_samples_per_second": 162.137,
1735
+ "eval_steps_per_second": 10.161,
1736
+ "eval_sts_dev_pearson_cosine": 0.8272187422275056,
1737
+ "eval_sts_dev_pearson_dot": 0.7283947785380285,
1738
+ "eval_sts_dev_pearson_euclidean": 0.815899998489852,
1739
+ "eval_sts_dev_pearson_manhattan": 0.81565618483905,
1740
+ "eval_sts_dev_pearson_max": 0.8272187422275056,
1741
+ "eval_sts_dev_spearman_cosine": 0.836903996661572,
1742
+ "eval_sts_dev_spearman_dot": 0.7095745027354784,
1743
+ "eval_sts_dev_spearman_euclidean": 0.8251814652974586,
1744
+ "eval_sts_dev_spearman_manhattan": 0.8261966578035025,
1745
+ "eval_sts_dev_spearman_max": 0.836903996661572,
1746
+ "step": 2250
1747
+ },
1748
+ {
1749
+ "epoch": 0.6429816138554003,
1750
+ "grad_norm": 0.03634743019938469,
1751
+ "learning_rate": 1.6075397883880145e-05,
1752
+ "loss": 0.0044,
1753
+ "step": 2260
1754
+ },
1755
+ {
1756
+ "epoch": 0.645826665244141,
1757
+ "grad_norm": 0.04395711049437523,
1758
+ "learning_rate": 1.614652796301236e-05,
1759
+ "loss": 0.0045,
1760
+ "step": 2270
1761
+ },
1762
+ {
1763
+ "epoch": 0.6486717166328817,
1764
+ "grad_norm": 0.0334044024348259,
1765
+ "learning_rate": 1.6217658042144572e-05,
1766
+ "loss": 0.0041,
1767
+ "step": 2280
1768
+ },
1769
+ {
1770
+ "epoch": 0.6515167680216224,
1771
+ "grad_norm": 0.03832104057073593,
1772
+ "learning_rate": 1.6288788121276788e-05,
1773
+ "loss": 0.0042,
1774
+ "step": 2290
1775
+ },
1776
+ {
1777
+ "epoch": 0.6543618194103631,
1778
+ "grad_norm": 0.04635302349925041,
1779
+ "learning_rate": 1.6359918200409e-05,
1780
+ "loss": 0.0043,
1781
+ "step": 2300
1782
+ },
1783
+ {
1784
+ "epoch": 0.6572068707991038,
1785
+ "grad_norm": 0.03036418743431568,
1786
+ "learning_rate": 1.6431048279541212e-05,
1787
+ "loss": 0.004,
1788
+ "step": 2310
1789
+ },
1790
+ {
1791
+ "epoch": 0.6600519221878445,
1792
+ "grad_norm": 0.025319894775748253,
1793
+ "learning_rate": 1.6502178358673427e-05,
1794
+ "loss": 0.0042,
1795
+ "step": 2320
1796
+ },
1797
+ {
1798
+ "epoch": 0.6628969735765853,
1799
+ "grad_norm": 0.03628432750701904,
1800
+ "learning_rate": 1.6573308437805636e-05,
1801
+ "loss": 0.0041,
1802
+ "step": 2330
1803
+ },
1804
+ {
1805
+ "epoch": 0.665742024965326,
1806
+ "grad_norm": 0.03586098179221153,
1807
+ "learning_rate": 1.664443851693785e-05,
1808
+ "loss": 0.0045,
1809
+ "step": 2340
1810
+ },
1811
+ {
1812
+ "epoch": 0.6685870763540667,
1813
+ "grad_norm": 0.03873738646507263,
1814
+ "learning_rate": 1.6715568596070064e-05,
1815
+ "loss": 0.0045,
1816
+ "step": 2350
1817
+ },
1818
+ {
1819
+ "epoch": 0.6714321277428074,
1820
+ "grad_norm": 0.038827184587717056,
1821
+ "learning_rate": 1.678669867520228e-05,
1822
+ "loss": 0.0042,
1823
+ "step": 2360
1824
+ },
1825
+ {
1826
+ "epoch": 0.674277179131548,
1827
+ "grad_norm": 0.03740520030260086,
1828
+ "learning_rate": 1.685782875433449e-05,
1829
+ "loss": 0.0045,
1830
+ "step": 2370
1831
+ },
1832
+ {
1833
+ "epoch": 0.6771222305202887,
1834
+ "grad_norm": 0.03753795102238655,
1835
+ "learning_rate": 1.6928958833466703e-05,
1836
+ "loss": 0.0044,
1837
+ "step": 2380
1838
+ },
1839
+ {
1840
+ "epoch": 0.6799672819090294,
1841
+ "grad_norm": 0.04462343081831932,
1842
+ "learning_rate": 1.700008891259892e-05,
1843
+ "loss": 0.0044,
1844
+ "step": 2390
1845
+ },
1846
+ {
1847
+ "epoch": 0.6828123332977702,
1848
+ "grad_norm": 0.0419158972799778,
1849
+ "learning_rate": 1.7071218991731128e-05,
1850
+ "loss": 0.0041,
1851
+ "step": 2400
1852
+ },
1853
+ {
1854
+ "epoch": 0.6856573846865109,
1855
+ "grad_norm": 0.04727786406874657,
1856
+ "learning_rate": 1.7142349070863343e-05,
1857
+ "loss": 0.0045,
1858
+ "step": 2410
1859
+ },
1860
+ {
1861
+ "epoch": 0.6885024360752516,
1862
+ "grad_norm": 0.03223787620663643,
1863
+ "learning_rate": 1.7213479149995555e-05,
1864
+ "loss": 0.0046,
1865
+ "step": 2420
1866
+ },
1867
+ {
1868
+ "epoch": 0.6913474874639923,
1869
+ "grad_norm": 0.04408884793519974,
1870
+ "learning_rate": 1.7284609229127767e-05,
1871
+ "loss": 0.0041,
1872
+ "step": 2430
1873
+ },
1874
+ {
1875
+ "epoch": 0.694192538852733,
1876
+ "grad_norm": 0.039247192442417145,
1877
+ "learning_rate": 1.7355739308259983e-05,
1878
+ "loss": 0.0048,
1879
+ "step": 2440
1880
+ },
1881
+ {
1882
+ "epoch": 0.6970375902414737,
1883
+ "grad_norm": 0.032358501106500626,
1884
+ "learning_rate": 1.7426869387392195e-05,
1885
+ "loss": 0.0041,
1886
+ "step": 2450
1887
+ },
1888
+ {
1889
+ "epoch": 0.6998826416302144,
1890
+ "grad_norm": 0.039968185126781464,
1891
+ "learning_rate": 1.7497999466524407e-05,
1892
+ "loss": 0.0043,
1893
+ "step": 2460
1894
+ },
1895
+ {
1896
+ "epoch": 0.7027276930189551,
1897
+ "grad_norm": 0.038930464535951614,
1898
+ "learning_rate": 1.756912954565662e-05,
1899
+ "loss": 0.0043,
1900
+ "step": 2470
1901
+ },
1902
+ {
1903
+ "epoch": 0.7055727444076959,
1904
+ "grad_norm": 0.03470390662550926,
1905
+ "learning_rate": 1.7640259624788835e-05,
1906
+ "loss": 0.0037,
1907
+ "step": 2480
1908
+ },
1909
+ {
1910
+ "epoch": 0.7084177957964366,
1911
+ "grad_norm": 0.03886763006448746,
1912
+ "learning_rate": 1.7711389703921047e-05,
1913
+ "loss": 0.0042,
1914
+ "step": 2490
1915
+ },
1916
+ {
1917
+ "epoch": 0.7112628471851773,
1918
+ "grad_norm": 0.032276757061481476,
1919
+ "learning_rate": 1.778251978305326e-05,
1920
+ "loss": 0.0043,
1921
+ "step": 2500
1922
+ },
1923
+ {
1924
+ "epoch": 0.7112628471851773,
1925
+ "eval_loss": 0.04045585170388222,
1926
+ "eval_runtime": 9.2659,
1927
+ "eval_samples_per_second": 161.885,
1928
+ "eval_steps_per_second": 10.145,
1929
+ "eval_sts_dev_pearson_cosine": 0.8260847942632407,
1930
+ "eval_sts_dev_pearson_dot": 0.7260943559708607,
1931
+ "eval_sts_dev_pearson_euclidean": 0.8155324932216388,
1932
+ "eval_sts_dev_pearson_manhattan": 0.8152500275551733,
1933
+ "eval_sts_dev_pearson_max": 0.8260847942632407,
1934
+ "eval_sts_dev_spearman_cosine": 0.8365382060177997,
1935
+ "eval_sts_dev_spearman_dot": 0.7099426230017294,
1936
+ "eval_sts_dev_spearman_euclidean": 0.8247320291703734,
1937
+ "eval_sts_dev_spearman_manhattan": 0.8257079061230801,
1938
+ "eval_sts_dev_spearman_max": 0.8365382060177997,
1939
+ "step": 2500
1940
+ },
1941
+ {
1942
+ "epoch": 0.714107898573918,
1943
+ "grad_norm": 0.04151437059044838,
1944
+ "learning_rate": 1.7853649862185474e-05,
1945
+ "loss": 0.0045,
1946
+ "step": 2510
1947
+ },
1948
+ {
1949
+ "epoch": 0.7169529499626587,
1950
+ "grad_norm": 0.04570092633366585,
1951
+ "learning_rate": 1.7924779941317686e-05,
1952
+ "loss": 0.0044,
1953
+ "step": 2520
1954
+ },
1955
+ {
1956
+ "epoch": 0.7197980013513994,
1957
+ "grad_norm": 0.03871253505349159,
1958
+ "learning_rate": 1.79959100204499e-05,
1959
+ "loss": 0.0042,
1960
+ "step": 2530
1961
+ },
1962
+ {
1963
+ "epoch": 0.7226430527401401,
1964
+ "grad_norm": 0.04794856533408165,
1965
+ "learning_rate": 1.806704009958211e-05,
1966
+ "loss": 0.0042,
1967
+ "step": 2540
1968
+ },
1969
+ {
1970
+ "epoch": 0.7254881041288809,
1971
+ "grad_norm": 0.03689658269286156,
1972
+ "learning_rate": 1.8138170178714326e-05,
1973
+ "loss": 0.0041,
1974
+ "step": 2550
1975
+ },
1976
+ {
1977
+ "epoch": 0.7283331555176216,
1978
+ "grad_norm": 0.04062201827764511,
1979
+ "learning_rate": 1.8209300257846538e-05,
1980
+ "loss": 0.0042,
1981
+ "step": 2560
1982
+ },
1983
+ {
1984
+ "epoch": 0.7311782069063623,
1985
+ "grad_norm": 0.04463518038392067,
1986
+ "learning_rate": 1.828043033697875e-05,
1987
+ "loss": 0.0041,
1988
+ "step": 2570
1989
+ },
1990
+ {
1991
+ "epoch": 0.734023258295103,
1992
+ "grad_norm": 0.040007706731557846,
1993
+ "learning_rate": 1.8351560416110966e-05,
1994
+ "loss": 0.0042,
1995
+ "step": 2580
1996
+ },
1997
+ {
1998
+ "epoch": 0.7368683096838436,
1999
+ "grad_norm": 0.0314442440867424,
2000
+ "learning_rate": 1.8422690495243178e-05,
2001
+ "loss": 0.0041,
2002
+ "step": 2590
2003
+ },
2004
+ {
2005
+ "epoch": 0.7397133610725843,
2006
+ "grad_norm": 0.0411388985812664,
2007
+ "learning_rate": 1.849382057437539e-05,
2008
+ "loss": 0.0047,
2009
+ "step": 2600
2010
+ },
2011
+ {
2012
+ "epoch": 0.742558412461325,
2013
+ "grad_norm": 0.04951240494847298,
2014
+ "learning_rate": 1.8564950653507602e-05,
2015
+ "loss": 0.0038,
2016
+ "step": 2610
2017
+ },
2018
+ {
2019
+ "epoch": 0.7454034638500658,
2020
+ "grad_norm": 0.06948298215866089,
2021
+ "learning_rate": 1.8636080732639818e-05,
2022
+ "loss": 0.0041,
2023
+ "step": 2620
2024
+ },
2025
+ {
2026
+ "epoch": 0.7482485152388065,
2027
+ "grad_norm": 0.0375785268843174,
2028
+ "learning_rate": 1.870721081177203e-05,
2029
+ "loss": 0.0042,
2030
+ "step": 2630
2031
+ },
2032
+ {
2033
+ "epoch": 0.7510935666275472,
2034
+ "grad_norm": 0.04124037176370621,
2035
+ "learning_rate": 1.8778340890904242e-05,
2036
+ "loss": 0.0042,
2037
+ "step": 2640
2038
+ },
2039
+ {
2040
+ "epoch": 0.7539386180162879,
2041
+ "grad_norm": 0.03421395272016525,
2042
+ "learning_rate": 1.8849470970036457e-05,
2043
+ "loss": 0.0042,
2044
+ "step": 2650
2045
+ },
2046
+ {
2047
+ "epoch": 0.7567836694050286,
2048
+ "grad_norm": 0.03397843614220619,
2049
+ "learning_rate": 1.892060104916867e-05,
2050
+ "loss": 0.0041,
2051
+ "step": 2660
2052
+ },
2053
+ {
2054
+ "epoch": 0.7596287207937693,
2055
+ "grad_norm": 0.039120275527238846,
2056
+ "learning_rate": 1.899173112830088e-05,
2057
+ "loss": 0.0042,
2058
+ "step": 2670
2059
+ },
2060
+ {
2061
+ "epoch": 0.76247377218251,
2062
+ "grad_norm": 0.04069869592785835,
2063
+ "learning_rate": 1.9062861207433093e-05,
2064
+ "loss": 0.0044,
2065
+ "step": 2680
2066
+ },
2067
+ {
2068
+ "epoch": 0.7653188235712508,
2069
+ "grad_norm": 0.03612830862402916,
2070
+ "learning_rate": 1.913399128656531e-05,
2071
+ "loss": 0.0039,
2072
+ "step": 2690
2073
+ },
2074
+ {
2075
+ "epoch": 0.7681638749599915,
2076
+ "grad_norm": 0.034753765910863876,
2077
+ "learning_rate": 1.920512136569752e-05,
2078
+ "loss": 0.0037,
2079
+ "step": 2700
2080
+ },
2081
+ {
2082
+ "epoch": 0.7710089263487322,
2083
+ "grad_norm": 0.05360863357782364,
2084
+ "learning_rate": 1.9276251444829733e-05,
2085
+ "loss": 0.0044,
2086
+ "step": 2710
2087
+ },
2088
+ {
2089
+ "epoch": 0.7738539777374729,
2090
+ "grad_norm": 0.04032083973288536,
2091
+ "learning_rate": 1.934738152396195e-05,
2092
+ "loss": 0.0043,
2093
+ "step": 2720
2094
+ },
2095
+ {
2096
+ "epoch": 0.7766990291262136,
2097
+ "grad_norm": 0.0377604104578495,
2098
+ "learning_rate": 1.9418511603094157e-05,
2099
+ "loss": 0.0042,
2100
+ "step": 2730
2101
+ },
2102
+ {
2103
+ "epoch": 0.7795440805149543,
2104
+ "grad_norm": 0.03419530391693115,
2105
+ "learning_rate": 1.9489641682226373e-05,
2106
+ "loss": 0.0041,
2107
+ "step": 2740
2108
+ },
2109
+ {
2110
+ "epoch": 0.782389131903695,
2111
+ "grad_norm": 0.040193505585193634,
2112
+ "learning_rate": 1.9560771761358585e-05,
2113
+ "loss": 0.0039,
2114
+ "step": 2750
2115
+ },
2116
+ {
2117
+ "epoch": 0.782389131903695,
2118
+ "eval_loss": 0.038744423538446426,
2119
+ "eval_runtime": 9.2929,
2120
+ "eval_samples_per_second": 161.414,
2121
+ "eval_steps_per_second": 10.115,
2122
+ "eval_sts_dev_pearson_cosine": 0.8299324608581861,
2123
+ "eval_sts_dev_pearson_dot": 0.7176863363076469,
2124
+ "eval_sts_dev_pearson_euclidean": 0.8182959597932262,
2125
+ "eval_sts_dev_pearson_manhattan": 0.8182320853696063,
2126
+ "eval_sts_dev_pearson_max": 0.8299324608581861,
2127
+ "eval_sts_dev_spearman_cosine": 0.8375751383739057,
2128
+ "eval_sts_dev_spearman_dot": 0.702295742543807,
2129
+ "eval_sts_dev_spearman_euclidean": 0.8274009374359347,
2130
+ "eval_sts_dev_spearman_manhattan": 0.8284301537034146,
2131
+ "eval_sts_dev_spearman_max": 0.8375751383739057,
2132
+ "step": 2750
2133
+ },
2134
+ {
2135
+ "epoch": 0.7852341832924358,
2136
+ "grad_norm": 0.0465560108423233,
2137
+ "learning_rate": 1.96319018404908e-05,
2138
+ "loss": 0.0047,
2139
+ "step": 2760
2140
+ },
2141
+ {
2142
+ "epoch": 0.7880792346811765,
2143
+ "grad_norm": 0.04297973960638046,
2144
+ "learning_rate": 1.9703031919623013e-05,
2145
+ "loss": 0.004,
2146
+ "step": 2770
2147
+ },
2148
+ {
2149
+ "epoch": 0.7909242860699172,
2150
+ "grad_norm": 0.03678404539823532,
2151
+ "learning_rate": 1.9774161998755225e-05,
2152
+ "loss": 0.0039,
2153
+ "step": 2780
2154
+ },
2155
+ {
2156
+ "epoch": 0.7937693374586579,
2157
+ "grad_norm": 0.05512017384171486,
2158
+ "learning_rate": 1.984529207788744e-05,
2159
+ "loss": 0.0039,
2160
+ "step": 2790
2161
+ },
2162
+ {
2163
+ "epoch": 0.7966143888473985,
2164
+ "grad_norm": 0.03251376003026962,
2165
+ "learning_rate": 1.991642215701965e-05,
2166
+ "loss": 0.0039,
2167
+ "step": 2800
2168
+ },
2169
+ {
2170
+ "epoch": 0.7994594402361392,
2171
+ "grad_norm": 0.04584107547998428,
2172
+ "learning_rate": 1.9987552236151864e-05,
2173
+ "loss": 0.0039,
2174
+ "step": 2810
2175
+ },
2176
+ {
2177
+ "epoch": 0.8023044916248799,
2178
+ "grad_norm": 0.03864232823252678,
2179
+ "learning_rate": 2.0058682315284076e-05,
2180
+ "loss": 0.0039,
2181
+ "step": 2820
2182
+ },
2183
+ {
2184
+ "epoch": 0.8051495430136207,
2185
+ "grad_norm": 0.038527827709913254,
2186
+ "learning_rate": 2.0129812394416292e-05,
2187
+ "loss": 0.0041,
2188
+ "step": 2830
2189
+ },
2190
+ {
2191
+ "epoch": 0.8079945944023614,
2192
+ "grad_norm": 0.02851305902004242,
2193
+ "learning_rate": 2.0200942473548504e-05,
2194
+ "loss": 0.0037,
2195
+ "step": 2840
2196
+ },
2197
+ {
2198
+ "epoch": 0.8108396457911021,
2199
+ "grad_norm": 0.02903963066637516,
2200
+ "learning_rate": 2.0272072552680716e-05,
2201
+ "loss": 0.0044,
2202
+ "step": 2850
2203
+ },
2204
+ {
2205
+ "epoch": 0.8136846971798428,
2206
+ "grad_norm": 0.03449387475848198,
2207
+ "learning_rate": 2.034320263181293e-05,
2208
+ "loss": 0.0043,
2209
+ "step": 2860
2210
+ },
2211
+ {
2212
+ "epoch": 0.8165297485685835,
2213
+ "grad_norm": 0.03411369025707245,
2214
+ "learning_rate": 2.041433271094514e-05,
2215
+ "loss": 0.0041,
2216
+ "step": 2870
2217
+ },
2218
+ {
2219
+ "epoch": 0.8193747999573242,
2220
+ "grad_norm": 0.040192630141973495,
2221
+ "learning_rate": 2.0485462790077356e-05,
2222
+ "loss": 0.0043,
2223
+ "step": 2880
2224
+ },
2225
+ {
2226
+ "epoch": 0.8222198513460649,
2227
+ "grad_norm": 0.031003862619400024,
2228
+ "learning_rate": 2.0556592869209568e-05,
2229
+ "loss": 0.0039,
2230
+ "step": 2890
2231
+ },
2232
+ {
2233
+ "epoch": 0.8250649027348056,
2234
+ "grad_norm": 0.04759204015135765,
2235
+ "learning_rate": 2.062772294834178e-05,
2236
+ "loss": 0.0041,
2237
+ "step": 2900
2238
+ },
2239
+ {
2240
+ "epoch": 0.8279099541235464,
2241
+ "grad_norm": 0.036447394639253616,
2242
+ "learning_rate": 2.0698853027473996e-05,
2243
+ "loss": 0.0044,
2244
+ "step": 2910
2245
+ },
2246
+ {
2247
+ "epoch": 0.8307550055122871,
2248
+ "grad_norm": 0.04358125478029251,
2249
+ "learning_rate": 2.0769983106606208e-05,
2250
+ "loss": 0.004,
2251
+ "step": 2920
2252
+ },
2253
+ {
2254
+ "epoch": 0.8336000569010278,
2255
+ "grad_norm": 0.038935497403144836,
2256
+ "learning_rate": 2.0841113185738423e-05,
2257
+ "loss": 0.0042,
2258
+ "step": 2930
2259
+ },
2260
+ {
2261
+ "epoch": 0.8364451082897685,
2262
+ "grad_norm": 0.03385254368185997,
2263
+ "learning_rate": 2.0912243264870632e-05,
2264
+ "loss": 0.0039,
2265
+ "step": 2940
2266
+ },
2267
+ {
2268
+ "epoch": 0.8392901596785092,
2269
+ "grad_norm": 0.030839361250400543,
2270
+ "learning_rate": 2.0983373344002847e-05,
2271
+ "loss": 0.004,
2272
+ "step": 2950
2273
+ },
2274
+ {
2275
+ "epoch": 0.8421352110672499,
2276
+ "grad_norm": 0.03932899981737137,
2277
+ "learning_rate": 2.105450342313506e-05,
2278
+ "loss": 0.0042,
2279
+ "step": 2960
2280
+ },
2281
+ {
2282
+ "epoch": 0.8449802624559906,
2283
+ "grad_norm": 0.04810210317373276,
2284
+ "learning_rate": 2.112563350226727e-05,
2285
+ "loss": 0.004,
2286
+ "step": 2970
2287
+ },
2288
+ {
2289
+ "epoch": 0.8478253138447314,
2290
+ "grad_norm": 0.03614366054534912,
2291
+ "learning_rate": 2.1196763581399487e-05,
2292
+ "loss": 0.0039,
2293
+ "step": 2980
2294
+ },
2295
+ {
2296
+ "epoch": 0.8506703652334721,
2297
+ "grad_norm": 0.03652822598814964,
2298
+ "learning_rate": 2.12678936605317e-05,
2299
+ "loss": 0.0037,
2300
+ "step": 2990
2301
+ },
2302
+ {
2303
+ "epoch": 0.8535154166222128,
2304
+ "grad_norm": 0.04175693169236183,
2305
+ "learning_rate": 2.1339023739663915e-05,
2306
+ "loss": 0.0039,
2307
+ "step": 3000
2308
+ },
2309
+ {
2310
+ "epoch": 0.8535154166222128,
2311
+ "eval_loss": 0.038587357848882675,
2312
+ "eval_runtime": 9.2854,
2313
+ "eval_samples_per_second": 161.545,
2314
+ "eval_steps_per_second": 10.123,
2315
+ "eval_sts_dev_pearson_cosine": 0.8307702264806358,
2316
+ "eval_sts_dev_pearson_dot": 0.7156790570711026,
2317
+ "eval_sts_dev_pearson_euclidean": 0.819779764199181,
2318
+ "eval_sts_dev_pearson_manhattan": 0.8197553048460411,
2319
+ "eval_sts_dev_pearson_max": 0.8307702264806358,
2320
+ "eval_sts_dev_spearman_cosine": 0.8386149894761288,
2321
+ "eval_sts_dev_spearman_dot": 0.7013839487919926,
2322
+ "eval_sts_dev_spearman_euclidean": 0.8286781992701565,
2323
+ "eval_sts_dev_spearman_manhattan": 0.8296307452835905,
2324
+ "eval_sts_dev_spearman_max": 0.8386149894761288,
2325
+ "step": 3000
2326
+ },
2327
+ {
2328
+ "epoch": 0.8563604680109534,
2329
+ "grad_norm": 0.035189032554626465,
2330
+ "learning_rate": 2.1410153818796123e-05,
2331
+ "loss": 0.0041,
2332
+ "step": 3010
2333
+ },
2334
+ {
2335
+ "epoch": 0.8592055193996941,
2336
+ "grad_norm": 0.030665524303913116,
2337
+ "learning_rate": 2.148128389792834e-05,
2338
+ "loss": 0.0043,
2339
+ "step": 3020
2340
+ },
2341
+ {
2342
+ "epoch": 0.8620505707884348,
2343
+ "grad_norm": 0.03555646166205406,
2344
+ "learning_rate": 2.155241397706055e-05,
2345
+ "loss": 0.0041,
2346
+ "step": 3030
2347
+ },
2348
+ {
2349
+ "epoch": 0.8648956221771755,
2350
+ "grad_norm": 0.04289074242115021,
2351
+ "learning_rate": 2.1623544056192763e-05,
2352
+ "loss": 0.0041,
2353
+ "step": 3040
2354
+ },
2355
+ {
2356
+ "epoch": 0.8677406735659163,
2357
+ "grad_norm": 0.03874215856194496,
2358
+ "learning_rate": 2.169467413532498e-05,
2359
+ "loss": 0.0043,
2360
+ "step": 3050
2361
+ },
2362
+ {
2363
+ "epoch": 0.870585724954657,
2364
+ "grad_norm": 0.040797311812639236,
2365
+ "learning_rate": 2.176580421445719e-05,
2366
+ "loss": 0.0042,
2367
+ "step": 3060
2368
+ },
2369
+ {
2370
+ "epoch": 0.8734307763433977,
2371
+ "grad_norm": 0.043382417410612106,
2372
+ "learning_rate": 2.1836934293589406e-05,
2373
+ "loss": 0.0039,
2374
+ "step": 3070
2375
+ },
2376
+ {
2377
+ "epoch": 0.8762758277321384,
2378
+ "grad_norm": 0.03589191287755966,
2379
+ "learning_rate": 2.1908064372721615e-05,
2380
+ "loss": 0.004,
2381
+ "step": 3080
2382
+ },
2383
+ {
2384
+ "epoch": 0.8791208791208791,
2385
+ "grad_norm": 0.030675871297717094,
2386
+ "learning_rate": 2.197919445185383e-05,
2387
+ "loss": 0.0039,
2388
+ "step": 3090
2389
+ },
2390
+ {
2391
+ "epoch": 0.8819659305096198,
2392
+ "grad_norm": 0.03458893299102783,
2393
+ "learning_rate": 2.2050324530986042e-05,
2394
+ "loss": 0.0039,
2395
+ "step": 3100
2396
+ },
2397
+ {
2398
+ "epoch": 0.8848109818983605,
2399
+ "grad_norm": 0.03687838092446327,
2400
+ "learning_rate": 2.2121454610118254e-05,
2401
+ "loss": 0.004,
2402
+ "step": 3110
2403
+ },
2404
+ {
2405
+ "epoch": 0.8876560332871013,
2406
+ "grad_norm": 0.03154761716723442,
2407
+ "learning_rate": 2.219258468925047e-05,
2408
+ "loss": 0.0039,
2409
+ "step": 3120
2410
+ },
2411
+ {
2412
+ "epoch": 0.890501084675842,
2413
+ "grad_norm": 0.036181699484586716,
2414
+ "learning_rate": 2.2263714768382682e-05,
2415
+ "loss": 0.0038,
2416
+ "step": 3130
2417
+ },
2418
+ {
2419
+ "epoch": 0.8933461360645827,
2420
+ "grad_norm": 0.036393746733665466,
2421
+ "learning_rate": 2.2334844847514898e-05,
2422
+ "loss": 0.0036,
2423
+ "step": 3140
2424
+ },
2425
+ {
2426
+ "epoch": 0.8961911874533234,
2427
+ "grad_norm": 0.04613763466477394,
2428
+ "learning_rate": 2.2405974926647106e-05,
2429
+ "loss": 0.0039,
2430
+ "step": 3150
2431
+ },
2432
+ {
2433
+ "epoch": 0.8990362388420641,
2434
+ "grad_norm": 0.028627606108784676,
2435
+ "learning_rate": 2.2477105005779322e-05,
2436
+ "loss": 0.0039,
2437
+ "step": 3160
2438
+ },
2439
+ {
2440
+ "epoch": 0.9018812902308048,
2441
+ "grad_norm": 0.03881096839904785,
2442
+ "learning_rate": 2.2548235084911534e-05,
2443
+ "loss": 0.0038,
2444
+ "step": 3170
2445
+ },
2446
+ {
2447
+ "epoch": 0.9047263416195455,
2448
+ "grad_norm": 0.030927816405892372,
2449
+ "learning_rate": 2.2619365164043746e-05,
2450
+ "loss": 0.0039,
2451
+ "step": 3180
2452
+ },
2453
+ {
2454
+ "epoch": 0.9075713930082863,
2455
+ "grad_norm": 0.033624544739723206,
2456
+ "learning_rate": 2.269049524317596e-05,
2457
+ "loss": 0.0041,
2458
+ "step": 3190
2459
+ },
2460
+ {
2461
+ "epoch": 0.910416444397027,
2462
+ "grad_norm": 0.03509572893381119,
2463
+ "learning_rate": 2.276162532230817e-05,
2464
+ "loss": 0.004,
2465
+ "step": 3200
2466
+ },
2467
+ {
2468
+ "epoch": 0.9132614957857677,
2469
+ "grad_norm": 0.03826339542865753,
2470
+ "learning_rate": 2.283275540144039e-05,
2471
+ "loss": 0.0041,
2472
+ "step": 3210
2473
+ },
2474
+ {
2475
+ "epoch": 0.9161065471745083,
2476
+ "grad_norm": 0.033389002084732056,
2477
+ "learning_rate": 2.2903885480572598e-05,
2478
+ "loss": 0.0042,
2479
+ "step": 3220
2480
+ },
2481
+ {
2482
+ "epoch": 0.918951598563249,
2483
+ "grad_norm": 0.03484668210148811,
2484
+ "learning_rate": 2.2975015559704813e-05,
2485
+ "loss": 0.004,
2486
+ "step": 3230
2487
+ },
2488
+ {
2489
+ "epoch": 0.9217966499519897,
2490
+ "grad_norm": 0.0346749946475029,
2491
+ "learning_rate": 2.3046145638837025e-05,
2492
+ "loss": 0.0041,
2493
+ "step": 3240
2494
+ },
2495
+ {
2496
+ "epoch": 0.9246417013407304,
2497
+ "grad_norm": 0.031990326941013336,
2498
+ "learning_rate": 2.3117275717969237e-05,
2499
+ "loss": 0.0041,
2500
+ "step": 3250
2501
+ },
2502
+ {
2503
+ "epoch": 0.9246417013407304,
2504
+ "eval_loss": 0.04199736937880516,
2505
+ "eval_runtime": 9.3223,
2506
+ "eval_samples_per_second": 160.904,
2507
+ "eval_steps_per_second": 10.083,
2508
+ "eval_sts_dev_pearson_cosine": 0.8329326037706698,
2509
+ "eval_sts_dev_pearson_dot": 0.7244653232060949,
2510
+ "eval_sts_dev_pearson_euclidean": 0.8208614363276345,
2511
+ "eval_sts_dev_pearson_manhattan": 0.8208271942763237,
2512
+ "eval_sts_dev_pearson_max": 0.8329326037706698,
2513
+ "eval_sts_dev_spearman_cosine": 0.8408210792535609,
2514
+ "eval_sts_dev_spearman_dot": 0.7059199525474262,
2515
+ "eval_sts_dev_spearman_euclidean": 0.8293341205092732,
2516
+ "eval_sts_dev_spearman_manhattan": 0.8305099059292741,
2517
+ "eval_sts_dev_spearman_max": 0.8408210792535609,
2518
+ "step": 3250
2519
+ },
2520
+ {
2521
+ "epoch": 0.9274867527294712,
2522
+ "grad_norm": 0.04185841232538223,
2523
+ "learning_rate": 2.3188405797101453e-05,
2524
+ "loss": 0.0041,
2525
+ "step": 3260
2526
+ },
2527
+ {
2528
+ "epoch": 0.9303318041182119,
2529
+ "grad_norm": 0.035076793283224106,
2530
+ "learning_rate": 2.325953587623366e-05,
2531
+ "loss": 0.004,
2532
+ "step": 3270
2533
+ },
2534
+ {
2535
+ "epoch": 0.9331768555069526,
2536
+ "grad_norm": 0.038307491689920425,
2537
+ "learning_rate": 2.3330665955365877e-05,
2538
+ "loss": 0.0042,
2539
+ "step": 3280
2540
+ },
2541
+ {
2542
+ "epoch": 0.9360219068956933,
2543
+ "grad_norm": 0.03905055299401283,
2544
+ "learning_rate": 2.340179603449809e-05,
2545
+ "loss": 0.004,
2546
+ "step": 3290
2547
+ },
2548
+ {
2549
+ "epoch": 0.938866958284434,
2550
+ "grad_norm": 0.03548280894756317,
2551
+ "learning_rate": 2.3472926113630305e-05,
2552
+ "loss": 0.0037,
2553
+ "step": 3300
2554
+ },
2555
+ {
2556
+ "epoch": 0.9417120096731747,
2557
+ "grad_norm": 0.031215885654091835,
2558
+ "learning_rate": 2.3544056192762517e-05,
2559
+ "loss": 0.0038,
2560
+ "step": 3310
2561
+ },
2562
+ {
2563
+ "epoch": 0.9445570610619154,
2564
+ "grad_norm": 0.03097504749894142,
2565
+ "learning_rate": 2.361518627189473e-05,
2566
+ "loss": 0.0039,
2567
+ "step": 3320
2568
+ },
2569
+ {
2570
+ "epoch": 0.9474021124506561,
2571
+ "grad_norm": 0.027599211782217026,
2572
+ "learning_rate": 2.3686316351026944e-05,
2573
+ "loss": 0.004,
2574
+ "step": 3330
2575
+ },
2576
+ {
2577
+ "epoch": 0.9502471638393969,
2578
+ "grad_norm": 0.028425293043255806,
2579
+ "learning_rate": 2.3757446430159153e-05,
2580
+ "loss": 0.0037,
2581
+ "step": 3340
2582
+ },
2583
+ {
2584
+ "epoch": 0.9530922152281376,
2585
+ "grad_norm": 0.048545658588409424,
2586
+ "learning_rate": 2.382857650929137e-05,
2587
+ "loss": 0.0038,
2588
+ "step": 3350
2589
+ },
2590
+ {
2591
+ "epoch": 0.9559372666168783,
2592
+ "grad_norm": 0.031001316383481026,
2593
+ "learning_rate": 2.389970658842358e-05,
2594
+ "loss": 0.0037,
2595
+ "step": 3360
2596
+ },
2597
+ {
2598
+ "epoch": 0.958782318005619,
2599
+ "grad_norm": 0.032509058713912964,
2600
+ "learning_rate": 2.3970836667555793e-05,
2601
+ "loss": 0.0042,
2602
+ "step": 3370
2603
+ },
2604
+ {
2605
+ "epoch": 0.9616273693943597,
2606
+ "grad_norm": 0.044801611453294754,
2607
+ "learning_rate": 2.4041966746688008e-05,
2608
+ "loss": 0.0042,
2609
+ "step": 3380
2610
+ },
2611
+ {
2612
+ "epoch": 0.9644724207831004,
2613
+ "grad_norm": 0.03758518770337105,
2614
+ "learning_rate": 2.411309682582022e-05,
2615
+ "loss": 0.0042,
2616
+ "step": 3390
2617
+ },
2618
+ {
2619
+ "epoch": 0.967317472171841,
2620
+ "grad_norm": 0.0497860424220562,
2621
+ "learning_rate": 2.4184226904952436e-05,
2622
+ "loss": 0.0037,
2623
+ "step": 3400
2624
+ },
2625
+ {
2626
+ "epoch": 0.9701625235605819,
2627
+ "grad_norm": 0.029705649241805077,
2628
+ "learning_rate": 2.4255356984084645e-05,
2629
+ "loss": 0.0038,
2630
+ "step": 3410
2631
+ },
2632
+ {
2633
+ "epoch": 0.9730075749493226,
2634
+ "grad_norm": 0.034794483333826065,
2635
+ "learning_rate": 2.432648706321686e-05,
2636
+ "loss": 0.0039,
2637
+ "step": 3420
2638
+ },
2639
+ {
2640
+ "epoch": 0.9758526263380632,
2641
+ "grad_norm": 0.03679205849766731,
2642
+ "learning_rate": 2.4397617142349072e-05,
2643
+ "loss": 0.0038,
2644
+ "step": 3430
2645
+ },
2646
+ {
2647
+ "epoch": 0.9786976777268039,
2648
+ "grad_norm": 0.03937937691807747,
2649
+ "learning_rate": 2.4468747221481284e-05,
2650
+ "loss": 0.0041,
2651
+ "step": 3440
2652
+ },
2653
+ {
2654
+ "epoch": 0.9815427291155446,
2655
+ "grad_norm": 0.0480770580470562,
2656
+ "learning_rate": 2.45398773006135e-05,
2657
+ "loss": 0.004,
2658
+ "step": 3450
2659
+ },
2660
+ {
2661
+ "epoch": 0.9843877805042853,
2662
+ "grad_norm": 0.0545036643743515,
2663
+ "learning_rate": 2.4611007379745712e-05,
2664
+ "loss": 0.0039,
2665
+ "step": 3460
2666
+ },
2667
+ {
2668
+ "epoch": 0.987232831893026,
2669
+ "grad_norm": 0.02787315472960472,
2670
+ "learning_rate": 2.4682137458877927e-05,
2671
+ "loss": 0.0036,
2672
+ "step": 3470
2673
+ },
2674
+ {
2675
+ "epoch": 0.9900778832817668,
2676
+ "grad_norm": 0.026936234906315804,
2677
+ "learning_rate": 2.4753267538010136e-05,
2678
+ "loss": 0.0037,
2679
+ "step": 3480
2680
+ },
2681
+ {
2682
+ "epoch": 0.9929229346705075,
2683
+ "grad_norm": 0.04028989002108574,
2684
+ "learning_rate": 2.482439761714235e-05,
2685
+ "loss": 0.0039,
2686
+ "step": 3490
2687
+ },
2688
+ {
2689
+ "epoch": 0.9957679860592482,
2690
+ "grad_norm": 0.036293208599090576,
2691
+ "learning_rate": 2.4895527696274564e-05,
2692
+ "loss": 0.0036,
2693
+ "step": 3500
2694
+ },
2695
+ {
2696
+ "epoch": 0.9957679860592482,
2697
+ "eval_loss": 0.04033274203538895,
2698
+ "eval_runtime": 9.5163,
2699
+ "eval_samples_per_second": 157.625,
2700
+ "eval_steps_per_second": 9.878,
2701
+ "eval_sts_dev_pearson_cosine": 0.8319192467999278,
2702
+ "eval_sts_dev_pearson_dot": 0.7234705763545461,
2703
+ "eval_sts_dev_pearson_euclidean": 0.8198226408074469,
2704
+ "eval_sts_dev_pearson_manhattan": 0.8199114649719743,
2705
+ "eval_sts_dev_pearson_max": 0.8319192467999278,
2706
+ "eval_sts_dev_spearman_cosine": 0.8396159085327265,
2707
+ "eval_sts_dev_spearman_dot": 0.7094397491074207,
2708
+ "eval_sts_dev_spearman_euclidean": 0.8285927601564604,
2709
+ "eval_sts_dev_spearman_manhattan": 0.8295556212626334,
2710
+ "eval_sts_dev_spearman_max": 0.8396159085327265,
2711
+ "step": 3500
2712
+ }
2713
+ ],
2714
+ "logging_steps": 10,
2715
+ "max_steps": 7028,
2716
+ "num_input_tokens_seen": 0,
2717
+ "num_train_epochs": 2,
2718
+ "save_steps": 3500,
2719
+ "stateful_callbacks": {
2720
+ "TrainerControl": {
2721
+ "args": {
2722
+ "should_epoch_stop": false,
2723
+ "should_evaluate": false,
2724
+ "should_log": false,
2725
+ "should_save": true,
2726
+ "should_training_stop": false
2727
+ },
2728
+ "attributes": {}
2729
+ }
2730
+ },
2731
+ "total_flos": 0.0,
2732
+ "train_batch_size": 16,
2733
+ "trial_name": null,
2734
+ "trial_params": null
2735
+ }
last-checkpoint/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d1e96dfaaf64807e921d25c3a230f147a0819110f8c7996410687b38f35bbf90
3
+ size 5688