Training in progress, step 3500, checkpoint

last-checkpoint/1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

last-checkpoint/2_Dense/config.json

@@ -0,0 +1 @@
+{"in_features": 768, "out_features": 768, "bias": true, "activation_function": "torch.nn.modules.activation.Tanh"}

last-checkpoint/2_Dense/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a3c95fe726d250267ae8ce38ee50a3be7ef98a1a04bed1b2c4e44baca3982f85
+size 2362528

last-checkpoint/README.md

@@ -0,0 +1,805 @@
+---
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:449904
+- loss:CosineSimilarityLoss
+base_model: CocoRoF/ModernBERT-SimCSE_v04
+widget:
+- source_sentence: 우리는 움직이는 동행 우주 정지 좌표계에 비례하여 이동하고 있습니다 ... 약 371km / s에서 별자리 leo
+    쪽으로. "
+  sentences:
+  - 두 마리의 독수리가 가지에 앉는다.
+  - 다른 물체와는 관련이 없는 '정지'는 없다.
+  - 소녀는 버스의 열린 문 앞에 서 있다.
+- source_sentence: 숲에는 개들이 있다.
+  sentences:
+  - 양을 보는 아이들.
+  - 여왕의 배우자를 "왕"이라고 부르지 않는 것은 아주 좋은 이유가 있다. 왜냐하면 그들은 왕이 아니기 때문이다.
+  - 개들은 숲속에 혼자 있다.
+- source_sentence: '첫째, 두 가지 다른 종류의 대시가 있다는 것을 알아야 합니다 : en 대시와 em 대시.'
+  sentences:
+  - 그들은 그 물건들을 집 주변에 두고 가거나 집의 정리를 해칠 의도가 없다.
+  - 세미콜론은 혼자 있을 수 있는 문장에 참여하는데 사용되지만, 그들의 관계를 강조하기 위해 결합됩니다.
+  - 그의 남동생이 지켜보는 동안 집 앞에서 트럼펫을 연주하는 금발의 아이.
+- source_sentence: 한 여성이 생선 껍질을 벗기고 있다.
+  sentences:
+  - 한 남자가 수영장으로 뛰어들었다.
+  - 한 여성이 프라이팬에 노란 혼합물을 부어 넣고 있다.
+  - 두 마리의 갈색 개가 눈 속에서 서로 놀고 있다.
+- source_sentence: 버스가 바쁜 길을 따라 운전한다.
+  sentences:
+  - 우리와 같은 태양계가 은하계 밖에서 존재할 수도 있을 것입니다.
+  - 그 여자는 데이트하러 가는 중이다.
+  - 녹색 버스가 도로를 따라 내려간다.
+datasets:
+- x2bee/misc_sts_pairs_v2_kor_kosimcse
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+metrics:
+- pearson_cosine
+- spearman_cosine
+- pearson_euclidean
+- spearman_euclidean
+- pearson_manhattan
+- spearman_manhattan
+- pearson_dot
+- spearman_dot
+- pearson_max
+- spearman_max
+model-index:
+- name: SentenceTransformer based on CocoRoF/ModernBERT-SimCSE_v04
+  results:
+  - task:
+      type: semantic-similarity
+      name: Semantic Similarity
+    dataset:
+      name: sts dev
+      type: sts_dev
+    metrics:
+    - type: pearson_cosine
+      value: 0.7873863893875668
+      name: Pearson Cosine
+    - type: spearman_cosine
+      value: 0.7957546813665759
+      name: Spearman Cosine
+    - type: pearson_euclidean
+      value: 0.7578432355248655
+      name: Pearson Euclidean
+    - type: spearman_euclidean
+      value: 0.7559256690552184
+      name: Spearman Euclidean
+    - type: pearson_manhattan
+      value: 0.7576106133017201
+      name: Pearson Manhattan
+    - type: spearman_manhattan
+      value: 0.7557621348675576
+      name: Spearman Manhattan
+    - type: pearson_dot
+      value: 0.590451098760234
+      name: Pearson Dot
+    - type: spearman_dot
+      value: 0.5781029293331612
+      name: Spearman Dot
+    - type: pearson_max
+      value: 0.7873863893875668
+      name: Pearson Max
+    - type: spearman_max
+      value: 0.7957546813665759
+      name: Spearman Max
+---
+
+# SentenceTransformer based on CocoRoF/ModernBERT-SimCSE_v04
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [CocoRoF/ModernBERT-SimCSE_v04](https://huggingface.co/CocoRoF/ModernBERT-SimCSE_v04) on the [misc_sts_pairs_v2_kor_kosimcse](https://huggingface.co/datasets/x2bee/misc_sts_pairs_v2_kor_kosimcse) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [CocoRoF/ModernBERT-SimCSE_v04](https://huggingface.co/CocoRoF/ModernBERT-SimCSE_v04) <!-- at revision 7d23b869258e5c726c0f536bccac7e873d510d66 -->
+- **Maximum Sequence Length:** 512 tokens
+- **Output Dimensionality:** 768 dimensions
+- **Similarity Function:** Cosine Similarity
+- **Training Dataset:**
+    - [misc_sts_pairs_v2_kor_kosimcse](https://huggingface.co/datasets/x2bee/misc_sts_pairs_v2_kor_kosimcse)
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: ModernBertModel 
+  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+  (2): Dense({'in_features': 768, 'out_features': 768, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh'})
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("CocoRoF/ModernBERT-SimCSE-multitask_v05")
+# Run inference
+sentences = [
+    '버스가 바쁜 길을 따라 운전한다.',
+    '녹색 버스가 도로를 따라 내려간다.',
+    '그 여자는 데이트하러 가는 중이다.',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 768]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Semantic Similarity
+
+* Dataset: `sts_dev`
+* Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
+
+| Metric             | Value      |
+|:-------------------|:-----------|
+| pearson_cosine     | 0.7874     |
+| spearman_cosine    | 0.7958     |
+| pearson_euclidean  | 0.7578     |
+| spearman_euclidean | 0.7559     |
+| pearson_manhattan  | 0.7576     |
+| spearman_manhattan | 0.7558     |
+| pearson_dot        | 0.5905     |
+| spearman_dot       | 0.5781     |
+| pearson_max        | 0.7874     |
+| **spearman_max**   | **0.7958** |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### misc_sts_pairs_v2_kor_kosimcse
+
+* Dataset: [misc_sts_pairs_v2_kor_kosimcse](https://huggingface.co/datasets/x2bee/misc_sts_pairs_v2_kor_kosimcse) at [e747415](https://huggingface.co/datasets/x2bee/misc_sts_pairs_v2_kor_kosimcse/tree/e747415cfe9ff51d1c1550b8a07e5014c01dea59)
+* Size: 449,904 training samples
+* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | sentence1                                                                         | sentence2                                                                         | score                                                           |
+  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                            | float                                                           |
+  | details | <ul><li>min: 6 tokens</li><li>mean: 15.81 tokens</li><li>max: 66 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 16.18 tokens</li><li>max: 48 tokens</li></ul> | <ul><li>min: 0.11</li><li>mean: 0.77</li><li>max: 1.0</li></ul> |
+* Samples:
+  | sentence1                                        | sentence2                                  | score                           |
+  |:-------------------------------------------------|:-------------------------------------------|:--------------------------------|
+  | <code>주홍글씨는 언제 출판되었습니까?</code>                   | <code>《주홍글씨》는 몇 년에 출판되었습니까?</code>         | <code>0.8638778924942017</code> |
+  | <code>폴란드에서 빨간색과 흰색은 무엇을 의미합니까?</code>           | <code>폴란드 국기의 색상은 무엇입니까?</code>            | <code>0.6773715019226074</code> |
+  | <code>노르만인들은 방어를 위해 모트와 베일리 성을 어떻게 사용했는가?</code> | <code>11세기에는 어떻게 모트와 베일리 성을 만들었습니까?</code> | <code>0.7460665702819824</code> |
+* Loss: [<code>CosineSimilarityLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters:
+  ```json
+  {
+      "loss_fct": "torch.nn.modules.loss.MSELoss"
+  }
+  ```
+
+### Evaluation Dataset
+
+#### Unnamed Dataset
+
+
+* Size: 1,500 evaluation samples
+* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | sentence1                                                                         | sentence2                                                                         | score                                                          |
+  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                            | float                                                          |
+  | details | <ul><li>min: 6 tokens</li><li>mean: 18.89 tokens</li><li>max: 51 tokens</li></ul> | <ul><li>min: 5 tokens</li><li>mean: 18.92 tokens</li><li>max: 50 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.42</li><li>max: 1.0</li></ul> |
+* Samples:
+  | sentence1                            | sentence2                           | score             |
+  |:-------------------------------------|:------------------------------------|:------------------|
+  | <code>안전모를 가진 한 남자가 춤을 추고 있다.</code> | <code>안전모를 쓴 한 남자가 춤을 추고 있다.</code> | <code>1.0</code>  |
+  | <code>어린아이가 말을 타고 있다.</code>         | <code>아이가 말을 타고 있다.</code>          | <code>0.95</code> |
+  | <code>한 남자가 뱀에게 쥐를 먹이고 있다.</code>    | <code>남자가 뱀에게 쥐를 먹이고 있다.</code>     | <code>1.0</code>  |
+* Loss: [<code>CosineSimilarityLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters:
+  ```json
+  {
+      "loss_fct": "torch.nn.modules.loss.MSELoss"
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `overwrite_output_dir`: True
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `gradient_accumulation_steps`: 8
+- `num_train_epochs`: 2.0
+- `warmup_ratio`: 0.2
+- `push_to_hub`: True
+- `hub_model_id`: CocoRoF/ModernBERT-SimCSE-multitask_v05
+- `hub_strategy`: checkpoint
+- `batch_sampler`: no_duplicates
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: True
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 8
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 5e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 2.0
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.2
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: True
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: True
+- `resume_from_checkpoint`: None
+- `hub_model_id`: CocoRoF/ModernBERT-SimCSE-multitask_v05
+- `hub_strategy`: checkpoint
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: no_duplicates
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+<details><summary>Click to expand</summary>
+
+| Epoch  | Step | Training Loss | Validation Loss | sts_dev_spearman_max |
+|:------:|:----:|:-------------:|:---------------:|:--------------------:|
+| 0.0028 | 10   | 0.0202        | -               | -                    |
+| 0.0057 | 20   | 0.0184        | -               | -                    |
+| 0.0085 | 30   | 0.018         | -               | -                    |
+| 0.0114 | 40   | 0.0173        | -               | -                    |
+| 0.0142 | 50   | 0.0193        | -               | -                    |
+| 0.0171 | 60   | 0.0158        | -               | -                    |
+| 0.0199 | 70   | 0.016         | -               | -                    |
+| 0.0228 | 80   | 0.0139        | -               | -                    |
+| 0.0256 | 90   | 0.0143        | -               | -                    |
+| 0.0285 | 100  | 0.0138        | -               | -                    |
+| 0.0313 | 110  | 0.0127        | -               | -                    |
+| 0.0341 | 120  | 0.0115        | -               | -                    |
+| 0.0370 | 130  | 0.0117        | -               | -                    |
+| 0.0398 | 140  | 0.0111        | -               | -                    |
+| 0.0427 | 150  | 0.0111        | -               | -                    |
+| 0.0455 | 160  | 0.0106        | -               | -                    |
+| 0.0484 | 170  | 0.01          | -               | -                    |
+| 0.0512 | 180  | 0.0103        | -               | -                    |
+| 0.0541 | 190  | 0.0106        | -               | -                    |
+| 0.0569 | 200  | 0.0102        | -               | -                    |
+| 0.0597 | 210  | 0.0103        | -               | -                    |
+| 0.0626 | 220  | 0.0109        | -               | -                    |
+| 0.0654 | 230  | 0.0099        | -               | -                    |
+| 0.0683 | 240  | 0.0086        | -               | -                    |
+| 0.0711 | 250  | 0.01          | 0.0448          | 0.7642               |
+| 0.0740 | 260  | 0.0098        | -               | -                    |
+| 0.0768 | 270  | 0.0094        | -               | -                    |
+| 0.0797 | 280  | 0.0097        | -               | -                    |
+| 0.0825 | 290  | 0.0094        | -               | -                    |
+| 0.0854 | 300  | 0.0095        | -               | -                    |
+| 0.0882 | 310  | 0.0098        | -               | -                    |
+| 0.0910 | 320  | 0.0092        | -               | -                    |
+| 0.0939 | 330  | 0.0095        | -               | -                    |
+| 0.0967 | 340  | 0.0103        | -               | -                    |
+| 0.0996 | 350  | 0.0097        | -               | -                    |
+| 0.1024 | 360  | 0.0091        | -               | -                    |
+| 0.1053 | 370  | 0.0094        | -               | -                    |
+| 0.1081 | 380  | 0.0088        | -               | -                    |
+| 0.1110 | 390  | 0.009         | -               | -                    |
+| 0.1138 | 400  | 0.0098        | -               | -                    |
+| 0.1166 | 410  | 0.0083        | -               | -                    |
+| 0.1195 | 420  | 0.0099        | -               | -                    |
+| 0.1223 | 430  | 0.0094        | -               | -                    |
+| 0.1252 | 440  | 0.0092        | -               | -                    |
+| 0.1280 | 450  | 0.009         | -               | -                    |
+| 0.1309 | 460  | 0.0088        | -               | -                    |
+| 0.1337 | 470  | 0.0092        | -               | -                    |
+| 0.1366 | 480  | 0.0083        | -               | -                    |
+| 0.1394 | 490  | 0.0089        | -               | -                    |
+| 0.1423 | 500  | 0.0089        | 0.0444          | 0.7725               |
+| 0.1451 | 510  | 0.0095        | -               | -                    |
+| 0.1479 | 520  | 0.0095        | -               | -                    |
+| 0.1508 | 530  | 0.0091        | -               | -                    |
+| 0.1536 | 540  | 0.0082        | -               | -                    |
+| 0.1565 | 550  | 0.0091        | -               | -                    |
+| 0.1593 | 560  | 0.0086        | -               | -                    |
+| 0.1622 | 570  | 0.009         | -               | -                    |
+| 0.1650 | 580  | 0.0088        | -               | -                    |
+| 0.1679 | 590  | 0.0087        | -               | -                    |
+| 0.1707 | 600  | 0.0089        | -               | -                    |
+| 0.1735 | 610  | 0.009         | -               | -                    |
+| 0.1764 | 620  | 0.0088        | -               | -                    |
+| 0.1792 | 630  | 0.0088        | -               | -                    |
+| 0.1821 | 640  | 0.0081        | -               | -                    |
+| 0.1849 | 650  | 0.0082        | -               | -                    |
+| 0.1878 | 660  | 0.0088        | -               | -                    |
+| 0.1906 | 670  | 0.0086        | -               | -                    |
+| 0.1935 | 680  | 0.0085        | -               | -                    |
+| 0.1963 | 690  | 0.009         | -               | -                    |
+| 0.1992 | 700  | 0.0083        | -               | -                    |
+| 0.2020 | 710  | 0.0088        | -               | -                    |
+| 0.2048 | 720  | 0.0088        | -               | -                    |
+| 0.2077 | 730  | 0.0087        | -               | -                    |
+| 0.2105 | 740  | 0.0088        | -               | -                    |
+| 0.2134 | 750  | 0.008         | 0.0465          | 0.7798               |
+| 0.2162 | 760  | 0.0087        | -               | -                    |
+| 0.2191 | 770  | 0.0087        | -               | -                    |
+| 0.2219 | 780  | 0.009         | -               | -                    |
+| 0.2248 | 790  | 0.0085        | -               | -                    |
+| 0.2276 | 800  | 0.009         | -               | -                    |
+| 0.2304 | 810  | 0.0082        | -               | -                    |
+| 0.2333 | 820  | 0.0073        | -               | -                    |
+| 0.2361 | 830  | 0.0078        | -               | -                    |
+| 0.2390 | 840  | 0.0088        | -               | -                    |
+| 0.2418 | 850  | 0.0077        | -               | -                    |
+| 0.2447 | 860  | 0.008         | -               | -                    |
+| 0.2475 | 870  | 0.008         | -               | -                    |
+| 0.2504 | 880  | 0.0086        | -               | -                    |
+| 0.2532 | 890  | 0.0083        | -               | -                    |
+| 0.2561 | 900  | 0.0081        | -               | -                    |
+| 0.2589 | 910  | 0.0081        | -               | -                    |
+| 0.2617 | 920  | 0.0077        | -               | -                    |
+| 0.2646 | 930  | 0.0083        | -               | -                    |
+| 0.2674 | 940  | 0.0081        | -               | -                    |
+| 0.2703 | 950  | 0.0069        | -               | -                    |
+| 0.2731 | 960  | 0.0084        | -               | -                    |
+| 0.2760 | 970  | 0.0075        | -               | -                    |
+| 0.2788 | 980  | 0.0081        | -               | -                    |
+| 0.2817 | 990  | 0.0086        | -               | -                    |
+| 0.2845 | 1000 | 0.0079        | 0.0473          | 0.7855               |
+| 0.2874 | 1010 | 0.0088        | -               | -                    |
+| 0.2902 | 1020 | 0.0073        | -               | -                    |
+| 0.2930 | 1030 | 0.008         | -               | -                    |
+| 0.2959 | 1040 | 0.0073        | -               | -                    |
+| 0.2987 | 1050 | 0.008         | -               | -                    |
+| 0.3016 | 1060 | 0.0074        | -               | -                    |
+| 0.3044 | 1070 | 0.007         | -               | -                    |
+| 0.3073 | 1080 | 0.0075        | -               | -                    |
+| 0.3101 | 1090 | 0.0077        | -               | -                    |
+| 0.3130 | 1100 | 0.0076        | -               | -                    |
+| 0.3158 | 1110 | 0.0082        | -               | -                    |
+| 0.3186 | 1120 | 0.0073        | -               | -                    |
+| 0.3215 | 1130 | 0.007         | -               | -                    |
+| 0.3243 | 1140 | 0.0077        | -               | -                    |
+| 0.3272 | 1150 | 0.0074        | -               | -                    |
+| 0.3300 | 1160 | 0.0076        | -               | -                    |
+| 0.3329 | 1170 | 0.0078        | -               | -                    |
+| 0.3357 | 1180 | 0.0073        | -               | -                    |
+| 0.3386 | 1190 | 0.0077        | -               | -                    |
+| 0.3414 | 1200 | 0.0068        | -               | -                    |
+| 0.3443 | 1210 | 0.0079        | -               | -                    |
+| 0.3471 | 1220 | 0.0073        | -               | -                    |
+| 0.3499 | 1230 | 0.0075        | -               | -                    |
+| 0.3528 | 1240 | 0.0078        | -               | -                    |
+| 0.3556 | 1250 | 0.0073        | 0.0472          | 0.7855               |
+| 0.3585 | 1260 | 0.0073        | -               | -                    |
+| 0.3613 | 1270 | 0.007         | -               | -                    |
+| 0.3642 | 1280 | 0.0068        | -               | -                    |
+| 0.3670 | 1290 | 0.0067        | -               | -                    |
+| 0.3699 | 1300 | 0.0078        | -               | -                    |
+| 0.3727 | 1310 | 0.0072        | -               | -                    |
+| 0.3755 | 1320 | 0.0071        | -               | -                    |
+| 0.3784 | 1330 | 0.0068        | -               | -                    |
+| 0.3812 | 1340 | 0.0068        | -               | -                    |
+| 0.3841 | 1350 | 0.0074        | -               | -                    |
+| 0.3869 | 1360 | 0.0074        | -               | -                    |
+| 0.3898 | 1370 | 0.0077        | -               | -                    |
+| 0.3926 | 1380 | 0.0069        | -               | -                    |
+| 0.3955 | 1390 | 0.0079        | -               | -                    |
+| 0.3983 | 1400 | 0.0066        | -               | -                    |
+| 0.4012 | 1410 | 0.008         | -               | -                    |
+| 0.4040 | 1420 | 0.008         | -               | -                    |
+| 0.4068 | 1430 | 0.0071        | -               | -                    |
+| 0.4097 | 1440 | 0.0066        | -               | -                    |
+| 0.4125 | 1450 | 0.0079        | -               | -                    |
+| 0.4154 | 1460 | 0.0075        | -               | -                    |
+| 0.4182 | 1470 | 0.0066        | -               | -                    |
+| 0.4211 | 1480 | 0.007         | -               | -                    |
+| 0.4239 | 1490 | 0.0066        | -               | -                    |
+| 0.4268 | 1500 | 0.0066        | 0.0474          | 0.7908               |
+| 0.4296 | 1510 | 0.0075        | -               | -                    |
+| 0.4324 | 1520 | 0.0072        | -               | -                    |
+| 0.4353 | 1530 | 0.0072        | -               | -                    |
+| 0.4381 | 1540 | 0.0067        | -               | -                    |
+| 0.4410 | 1550 | 0.0073        | -               | -                    |
+| 0.4438 | 1560 | 0.0066        | -               | -                    |
+| 0.4467 | 1570 | 0.0063        | -               | -                    |
+| 0.4495 | 1580 | 0.0074        | -               | -                    |
+| 0.4524 | 1590 | 0.0075        | -               | -                    |
+| 0.4552 | 1600 | 0.0069        | -               | -                    |
+| 0.4581 | 1610 | 0.0065        | -               | -                    |
+| 0.4609 | 1620 | 0.007         | -               | -                    |
+| 0.4637 | 1630 | 0.0067        | -               | -                    |
+| 0.4666 | 1640 | 0.0067        | -               | -                    |
+| 0.4694 | 1650 | 0.0072        | -               | -                    |
+| 0.4723 | 1660 | 0.007         | -               | -                    |
+| 0.4751 | 1670 | 0.0078        | -               | -                    |
+| 0.4780 | 1680 | 0.0069        | -               | -                    |
+| 0.4808 | 1690 | 0.0067        | -               | -                    |
+| 0.4837 | 1700 | 0.0072        | -               | -                    |
+| 0.4865 | 1710 | 0.0071        | -               | -                    |
+| 0.4893 | 1720 | 0.0069        | -               | -                    |
+| 0.4922 | 1730 | 0.0074        | -               | -                    |
+| 0.4950 | 1740 | 0.0073        | -               | -                    |
+| 0.4979 | 1750 | 0.0064        | 0.0499          | 0.7938               |
+| 0.5007 | 1760 | 0.0064        | -               | -                    |
+| 0.5036 | 1770 | 0.0068        | -               | -                    |
+| 0.5064 | 1780 | 0.007         | -               | -                    |
+| 0.5093 | 1790 | 0.0065        | -               | -                    |
+| 0.5121 | 1800 | 0.0073        | -               | -                    |
+| 0.5150 | 1810 | 0.0061        | -               | -                    |
+| 0.5178 | 1820 | 0.0071        | -               | -                    |
+| 0.5206 | 1830 | 0.0058        | -               | -                    |
+| 0.5235 | 1840 | 0.0065        | -               | -                    |
+| 0.5263 | 1850 | 0.0067        | -               | -                    |
+| 0.5292 | 1860 | 0.0063        | -               | -                    |
+| 0.5320 | 1870 | 0.007         | -               | -                    |
+| 0.5349 | 1880 | 0.0069        | -               | -                    |
+| 0.5377 | 1890 | 0.0073        | -               | -                    |
+| 0.5406 | 1900 | 0.0067        | -               | -                    |
+| 0.5434 | 1910 | 0.0068        | -               | -                    |
+| 0.5462 | 1920 | 0.0066        | -               | -                    |
+| 0.5491 | 1930 | 0.007         | -               | -                    |
+| 0.5519 | 1940 | 0.006         | -               | -                    |
+| 0.5548 | 1950 | 0.0062        | -               | -                    |
+| 0.5576 | 1960 | 0.0062        | -               | -                    |
+| 0.5605 | 1970 | 0.0067        | -               | -                    |
+| 0.5633 | 1980 | 0.0063        | -               | -                    |
+| 0.5662 | 1990 | 0.006         | -               | -                    |
+| 0.5690 | 2000 | 0.0067        | 0.0478          | 0.7943               |
+| 0.5719 | 2010 | 0.0076        | -               | -                    |
+| 0.5747 | 2020 | 0.0069        | -               | -                    |
+| 0.5775 | 2030 | 0.0065        | -               | -                    |
+| 0.5804 | 2040 | 0.007         | -               | -                    |
+| 0.5832 | 2050 | 0.006         | -               | -                    |
+| 0.5861 | 2060 | 0.0064        | -               | -                    |
+| 0.5889 | 2070 | 0.0063        | -               | -                    |
+| 0.5918 | 2080 | 0.0067        | -               | -                    |
+| 0.5946 | 2090 | 0.0064        | -               | -                    |
+| 0.5975 | 2100 | 0.0062        | -               | -                    |
+| 0.6003 | 2110 | 0.0063        | -               | -                    |
+| 0.6032 | 2120 | 0.0063        | -               | -                    |
+| 0.6060 | 2130 | 0.0074        | -               | -                    |
+| 0.6088 | 2140 | 0.0067        | -               | -                    |
+| 0.6117 | 2150 | 0.006         | -               | -                    |
+| 0.6145 | 2160 | 0.0062        | -               | -                    |
+| 0.6174 | 2170 | 0.007         | -               | -                    |
+| 0.6202 | 2180 | 0.0069        | -               | -                    |
+| 0.6231 | 2190 | 0.007         | -               | -                    |
+| 0.6259 | 2200 | 0.0065        | -               | -                    |
+| 0.6288 | 2210 | 0.0071        | -               | -                    |
+| 0.6316 | 2220 | 0.007         | -               | -                    |
+| 0.6344 | 2230 | 0.0064        | -               | -                    |
+| 0.6373 | 2240 | 0.0061        | -               | -                    |
+| 0.6401 | 2250 | 0.0062        | 0.0464          | 0.7935               |
+| 0.6430 | 2260 | 0.0069        | -               | -                    |
+| 0.6458 | 2270 | 0.0062        | -               | -                    |
+| 0.6487 | 2280 | 0.0063        | -               | -                    |
+| 0.6515 | 2290 | 0.0063        | -               | -                    |
+| 0.6544 | 2300 | 0.006         | -               | -                    |
+| 0.6572 | 2310 | 0.0064        | -               | -                    |
+| 0.6601 | 2320 | 0.0061        | -               | -                    |
+| 0.6629 | 2330 | 0.0065        | -               | -                    |
+| 0.6657 | 2340 | 0.0061        | -               | -                    |
+| 0.6686 | 2350 | 0.0067        | -               | -                    |
+| 0.6714 | 2360 | 0.0066        | -               | -                    |
+| 0.6743 | 2370 | 0.0068        | -               | -                    |
+| 0.6771 | 2380 | 0.0071        | -               | -                    |
+| 0.6800 | 2390 | 0.0064        | -               | -                    |
+| 0.6828 | 2400 | 0.0064        | -               | -                    |
+| 0.6857 | 2410 | 0.0064        | -               | -                    |
+| 0.6885 | 2420 | 0.0064        | -               | -                    |
+| 0.6913 | 2430 | 0.0062        | -               | -                    |
+| 0.6942 | 2440 | 0.0067        | -               | -                    |
+| 0.6970 | 2450 | 0.0062        | -               | -                    |
+| 0.6999 | 2460 | 0.0059        | -               | -                    |
+| 0.7027 | 2470 | 0.0063        | -               | -                    |
+| 0.7056 | 2480 | 0.0055        | -               | -                    |
+| 0.7084 | 2490 | 0.0074        | -               | -                    |
+| 0.7113 | 2500 | 0.0064        | 0.0488          | 0.7939               |
+| 0.7141 | 2510 | 0.006         | -               | -                    |
+| 0.7170 | 2520 | 0.0061        | -               | -                    |
+| 0.7198 | 2530 | 0.0064        | -               | -                    |
+| 0.7226 | 2540 | 0.0059        | -               | -                    |
+| 0.7255 | 2550 | 0.0064        | -               | -                    |
+| 0.7283 | 2560 | 0.0061        | -               | -                    |
+| 0.7312 | 2570 | 0.0062        | -               | -                    |
+| 0.7340 | 2580 | 0.0068        | -               | -                    |
+| 0.7369 | 2590 | 0.0061        | -               | -                    |
+| 0.7397 | 2600 | 0.0065        | -               | -                    |
+| 0.7426 | 2610 | 0.0055        | -               | -                    |
+| 0.7454 | 2620 | 0.0057        | -               | -                    |
+| 0.7482 | 2630 | 0.0064        | -               | -                    |
+| 0.7511 | 2640 | 0.0056        | -               | -                    |
+| 0.7539 | 2650 | 0.0059        | -               | -                    |
+| 0.7568 | 2660 | 0.0059        | -               | -                    |
+| 0.7596 | 2670 | 0.0064        | -               | -                    |
+| 0.7625 | 2680 | 0.0067        | -               | -                    |
+| 0.7653 | 2690 | 0.0062        | -               | -                    |
+| 0.7682 | 2700 | 0.0056        | -               | -                    |
+| 0.7710 | 2710 | 0.0063        | -               | -                    |
+| 0.7739 | 2720 | 0.0064        | -               | -                    |
+| 0.7767 | 2730 | 0.0063        | -               | -                    |
+| 0.7795 | 2740 | 0.0062        | -               | -                    |
+| 0.7824 | 2750 | 0.0058        | 0.0479          | 0.7987               |
+| 0.7852 | 2760 | 0.0063        | -               | -                    |
+| 0.7881 | 2770 | 0.0061        | -               | -                    |
+| 0.7909 | 2780 | 0.0059        | -               | -                    |
+| 0.7938 | 2790 | 0.0061        | -               | -                    |
+| 0.7966 | 2800 | 0.0059        | -               | -                    |
+| 0.7995 | 2810 | 0.0058        | -               | -                    |
+| 0.8023 | 2820 | 0.0057        | -               | -                    |
+| 0.8051 | 2830 | 0.0059        | -               | -                    |
+| 0.8080 | 2840 | 0.0058        | -               | -                    |
+| 0.8108 | 2850 | 0.0068        | -               | -                    |
+| 0.8137 | 2860 | 0.006         | -               | -                    |
+| 0.8165 | 2870 | 0.0058        | -               | -                    |
+| 0.8194 | 2880 | 0.0061        | -               | -                    |
+| 0.8222 | 2890 | 0.0058        | -               | -                    |
+| 0.8251 | 2900 | 0.0055        | -               | -                    |
+| 0.8279 | 2910 | 0.006         | -               | -                    |
+| 0.8308 | 2920 | 0.0063        | -               | -                    |
+| 0.8336 | 2930 | 0.0066        | -               | -                    |
+| 0.8364 | 2940 | 0.0059        | -               | -                    |
+| 0.8393 | 2950 | 0.0056        | -               | -                    |
+| 0.8421 | 2960 | 0.006         | -               | -                    |
+| 0.8450 | 2970 | 0.0058        | -               | -                    |
+| 0.8478 | 2980 | 0.006         | -               | -                    |
+| 0.8507 | 2990 | 0.0056        | -               | -                    |
+| 0.8535 | 3000 | 0.0062        | 0.0511          | 0.7996               |
+| 0.8564 | 3010 | 0.0059        | -               | -                    |
+| 0.8592 | 3020 | 0.0064        | -               | -                    |
+| 0.8621 | 3030 | 0.0064        | -               | -                    |
+| 0.8649 | 3040 | 0.006         | -               | -                    |
+| 0.8677 | 3050 | 0.0059        | -               | -                    |
+| 0.8706 | 3060 | 0.0055        | -               | -                    |
+| 0.8734 | 3070 | 0.0056        | -               | -                    |
+| 0.8763 | 3080 | 0.0058        | -               | -                    |
+| 0.8791 | 3090 | 0.0057        | -               | -                    |
+| 0.8820 | 3100 | 0.0058        | -               | -                    |
+| 0.8848 | 3110 | 0.0062        | -               | -                    |
+| 0.8877 | 3120 | 0.0058        | -               | -                    |
+| 0.8905 | 3130 | 0.0058        | -               | -                    |
+| 0.8933 | 3140 | 0.0055        | -               | -                    |
+| 0.8962 | 3150 | 0.0056        | -               | -                    |
+| 0.8990 | 3160 | 0.0055        | -               | -                    |
+| 0.9019 | 3170 | 0.0054        | -               | -                    |
+| 0.9047 | 3180 | 0.0059        | -               | -                    |
+| 0.9076 | 3190 | 0.0056        | -               | -                    |
+| 0.9104 | 3200 | 0.0057        | -               | -                    |
+| 0.9133 | 3210 | 0.0055        | -               | -                    |
+| 0.9161 | 3220 | 0.0061        | -               | -                    |
+| 0.9190 | 3230 | 0.0055        | -               | -                    |
+| 0.9218 | 3240 | 0.0062        | -               | -                    |
+| 0.9246 | 3250 | 0.006         | 0.0508          | 0.7989               |
+| 0.9275 | 3260 | 0.0058        | -               | -                    |
+| 0.9303 | 3270 | 0.0053        | -               | -                    |
+| 0.9332 | 3280 | 0.0064        | -               | -                    |
+| 0.9360 | 3290 | 0.006         | -               | -                    |
+| 0.9389 | 3300 | 0.0057        | -               | -                    |
+| 0.9417 | 3310 | 0.0059        | -               | -                    |
+| 0.9446 | 3320 | 0.0057        | -               | -                    |
+| 0.9474 | 3330 | 0.0056        | -               | -                    |
+| 0.9502 | 3340 | 0.0056        | -               | -                    |
+| 0.9531 | 3350 | 0.0061        | -               | -                    |
+| 0.9559 | 3360 | 0.0053        | -               | -                    |
+| 0.9588 | 3370 | 0.0056        | -               | -                    |
+| 0.9616 | 3380 | 0.006         | -               | -                    |
+| 0.9645 | 3390 | 0.0066        | -               | -                    |
+| 0.9673 | 3400 | 0.0062        | -               | -                    |
+| 0.9702 | 3410 | 0.0053        | -               | -                    |
+| 0.9730 | 3420 | 0.0062        | -               | -                    |
+| 0.9759 | 3430 | 0.0057        | -               | -                    |
+| 0.9787 | 3440 | 0.0059        | -               | -                    |
+| 0.9815 | 3450 | 0.0061        | -               | -                    |
+| 0.9844 | 3460 | 0.0057        | -               | -                    |
+| 0.9872 | 3470 | 0.0054        | -               | -                    |
+| 0.9901 | 3480 | 0.0054        | -               | -                    |
+| 0.9929 | 3490 | 0.0057        | -               | -                    |
+| 0.9958 | 3500 | 0.0056        | 0.0485          | 0.7958               |
+
+</details>
+
+### Framework Versions
+- Python: 3.11.10
+- Sentence Transformers: 3.3.1
+- Transformers: 4.48.0.dev0
+- PyTorch: 2.5.1+cu121
+- Accelerate: 1.1.0
+- Datasets: 3.1.0
+- Tokenizers: 0.21.0
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

last-checkpoint/config.json

@@ -0,0 +1,47 @@
+{
+  "_name_or_path": "CocoRoF/ModernBERT-SimCSE_v04",
+  "architectures": [
+    "ModernBertModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 50281,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 50281,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "embedding_dropout": 0.0,
+  "eos_token_id": 50282,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000.0,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "layer_norm_eps": 1e-05,
+  "local_attention": 128,
+  "local_rope_theta": 10000.0,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 50283,
+  "position_embedding_type": "absolute",
+  "reference_compile": false,
+  "repad_logits_with_grad": false,
+  "sep_token_id": 50282,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.48.0.dev0",
+  "vocab_size": 55111
+}

last-checkpoint/config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.3.1",
+    "transformers": "4.48.0.dev0",
+    "pytorch": "2.5.1+cu121"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

last-checkpoint/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73d71a67c1540a4198edfec85a4d055661cab5f5afc320ab59832d468dec906b
+size 610640632

last-checkpoint/modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Dense",
+    "type": "sentence_transformers.models.Dense"
+  }
+]

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f66fa8d3c164f862f611a6715d94383ebe8cf33bb65807628e137d39ceb12af2
+size 1226096954

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fa62fd12355b012a09afe1496b5d8596451c15a706eac24818bf03a9e9fc4738
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a913c1e668bb2922898ee65a01cca66ffbca50d0de1d668a4132704dc21a9b2
+size 1000

last-checkpoint/sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": false
+}

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,945 @@
+{
+  "added_tokens_decoder": {
+    "50280": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50281": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50282": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50283": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50284": {
+      "content": "[MASK]",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "55000": {
+      "content": "|||IP_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55001": {
+      "content": "<|padding|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "55002": {
+      "content": "                        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55003": {
+      "content": "                       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55004": {
+      "content": "                      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55005": {
+      "content": "                     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55006": {
+      "content": "                    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55007": {
+      "content": "                   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55008": {
+      "content": "                  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55009": {
+      "content": "                 ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55010": {
+      "content": "                ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55011": {
+      "content": "               ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55012": {
+      "content": "              ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55013": {
+      "content": "             ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55014": {
+      "content": "            ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55015": {
+      "content": "           ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55016": {
+      "content": "          ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55017": {
+      "content": "         ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55018": {
+      "content": "        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55019": {
+      "content": "       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55020": {
+      "content": "      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55021": {
+      "content": "     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55022": {
+      "content": "    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55023": {
+      "content": "   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55024": {
+      "content": "  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55025": {
+      "content": "|||EMAIL_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55026": {
+      "content": "|||PHONE_NUMBER|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55027": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "55028": {
+      "content": "[unused0]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55029": {
+      "content": "[unused1]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55030": {
+      "content": "[unused2]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55031": {
+      "content": "[unused3]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55032": {
+      "content": "[unused4]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55033": {
+      "content": "[unused5]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55034": {
+      "content": "[unused6]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55035": {
+      "content": "[unused7]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55036": {
+      "content": "[unused8]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55037": {
+      "content": "[unused9]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55038": {
+      "content": "[unused10]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55039": {
+      "content": "[unused11]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55040": {
+      "content": "[unused12]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55041": {
+      "content": "[unused13]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55042": {
+      "content": "[unused14]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55043": {
+      "content": "[unused15]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55044": {
+      "content": "[unused16]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55045": {
+      "content": "[unused17]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55046": {
+      "content": "[unused18]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55047": {
+      "content": "[unused19]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55048": {
+      "content": "[unused20]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55049": {
+      "content": "[unused21]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55050": {
+      "content": "[unused22]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55051": {
+      "content": "[unused23]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55052": {
+      "content": "[unused24]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55053": {
+      "content": "[unused25]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55054": {
+      "content": "[unused26]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55055": {
+      "content": "[unused27]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55056": {
+      "content": "[unused28]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55057": {
+      "content": "[unused29]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55058": {
+      "content": "[unused30]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55059": {
+      "content": "[unused31]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55060": {
+      "content": "[unused32]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55061": {
+      "content": "[unused33]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55062": {
+      "content": "[unused34]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55063": {
+      "content": "[unused35]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55064": {
+      "content": "[unused36]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55065": {
+      "content": "[unused37]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55066": {
+      "content": "[unused38]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55067": {
+      "content": "[unused39]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55068": {
+      "content": "[unused40]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55069": {
+      "content": "[unused41]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55070": {
+      "content": "[unused42]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55071": {
+      "content": "[unused43]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55072": {
+      "content": "[unused44]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55073": {
+      "content": "[unused45]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55074": {
+      "content": "[unused46]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55075": {
+      "content": "[unused47]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55076": {
+      "content": "[unused48]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55077": {
+      "content": "[unused49]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55078": {
+      "content": "[unused50]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55079": {
+      "content": "[unused51]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55080": {
+      "content": "[unused52]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55081": {
+      "content": "[unused53]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55082": {
+      "content": "[unused54]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55083": {
+      "content": "[unused55]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55084": {
+      "content": "[unused56]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55085": {
+      "content": "[unused57]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55086": {
+      "content": "[unused58]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55087": {
+      "content": "[unused59]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55088": {
+      "content": "[unused60]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55089": {
+      "content": "[unused61]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55090": {
+      "content": "[unused62]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55091": {
+      "content": "[unused63]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55092": {
+      "content": "[unused64]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55093": {
+      "content": "[unused65]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55094": {
+      "content": "[unused66]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55095": {
+      "content": "[unused67]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55096": {
+      "content": "[unused68]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55097": {
+      "content": "[unused69]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55098": {
+      "content": "[unused70]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55099": {
+      "content": "[unused71]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55100": {
+      "content": "[unused72]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55101": {
+      "content": "[unused73]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55102": {
+      "content": "[unused74]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55103": {
+      "content": "[unused75]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55104": {
+      "content": "[unused76]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55105": {
+      "content": "[unused77]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55106": {
+      "content": "[unused78]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55107": {
+      "content": "[unused79]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55108": {
+      "content": "[unused80]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55109": {
+      "content": "[unused81]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "55110": {
+      "content": "[unused82]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": "[UNK]"
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,2735 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.9957679860592482,
+  "eval_steps": 250,
+  "global_step": 3500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0028450513887407092,
+      "grad_norm": 0.2809712886810303,
+      "learning_rate": 4.445629945763315e-08,
+      "loss": 0.0202,
+      "step": 10
+    },
+    {
+      "epoch": 0.0056901027774814185,
+      "grad_norm": 0.2773303985595703,
+      "learning_rate": 8.89125989152663e-08,
+      "loss": 0.0184,
+      "step": 20
+    },
+    {
+      "epoch": 0.008535154166222128,
+      "grad_norm": 0.28651171922683716,
+      "learning_rate": 1.3336889837289946e-07,
+      "loss": 0.018,
+      "step": 30
+    },
+    {
+      "epoch": 0.011380205554962837,
+      "grad_norm": 0.24912181496620178,
+      "learning_rate": 1.778251978305326e-07,
+      "loss": 0.0173,
+      "step": 40
+    },
+    {
+      "epoch": 0.014225256943703546,
+      "grad_norm": 0.2654182016849518,
+      "learning_rate": 2.2228149728816572e-07,
+      "loss": 0.0193,
+      "step": 50
+    },
+    {
+      "epoch": 0.017070308332444255,
+      "grad_norm": 0.2645716965198517,
+      "learning_rate": 2.667377967457989e-07,
+      "loss": 0.0158,
+      "step": 60
+    },
+    {
+      "epoch": 0.019915359721184963,
+      "grad_norm": 0.28719639778137207,
+      "learning_rate": 3.1119409620343207e-07,
+      "loss": 0.016,
+      "step": 70
+    },
+    {
+      "epoch": 0.022760411109925674,
+      "grad_norm": 0.24205148220062256,
+      "learning_rate": 3.556503956610652e-07,
+      "loss": 0.0139,
+      "step": 80
+    },
+    {
+      "epoch": 0.02560546249866638,
+      "grad_norm": 0.1541828215122223,
+      "learning_rate": 4.0010669511869836e-07,
+      "loss": 0.0143,
+      "step": 90
+    },
+    {
+      "epoch": 0.028450513887407092,
+      "grad_norm": 0.24321648478507996,
+      "learning_rate": 4.4456299457633145e-07,
+      "loss": 0.0138,
+      "step": 100
+    },
+    {
+      "epoch": 0.0312955652761478,
+      "grad_norm": 0.13496069610118866,
+      "learning_rate": 4.890192940339646e-07,
+      "loss": 0.0127,
+      "step": 110
+    },
+    {
+      "epoch": 0.03414061666488851,
+      "grad_norm": 0.1574280858039856,
+      "learning_rate": 5.334755934915978e-07,
+      "loss": 0.0115,
+      "step": 120
+    },
+    {
+      "epoch": 0.03698566805362922,
+      "grad_norm": 0.16117185354232788,
+      "learning_rate": 5.779318929492309e-07,
+      "loss": 0.0117,
+      "step": 130
+    },
+    {
+      "epoch": 0.039830719442369926,
+      "grad_norm": 0.11417609453201294,
+      "learning_rate": 6.223881924068641e-07,
+      "loss": 0.0111,
+      "step": 140
+    },
+    {
+      "epoch": 0.04267577083111064,
+      "grad_norm": 0.13975922763347626,
+      "learning_rate": 6.668444918644972e-07,
+      "loss": 0.0111,
+      "step": 150
+    },
+    {
+      "epoch": 0.04552082221985135,
+      "grad_norm": 0.10122673958539963,
+      "learning_rate": 7.113007913221304e-07,
+      "loss": 0.0106,
+      "step": 160
+    },
+    {
+      "epoch": 0.048365873608592055,
+      "grad_norm": 0.10620034486055374,
+      "learning_rate": 7.557570907797635e-07,
+      "loss": 0.01,
+      "step": 170
+    },
+    {
+      "epoch": 0.05121092499733276,
+      "grad_norm": 0.10404311865568161,
+      "learning_rate": 8.002133902373967e-07,
+      "loss": 0.0103,
+      "step": 180
+    },
+    {
+      "epoch": 0.05405597638607347,
+      "grad_norm": 0.09400220960378647,
+      "learning_rate": 8.446696896950297e-07,
+      "loss": 0.0106,
+      "step": 190
+    },
+    {
+      "epoch": 0.056901027774814185,
+      "grad_norm": 0.0968412309885025,
+      "learning_rate": 8.891259891526629e-07,
+      "loss": 0.0102,
+      "step": 200
+    },
+    {
+      "epoch": 0.05974607916355489,
+      "grad_norm": 0.10527963936328888,
+      "learning_rate": 9.335822886102961e-07,
+      "loss": 0.0103,
+      "step": 210
+    },
+    {
+      "epoch": 0.0625911305522956,
+      "grad_norm": 0.11160361021757126,
+      "learning_rate": 9.780385880679293e-07,
+      "loss": 0.0109,
+      "step": 220
+    },
+    {
+      "epoch": 0.06543618194103631,
+      "grad_norm": 0.09594683349132538,
+      "learning_rate": 1.0224948875255625e-06,
+      "loss": 0.0099,
+      "step": 230
+    },
+    {
+      "epoch": 0.06828123332977702,
+      "grad_norm": 0.09191753715276718,
+      "learning_rate": 1.0669511869831957e-06,
+      "loss": 0.0086,
+      "step": 240
+    },
+    {
+      "epoch": 0.07112628471851773,
+      "grad_norm": 0.10616082698106766,
+      "learning_rate": 1.1114074864408287e-06,
+      "loss": 0.01,
+      "step": 250
+    },
+    {
+      "epoch": 0.07112628471851773,
+      "eval_loss": 0.04481230676174164,
+      "eval_runtime": 8.9152,
+      "eval_samples_per_second": 168.253,
+      "eval_steps_per_second": 10.544,
+      "eval_sts_dev_pearson_cosine": 0.7566652114171531,
+      "eval_sts_dev_pearson_dot": 0.6030696212508195,
+      "eval_sts_dev_pearson_euclidean": 0.723338411802702,
+      "eval_sts_dev_pearson_manhattan": 0.7229820157274984,
+      "eval_sts_dev_pearson_max": 0.7566652114171531,
+      "eval_sts_dev_spearman_cosine": 0.7641649126837959,
+      "eval_sts_dev_spearman_dot": 0.5924781561865081,
+      "eval_sts_dev_spearman_euclidean": 0.7161218303280887,
+      "eval_sts_dev_spearman_manhattan": 0.7161351439672354,
+      "eval_sts_dev_spearman_max": 0.7641649126837959,
+      "step": 250
+    },
+    {
+      "epoch": 0.07397133610725844,
+      "grad_norm": 0.09024439752101898,
+      "learning_rate": 1.1558637858984619e-06,
+      "loss": 0.0098,
+      "step": 260
+    },
+    {
+      "epoch": 0.07681638749599914,
+      "grad_norm": 0.10580305010080338,
+      "learning_rate": 1.200320085356095e-06,
+      "loss": 0.0094,
+      "step": 270
+    },
+    {
+      "epoch": 0.07966143888473985,
+      "grad_norm": 0.07469004392623901,
+      "learning_rate": 1.2447763848137283e-06,
+      "loss": 0.0097,
+      "step": 280
+    },
+    {
+      "epoch": 0.08250649027348056,
+      "grad_norm": 0.08532268553972244,
+      "learning_rate": 1.2892326842713615e-06,
+      "loss": 0.0094,
+      "step": 290
+    },
+    {
+      "epoch": 0.08535154166222128,
+      "grad_norm": 0.11069530993700027,
+      "learning_rate": 1.3336889837289944e-06,
+      "loss": 0.0095,
+      "step": 300
+    },
+    {
+      "epoch": 0.08819659305096199,
+      "grad_norm": 0.14181849360466003,
+      "learning_rate": 1.3781452831866276e-06,
+      "loss": 0.0098,
+      "step": 310
+    },
+    {
+      "epoch": 0.0910416444397027,
+      "grad_norm": 0.10291895270347595,
+      "learning_rate": 1.4226015826442608e-06,
+      "loss": 0.0092,
+      "step": 320
+    },
+    {
+      "epoch": 0.0938866958284434,
+      "grad_norm": 0.09878888726234436,
+      "learning_rate": 1.467057882101894e-06,
+      "loss": 0.0095,
+      "step": 330
+    },
+    {
+      "epoch": 0.09673174721718411,
+      "grad_norm": 0.11313822865486145,
+      "learning_rate": 1.511514181559527e-06,
+      "loss": 0.0103,
+      "step": 340
+    },
+    {
+      "epoch": 0.09957679860592482,
+      "grad_norm": 0.10922758281230927,
+      "learning_rate": 1.5559704810171602e-06,
+      "loss": 0.0097,
+      "step": 350
+    },
+    {
+      "epoch": 0.10242184999466553,
+      "grad_norm": 0.09477540105581284,
+      "learning_rate": 1.6004267804747934e-06,
+      "loss": 0.0091,
+      "step": 360
+    },
+    {
+      "epoch": 0.10526690138340623,
+      "grad_norm": 0.11776648461818695,
+      "learning_rate": 1.6448830799324264e-06,
+      "loss": 0.0094,
+      "step": 370
+    },
+    {
+      "epoch": 0.10811195277214694,
+      "grad_norm": 0.10813190042972565,
+      "learning_rate": 1.6893393793900594e-06,
+      "loss": 0.0088,
+      "step": 380
+    },
+    {
+      "epoch": 0.11095700416088766,
+      "grad_norm": 0.0988766998052597,
+      "learning_rate": 1.7337956788476928e-06,
+      "loss": 0.009,
+      "step": 390
+    },
+    {
+      "epoch": 0.11380205554962837,
+      "grad_norm": 0.11297037452459335,
+      "learning_rate": 1.7782519783053258e-06,
+      "loss": 0.0098,
+      "step": 400
+    },
+    {
+      "epoch": 0.11664710693836908,
+      "grad_norm": 0.11383142322301865,
+      "learning_rate": 1.8227082777629592e-06,
+      "loss": 0.0083,
+      "step": 410
+    },
+    {
+      "epoch": 0.11949215832710978,
+      "grad_norm": 0.09991955757141113,
+      "learning_rate": 1.8671645772205922e-06,
+      "loss": 0.0099,
+      "step": 420
+    },
+    {
+      "epoch": 0.12233720971585049,
+      "grad_norm": 0.12355756759643555,
+      "learning_rate": 1.911620876678225e-06,
+      "loss": 0.0094,
+      "step": 430
+    },
+    {
+      "epoch": 0.1251822611045912,
+      "grad_norm": 0.13183994591236115,
+      "learning_rate": 1.9560771761358586e-06,
+      "loss": 0.0092,
+      "step": 440
+    },
+    {
+      "epoch": 0.1280273124933319,
+      "grad_norm": 0.09731684625148773,
+      "learning_rate": 2.0005334755934916e-06,
+      "loss": 0.009,
+      "step": 450
+    },
+    {
+      "epoch": 0.13087236388207263,
+      "grad_norm": 0.10299917310476303,
+      "learning_rate": 2.044989775051125e-06,
+      "loss": 0.0088,
+      "step": 460
+    },
+    {
+      "epoch": 0.13371741527081332,
+      "grad_norm": 0.10299093276262283,
+      "learning_rate": 2.089446074508758e-06,
+      "loss": 0.0092,
+      "step": 470
+    },
+    {
+      "epoch": 0.13656246665955404,
+      "grad_norm": 0.10356424003839493,
+      "learning_rate": 2.1339023739663914e-06,
+      "loss": 0.0083,
+      "step": 480
+    },
+    {
+      "epoch": 0.13940751804829474,
+      "grad_norm": 0.09914368391036987,
+      "learning_rate": 2.1783586734240244e-06,
+      "loss": 0.0089,
+      "step": 490
+    },
+    {
+      "epoch": 0.14225256943703546,
+      "grad_norm": 0.09933792054653168,
+      "learning_rate": 2.2228149728816573e-06,
+      "loss": 0.0089,
+      "step": 500
+    },
+    {
+      "epoch": 0.14225256943703546,
+      "eval_loss": 0.044373366981744766,
+      "eval_runtime": 9.347,
+      "eval_samples_per_second": 160.479,
+      "eval_steps_per_second": 10.057,
+      "eval_sts_dev_pearson_cosine": 0.7648120608553326,
+      "eval_sts_dev_pearson_dot": 0.5968017368208518,
+      "eval_sts_dev_pearson_euclidean": 0.7279404616700573,
+      "eval_sts_dev_pearson_manhattan": 0.7275721595620177,
+      "eval_sts_dev_pearson_max": 0.7648120608553326,
+      "eval_sts_dev_spearman_cosine": 0.7725220548635785,
+      "eval_sts_dev_spearman_dot": 0.5821758472972447,
+      "eval_sts_dev_spearman_euclidean": 0.7216429612052142,
+      "eval_sts_dev_spearman_manhattan": 0.7215280543286006,
+      "eval_sts_dev_spearman_max": 0.7725220548635785,
+      "step": 500
+    },
+    {
+      "epoch": 0.14509762082577615,
+      "grad_norm": 0.09889407455921173,
+      "learning_rate": 2.2672712723392908e-06,
+      "loss": 0.0095,
+      "step": 510
+    },
+    {
+      "epoch": 0.14794267221451687,
+      "grad_norm": 0.08651946485042572,
+      "learning_rate": 2.3117275717969237e-06,
+      "loss": 0.0095,
+      "step": 520
+    },
+    {
+      "epoch": 0.1507877236032576,
+      "grad_norm": 0.11702941358089447,
+      "learning_rate": 2.356183871254557e-06,
+      "loss": 0.0091,
+      "step": 530
+    },
+    {
+      "epoch": 0.1536327749919983,
+      "grad_norm": 0.0989551842212677,
+      "learning_rate": 2.40064017071219e-06,
+      "loss": 0.0082,
+      "step": 540
+    },
+    {
+      "epoch": 0.156477826380739,
+      "grad_norm": 0.13362008333206177,
+      "learning_rate": 2.445096470169823e-06,
+      "loss": 0.0091,
+      "step": 550
+    },
+    {
+      "epoch": 0.1593228777694797,
+      "grad_norm": 0.09574282169342041,
+      "learning_rate": 2.4895527696274565e-06,
+      "loss": 0.0086,
+      "step": 560
+    },
+    {
+      "epoch": 0.16216792915822043,
+      "grad_norm": 0.10629838705062866,
+      "learning_rate": 2.5340090690850895e-06,
+      "loss": 0.009,
+      "step": 570
+    },
+    {
+      "epoch": 0.16501298054696112,
+      "grad_norm": 0.09541832655668259,
+      "learning_rate": 2.578465368542723e-06,
+      "loss": 0.0088,
+      "step": 580
+    },
+    {
+      "epoch": 0.16785803193570184,
+      "grad_norm": 0.1058318167924881,
+      "learning_rate": 2.622921668000356e-06,
+      "loss": 0.0087,
+      "step": 590
+    },
+    {
+      "epoch": 0.17070308332444256,
+      "grad_norm": 0.09181062877178192,
+      "learning_rate": 2.667377967457989e-06,
+      "loss": 0.0089,
+      "step": 600
+    },
+    {
+      "epoch": 0.17354813471318326,
+      "grad_norm": 0.1417740136384964,
+      "learning_rate": 2.7118342669156223e-06,
+      "loss": 0.009,
+      "step": 610
+    },
+    {
+      "epoch": 0.17639318610192398,
+      "grad_norm": 0.11063350737094879,
+      "learning_rate": 2.7562905663732553e-06,
+      "loss": 0.0088,
+      "step": 620
+    },
+    {
+      "epoch": 0.17923823749066467,
+      "grad_norm": 0.09422960132360458,
+      "learning_rate": 2.8007468658308887e-06,
+      "loss": 0.0088,
+      "step": 630
+    },
+    {
+      "epoch": 0.1820832888794054,
+      "grad_norm": 0.08875516802072525,
+      "learning_rate": 2.8452031652885217e-06,
+      "loss": 0.0081,
+      "step": 640
+    },
+    {
+      "epoch": 0.18492834026814609,
+      "grad_norm": 0.08624540269374847,
+      "learning_rate": 2.8896594647461547e-06,
+      "loss": 0.0082,
+      "step": 650
+    },
+    {
+      "epoch": 0.1877733916568868,
+      "grad_norm": 0.09943191707134247,
+      "learning_rate": 2.934115764203788e-06,
+      "loss": 0.0088,
+      "step": 660
+    },
+    {
+      "epoch": 0.1906184430456275,
+      "grad_norm": 0.08673301339149475,
+      "learning_rate": 2.978572063661421e-06,
+      "loss": 0.0086,
+      "step": 670
+    },
+    {
+      "epoch": 0.19346349443436822,
+      "grad_norm": 0.090702585875988,
+      "learning_rate": 3.023028363119054e-06,
+      "loss": 0.0085,
+      "step": 680
+    },
+    {
+      "epoch": 0.19630854582310894,
+      "grad_norm": 0.08411288261413574,
+      "learning_rate": 3.067484662576687e-06,
+      "loss": 0.009,
+      "step": 690
+    },
+    {
+      "epoch": 0.19915359721184964,
+      "grad_norm": 0.09590886533260345,
+      "learning_rate": 3.1119409620343205e-06,
+      "loss": 0.0083,
+      "step": 700
+    },
+    {
+      "epoch": 0.20199864860059036,
+      "grad_norm": 0.12336103618144989,
+      "learning_rate": 3.1563972614919534e-06,
+      "loss": 0.0088,
+      "step": 710
+    },
+    {
+      "epoch": 0.20484369998933105,
+      "grad_norm": 0.08143921196460724,
+      "learning_rate": 3.200853560949587e-06,
+      "loss": 0.0088,
+      "step": 720
+    },
+    {
+      "epoch": 0.20768875137807177,
+      "grad_norm": 0.09164416790008545,
+      "learning_rate": 3.24530986040722e-06,
+      "loss": 0.0087,
+      "step": 730
+    },
+    {
+      "epoch": 0.21053380276681247,
+      "grad_norm": 0.1068928986787796,
+      "learning_rate": 3.289766159864853e-06,
+      "loss": 0.0088,
+      "step": 740
+    },
+    {
+      "epoch": 0.2133788541555532,
+      "grad_norm": 0.08625414967536926,
+      "learning_rate": 3.3342224593224862e-06,
+      "loss": 0.008,
+      "step": 750
+    },
+    {
+      "epoch": 0.2133788541555532,
+      "eval_loss": 0.04651115834712982,
+      "eval_runtime": 9.062,
+      "eval_samples_per_second": 165.526,
+      "eval_steps_per_second": 10.373,
+      "eval_sts_dev_pearson_cosine": 0.769180082275811,
+      "eval_sts_dev_pearson_dot": 0.5978598585684491,
+      "eval_sts_dev_pearson_euclidean": 0.7336764223199792,
+      "eval_sts_dev_pearson_manhattan": 0.7333470603433799,
+      "eval_sts_dev_pearson_max": 0.769180082275811,
+      "eval_sts_dev_spearman_cosine": 0.779812353514179,
+      "eval_sts_dev_spearman_dot": 0.5813102081634336,
+      "eval_sts_dev_spearman_euclidean": 0.7286880899787377,
+      "eval_sts_dev_spearman_manhattan": 0.728602343078262,
+      "eval_sts_dev_spearman_max": 0.779812353514179,
+      "step": 750
+    },
+    {
+      "epoch": 0.21622390554429388,
+      "grad_norm": 0.13600626587867737,
+      "learning_rate": 3.378678758780119e-06,
+      "loss": 0.0087,
+      "step": 760
+    },
+    {
+      "epoch": 0.2190689569330346,
+      "grad_norm": 0.14500027894973755,
+      "learning_rate": 3.423135058237752e-06,
+      "loss": 0.0087,
+      "step": 770
+    },
+    {
+      "epoch": 0.22191400832177532,
+      "grad_norm": 0.10052921622991562,
+      "learning_rate": 3.4675913576953856e-06,
+      "loss": 0.009,
+      "step": 780
+    },
+    {
+      "epoch": 0.22475905971051602,
+      "grad_norm": 0.08056453615427017,
+      "learning_rate": 3.512047657153019e-06,
+      "loss": 0.0085,
+      "step": 790
+    },
+    {
+      "epoch": 0.22760411109925674,
+      "grad_norm": 0.08645027875900269,
+      "learning_rate": 3.5565039566106516e-06,
+      "loss": 0.009,
+      "step": 800
+    },
+    {
+      "epoch": 0.23044916248799743,
+      "grad_norm": 0.12497828155755997,
+      "learning_rate": 3.600960256068285e-06,
+      "loss": 0.0082,
+      "step": 810
+    },
+    {
+      "epoch": 0.23329421387673815,
+      "grad_norm": 0.06854041665792465,
+      "learning_rate": 3.6454165555259184e-06,
+      "loss": 0.0073,
+      "step": 820
+    },
+    {
+      "epoch": 0.23613926526547885,
+      "grad_norm": 0.0781393051147461,
+      "learning_rate": 3.689872854983551e-06,
+      "loss": 0.0078,
+      "step": 830
+    },
+    {
+      "epoch": 0.23898431665421957,
+      "grad_norm": 0.09048033505678177,
+      "learning_rate": 3.7343291544411844e-06,
+      "loss": 0.0088,
+      "step": 840
+    },
+    {
+      "epoch": 0.24182936804296026,
+      "grad_norm": 0.10866343975067139,
+      "learning_rate": 3.7787854538988178e-06,
+      "loss": 0.0077,
+      "step": 850
+    },
+    {
+      "epoch": 0.24467441943170098,
+      "grad_norm": 0.08551909774541855,
+      "learning_rate": 3.82324175335645e-06,
+      "loss": 0.008,
+      "step": 860
+    },
+    {
+      "epoch": 0.2475194708204417,
+      "grad_norm": 0.10334528237581253,
+      "learning_rate": 3.867698052814084e-06,
+      "loss": 0.008,
+      "step": 870
+    },
+    {
+      "epoch": 0.2503645222091824,
+      "grad_norm": 0.10668737441301346,
+      "learning_rate": 3.912154352271717e-06,
+      "loss": 0.0086,
+      "step": 880
+    },
+    {
+      "epoch": 0.2532095735979231,
+      "grad_norm": 0.09555123746395111,
+      "learning_rate": 3.9566106517293506e-06,
+      "loss": 0.0083,
+      "step": 890
+    },
+    {
+      "epoch": 0.2560546249866638,
+      "grad_norm": 0.09091876447200775,
+      "learning_rate": 4.001066951186983e-06,
+      "loss": 0.0081,
+      "step": 900
+    },
+    {
+      "epoch": 0.2588996763754045,
+      "grad_norm": 0.10903967916965485,
+      "learning_rate": 4.0455232506446165e-06,
+      "loss": 0.0081,
+      "step": 910
+    },
+    {
+      "epoch": 0.26174472776414526,
+      "grad_norm": 0.0973634123802185,
+      "learning_rate": 4.08997955010225e-06,
+      "loss": 0.0077,
+      "step": 920
+    },
+    {
+      "epoch": 0.26458977915288595,
+      "grad_norm": 0.10415139049291611,
+      "learning_rate": 4.1344358495598825e-06,
+      "loss": 0.0083,
+      "step": 930
+    },
+    {
+      "epoch": 0.26743483054162664,
+      "grad_norm": 0.07658711075782776,
+      "learning_rate": 4.178892149017516e-06,
+      "loss": 0.0081,
+      "step": 940
+    },
+    {
+      "epoch": 0.2702798819303674,
+      "grad_norm": 0.08397387713193893,
+      "learning_rate": 4.223348448475149e-06,
+      "loss": 0.0069,
+      "step": 950
+    },
+    {
+      "epoch": 0.2731249333191081,
+      "grad_norm": 0.07437604665756226,
+      "learning_rate": 4.267804747932783e-06,
+      "loss": 0.0084,
+      "step": 960
+    },
+    {
+      "epoch": 0.2759699847078488,
+      "grad_norm": 0.08432639390230179,
+      "learning_rate": 4.312261047390415e-06,
+      "loss": 0.0075,
+      "step": 970
+    },
+    {
+      "epoch": 0.2788150360965895,
+      "grad_norm": 0.10041897743940353,
+      "learning_rate": 4.356717346848049e-06,
+      "loss": 0.0081,
+      "step": 980
+    },
+    {
+      "epoch": 0.2816600874853302,
+      "grad_norm": 0.08802913874387741,
+      "learning_rate": 4.401173646305682e-06,
+      "loss": 0.0086,
+      "step": 990
+    },
+    {
+      "epoch": 0.2845051388740709,
+      "grad_norm": 0.10163892060518265,
+      "learning_rate": 4.445629945763315e-06,
+      "loss": 0.0079,
+      "step": 1000
+    },
+    {
+      "epoch": 0.2845051388740709,
+      "eval_loss": 0.047278326004743576,
+      "eval_runtime": 8.8427,
+      "eval_samples_per_second": 169.631,
+      "eval_steps_per_second": 10.63,
+      "eval_sts_dev_pearson_cosine": 0.7757447848847815,
+      "eval_sts_dev_pearson_dot": 0.6005493714862752,
+      "eval_sts_dev_pearson_euclidean": 0.7410592947515149,
+      "eval_sts_dev_pearson_manhattan": 0.7408111418700313,
+      "eval_sts_dev_pearson_max": 0.7757447848847815,
+      "eval_sts_dev_spearman_cosine": 0.78547452833307,
+      "eval_sts_dev_spearman_dot": 0.582949767142449,
+      "eval_sts_dev_spearman_euclidean": 0.7377389195713249,
+      "eval_sts_dev_spearman_manhattan": 0.7378881505306938,
+      "eval_sts_dev_spearman_max": 0.78547452833307,
+      "step": 1000
+    },
+    {
+      "epoch": 0.2873501902628116,
+      "grad_norm": 0.08119315654039383,
+      "learning_rate": 4.490086245220948e-06,
+      "loss": 0.0088,
+      "step": 1010
+    },
+    {
+      "epoch": 0.2901952416515523,
+      "grad_norm": 0.10737801343202591,
+      "learning_rate": 4.5345425446785815e-06,
+      "loss": 0.0073,
+      "step": 1020
+    },
+    {
+      "epoch": 0.29304029304029305,
+      "grad_norm": 0.08908016234636307,
+      "learning_rate": 4.578998844136214e-06,
+      "loss": 0.008,
+      "step": 1030
+    },
+    {
+      "epoch": 0.29588534442903375,
+      "grad_norm": 0.08246352523565292,
+      "learning_rate": 4.6234551435938475e-06,
+      "loss": 0.0073,
+      "step": 1040
+    },
+    {
+      "epoch": 0.29873039581777444,
+      "grad_norm": 0.08022474497556686,
+      "learning_rate": 4.667911443051481e-06,
+      "loss": 0.008,
+      "step": 1050
+    },
+    {
+      "epoch": 0.3015754472065152,
+      "grad_norm": 0.07938782870769501,
+      "learning_rate": 4.712367742509114e-06,
+      "loss": 0.0074,
+      "step": 1060
+    },
+    {
+      "epoch": 0.3044204985952559,
+      "grad_norm": 0.07806035876274109,
+      "learning_rate": 4.756824041966747e-06,
+      "loss": 0.007,
+      "step": 1070
+    },
+    {
+      "epoch": 0.3072655499839966,
+      "grad_norm": 0.08204073458909988,
+      "learning_rate": 4.80128034142438e-06,
+      "loss": 0.0075,
+      "step": 1080
+    },
+    {
+      "epoch": 0.31011060137273727,
+      "grad_norm": 0.07709292322397232,
+      "learning_rate": 4.845736640882014e-06,
+      "loss": 0.0077,
+      "step": 1090
+    },
+    {
+      "epoch": 0.312955652761478,
+      "grad_norm": 0.09081903845071793,
+      "learning_rate": 4.890192940339646e-06,
+      "loss": 0.0076,
+      "step": 1100
+    },
+    {
+      "epoch": 0.3158007041502187,
+      "grad_norm": 0.0967966839671135,
+      "learning_rate": 4.93464923979728e-06,
+      "loss": 0.0082,
+      "step": 1110
+    },
+    {
+      "epoch": 0.3186457555389594,
+      "grad_norm": 0.07934273034334183,
+      "learning_rate": 4.979105539254913e-06,
+      "loss": 0.0073,
+      "step": 1120
+    },
+    {
+      "epoch": 0.32149080692770016,
+      "grad_norm": 0.07694579660892487,
+      "learning_rate": 5.023561838712546e-06,
+      "loss": 0.007,
+      "step": 1130
+    },
+    {
+      "epoch": 0.32433585831644085,
+      "grad_norm": 0.07895845174789429,
+      "learning_rate": 5.068018138170179e-06,
+      "loss": 0.0077,
+      "step": 1140
+    },
+    {
+      "epoch": 0.32718090970518154,
+      "grad_norm": 0.07453285902738571,
+      "learning_rate": 5.1124744376278124e-06,
+      "loss": 0.0074,
+      "step": 1150
+    },
+    {
+      "epoch": 0.33002596109392224,
+      "grad_norm": 0.08011069148778915,
+      "learning_rate": 5.156930737085446e-06,
+      "loss": 0.0076,
+      "step": 1160
+    },
+    {
+      "epoch": 0.332871012482663,
+      "grad_norm": 0.08325564116239548,
+      "learning_rate": 5.201387036543078e-06,
+      "loss": 0.0078,
+      "step": 1170
+    },
+    {
+      "epoch": 0.3357160638714037,
+      "grad_norm": 0.08292325586080551,
+      "learning_rate": 5.245843336000712e-06,
+      "loss": 0.0073,
+      "step": 1180
+    },
+    {
+      "epoch": 0.3385611152601444,
+      "grad_norm": 0.08605830371379852,
+      "learning_rate": 5.290299635458345e-06,
+      "loss": 0.0077,
+      "step": 1190
+    },
+    {
+      "epoch": 0.3414061666488851,
+      "grad_norm": 0.08384672552347183,
+      "learning_rate": 5.334755934915978e-06,
+      "loss": 0.0068,
+      "step": 1200
+    },
+    {
+      "epoch": 0.3442512180376258,
+      "grad_norm": 0.10930886119604111,
+      "learning_rate": 5.379212234373611e-06,
+      "loss": 0.0079,
+      "step": 1210
+    },
+    {
+      "epoch": 0.3470962694263665,
+      "grad_norm": 0.09999439120292664,
+      "learning_rate": 5.423668533831245e-06,
+      "loss": 0.0073,
+      "step": 1220
+    },
+    {
+      "epoch": 0.3499413208151072,
+      "grad_norm": 0.0711401030421257,
+      "learning_rate": 5.468124833288877e-06,
+      "loss": 0.0075,
+      "step": 1230
+    },
+    {
+      "epoch": 0.35278637220384795,
+      "grad_norm": 0.12141191959381104,
+      "learning_rate": 5.512581132746511e-06,
+      "loss": 0.0078,
+      "step": 1240
+    },
+    {
+      "epoch": 0.35563142359258865,
+      "grad_norm": 0.08158909529447556,
+      "learning_rate": 5.557037432204144e-06,
+      "loss": 0.0073,
+      "step": 1250
+    },
+    {
+      "epoch": 0.35563142359258865,
+      "eval_loss": 0.04716332256793976,
+      "eval_runtime": 9.2017,
+      "eval_samples_per_second": 163.013,
+      "eval_steps_per_second": 10.215,
+      "eval_sts_dev_pearson_cosine": 0.7745797315927636,
+      "eval_sts_dev_pearson_dot": 0.6045460896217866,
+      "eval_sts_dev_pearson_euclidean": 0.7454842399458257,
+      "eval_sts_dev_pearson_manhattan": 0.7450723972536097,
+      "eval_sts_dev_pearson_max": 0.7745797315927636,
+      "eval_sts_dev_spearman_cosine": 0.7854745787288134,
+      "eval_sts_dev_spearman_dot": 0.5869807295128947,
+      "eval_sts_dev_spearman_euclidean": 0.7432930573697278,
+      "eval_sts_dev_spearman_manhattan": 0.7431996603127268,
+      "eval_sts_dev_spearman_max": 0.7854745787288134,
+      "step": 1250
+    },
+    {
+      "epoch": 0.35847647498132934,
+      "grad_norm": 0.07299906015396118,
+      "learning_rate": 5.601493731661777e-06,
+      "loss": 0.0073,
+      "step": 1260
+    },
+    {
+      "epoch": 0.36132152637007003,
+      "grad_norm": 0.08581911772489548,
+      "learning_rate": 5.64595003111941e-06,
+      "loss": 0.007,
+      "step": 1270
+    },
+    {
+      "epoch": 0.3641665777588108,
+      "grad_norm": 0.08339793235063553,
+      "learning_rate": 5.690406330577043e-06,
+      "loss": 0.0068,
+      "step": 1280
+    },
+    {
+      "epoch": 0.3670116291475515,
+      "grad_norm": 0.08051007241010666,
+      "learning_rate": 5.734862630034677e-06,
+      "loss": 0.0067,
+      "step": 1290
+    },
+    {
+      "epoch": 0.36985668053629217,
+      "grad_norm": 0.08804050087928772,
+      "learning_rate": 5.779318929492309e-06,
+      "loss": 0.0078,
+      "step": 1300
+    },
+    {
+      "epoch": 0.3727017319250329,
+      "grad_norm": 0.07765672355890274,
+      "learning_rate": 5.823775228949943e-06,
+      "loss": 0.0072,
+      "step": 1310
+    },
+    {
+      "epoch": 0.3755467833137736,
+      "grad_norm": 0.08375009894371033,
+      "learning_rate": 5.868231528407576e-06,
+      "loss": 0.0071,
+      "step": 1320
+    },
+    {
+      "epoch": 0.3783918347025143,
+      "grad_norm": 0.07235526293516159,
+      "learning_rate": 5.912687827865209e-06,
+      "loss": 0.0068,
+      "step": 1330
+    },
+    {
+      "epoch": 0.381236886091255,
+      "grad_norm": 0.08521237969398499,
+      "learning_rate": 5.957144127322842e-06,
+      "loss": 0.0068,
+      "step": 1340
+    },
+    {
+      "epoch": 0.38408193747999575,
+      "grad_norm": 0.08466946333646774,
+      "learning_rate": 6.001600426780475e-06,
+      "loss": 0.0074,
+      "step": 1350
+    },
+    {
+      "epoch": 0.38692698886873644,
+      "grad_norm": 0.08605194091796875,
+      "learning_rate": 6.046056726238108e-06,
+      "loss": 0.0074,
+      "step": 1360
+    },
+    {
+      "epoch": 0.38977204025747714,
+      "grad_norm": 0.10898251086473465,
+      "learning_rate": 6.0905130256957415e-06,
+      "loss": 0.0077,
+      "step": 1370
+    },
+    {
+      "epoch": 0.3926170916462179,
+      "grad_norm": 0.07759184390306473,
+      "learning_rate": 6.134969325153374e-06,
+      "loss": 0.0069,
+      "step": 1380
+    },
+    {
+      "epoch": 0.3954621430349586,
+      "grad_norm": 0.09418889880180359,
+      "learning_rate": 6.1794256246110075e-06,
+      "loss": 0.0079,
+      "step": 1390
+    },
+    {
+      "epoch": 0.3983071944236993,
+      "grad_norm": 0.07198388129472733,
+      "learning_rate": 6.223881924068641e-06,
+      "loss": 0.0066,
+      "step": 1400
+    },
+    {
+      "epoch": 0.40115224581243997,
+      "grad_norm": 0.1127256527543068,
+      "learning_rate": 6.2683382235262735e-06,
+      "loss": 0.008,
+      "step": 1410
+    },
+    {
+      "epoch": 0.4039972972011807,
+      "grad_norm": 0.09208247065544128,
+      "learning_rate": 6.312794522983907e-06,
+      "loss": 0.008,
+      "step": 1420
+    },
+    {
+      "epoch": 0.4068423485899214,
+      "grad_norm": 0.08608128875494003,
+      "learning_rate": 6.35725082244154e-06,
+      "loss": 0.0071,
+      "step": 1430
+    },
+    {
+      "epoch": 0.4096873999786621,
+      "grad_norm": 0.09031302481889725,
+      "learning_rate": 6.401707121899174e-06,
+      "loss": 0.0066,
+      "step": 1440
+    },
+    {
+      "epoch": 0.4125324513674028,
+      "grad_norm": 0.08052125573158264,
+      "learning_rate": 6.446163421356806e-06,
+      "loss": 0.0079,
+      "step": 1450
+    },
+    {
+      "epoch": 0.41537750275614355,
+      "grad_norm": 0.07241199910640717,
+      "learning_rate": 6.49061972081444e-06,
+      "loss": 0.0075,
+      "step": 1460
+    },
+    {
+      "epoch": 0.41822255414488424,
+      "grad_norm": 0.10164492577314377,
+      "learning_rate": 6.535076020272072e-06,
+      "loss": 0.0066,
+      "step": 1470
+    },
+    {
+      "epoch": 0.42106760553362493,
+      "grad_norm": 0.08544593304395676,
+      "learning_rate": 6.579532319729706e-06,
+      "loss": 0.007,
+      "step": 1480
+    },
+    {
+      "epoch": 0.4239126569223657,
+      "grad_norm": 0.11136358976364136,
+      "learning_rate": 6.623988619187339e-06,
+      "loss": 0.0066,
+      "step": 1490
+    },
+    {
+      "epoch": 0.4267577083111064,
+      "grad_norm": 0.07907264679670334,
+      "learning_rate": 6.6684449186449725e-06,
+      "loss": 0.0066,
+      "step": 1500
+    },
+    {
+      "epoch": 0.4267577083111064,
+      "eval_loss": 0.04738680273294449,
+      "eval_runtime": 8.8516,
+      "eval_samples_per_second": 169.462,
+      "eval_steps_per_second": 10.62,
+      "eval_sts_dev_pearson_cosine": 0.7790001769076627,
+      "eval_sts_dev_pearson_dot": 0.6181753135926376,
+      "eval_sts_dev_pearson_euclidean": 0.7499901311425706,
+      "eval_sts_dev_pearson_manhattan": 0.7495768624913272,
+      "eval_sts_dev_pearson_max": 0.7790001769076627,
+      "eval_sts_dev_spearman_cosine": 0.7907671099319872,
+      "eval_sts_dev_spearman_dot": 0.5999468232206078,
+      "eval_sts_dev_spearman_euclidean": 0.7486926337135288,
+      "eval_sts_dev_spearman_manhattan": 0.7484812166973952,
+      "eval_sts_dev_spearman_max": 0.7907671099319872,
+      "step": 1500
+    },
+    {
+      "epoch": 0.42960275969984707,
+      "grad_norm": 0.0868421345949173,
+      "learning_rate": 6.712901218102606e-06,
+      "loss": 0.0075,
+      "step": 1510
+    },
+    {
+      "epoch": 0.43244781108858776,
+      "grad_norm": 0.08232498168945312,
+      "learning_rate": 6.757357517560238e-06,
+      "loss": 0.0072,
+      "step": 1520
+    },
+    {
+      "epoch": 0.4352928624773285,
+      "grad_norm": 0.08831491321325302,
+      "learning_rate": 6.801813817017871e-06,
+      "loss": 0.0072,
+      "step": 1530
+    },
+    {
+      "epoch": 0.4381379138660692,
+      "grad_norm": 0.09035244584083557,
+      "learning_rate": 6.846270116475504e-06,
+      "loss": 0.0067,
+      "step": 1540
+    },
+    {
+      "epoch": 0.4409829652548099,
+      "grad_norm": 0.09386060386896133,
+      "learning_rate": 6.890726415933138e-06,
+      "loss": 0.0073,
+      "step": 1550
+    },
+    {
+      "epoch": 0.44382801664355065,
+      "grad_norm": 0.07013905048370361,
+      "learning_rate": 6.935182715390771e-06,
+      "loss": 0.0066,
+      "step": 1560
+    },
+    {
+      "epoch": 0.44667306803229134,
+      "grad_norm": 0.09007762372493744,
+      "learning_rate": 6.979639014848405e-06,
+      "loss": 0.0063,
+      "step": 1570
+    },
+    {
+      "epoch": 0.44951811942103204,
+      "grad_norm": 0.08053620904684067,
+      "learning_rate": 7.024095314306038e-06,
+      "loss": 0.0074,
+      "step": 1580
+    },
+    {
+      "epoch": 0.45236317080977273,
+      "grad_norm": 0.09655388444662094,
+      "learning_rate": 7.06855161376367e-06,
+      "loss": 0.0075,
+      "step": 1590
+    },
+    {
+      "epoch": 0.4552082221985135,
+      "grad_norm": 0.07398466765880585,
+      "learning_rate": 7.113007913221303e-06,
+      "loss": 0.0069,
+      "step": 1600
+    },
+    {
+      "epoch": 0.4580532735872542,
+      "grad_norm": 0.08047506213188171,
+      "learning_rate": 7.1574642126789366e-06,
+      "loss": 0.0065,
+      "step": 1610
+    },
+    {
+      "epoch": 0.46089832497599487,
+      "grad_norm": 0.09212318807840347,
+      "learning_rate": 7.20192051213657e-06,
+      "loss": 0.007,
+      "step": 1620
+    },
+    {
+      "epoch": 0.4637433763647356,
+      "grad_norm": 0.08342114090919495,
+      "learning_rate": 7.246376811594203e-06,
+      "loss": 0.0067,
+      "step": 1630
+    },
+    {
+      "epoch": 0.4665884277534763,
+      "grad_norm": 0.06753776967525482,
+      "learning_rate": 7.290833111051837e-06,
+      "loss": 0.0067,
+      "step": 1640
+    },
+    {
+      "epoch": 0.469433479142217,
+      "grad_norm": 0.06800476461648941,
+      "learning_rate": 7.33528941050947e-06,
+      "loss": 0.0072,
+      "step": 1650
+    },
+    {
+      "epoch": 0.4722785305309577,
+      "grad_norm": 0.08325930684804916,
+      "learning_rate": 7.379745709967102e-06,
+      "loss": 0.007,
+      "step": 1660
+    },
+    {
+      "epoch": 0.47512358191969845,
+      "grad_norm": 0.0933527797460556,
+      "learning_rate": 7.424202009424735e-06,
+      "loss": 0.0078,
+      "step": 1670
+    },
+    {
+      "epoch": 0.47796863330843914,
+      "grad_norm": 0.08404399454593658,
+      "learning_rate": 7.468658308882369e-06,
+      "loss": 0.0069,
+      "step": 1680
+    },
+    {
+      "epoch": 0.48081368469717983,
+      "grad_norm": 0.08951716870069504,
+      "learning_rate": 7.513114608340002e-06,
+      "loss": 0.0067,
+      "step": 1690
+    },
+    {
+      "epoch": 0.4836587360859205,
+      "grad_norm": 0.0985400453209877,
+      "learning_rate": 7.5575709077976356e-06,
+      "loss": 0.0072,
+      "step": 1700
+    },
+    {
+      "epoch": 0.4865037874746613,
+      "grad_norm": 0.08860517293214798,
+      "learning_rate": 7.602027207255269e-06,
+      "loss": 0.0071,
+      "step": 1710
+    },
+    {
+      "epoch": 0.48934883886340197,
+      "grad_norm": 0.09596813470125198,
+      "learning_rate": 7.6464835067129e-06,
+      "loss": 0.0069,
+      "step": 1720
+    },
+    {
+      "epoch": 0.49219389025214266,
+      "grad_norm": 0.08169304579496384,
+      "learning_rate": 7.690939806170534e-06,
+      "loss": 0.0074,
+      "step": 1730
+    },
+    {
+      "epoch": 0.4950389416408834,
+      "grad_norm": 0.08802200853824615,
+      "learning_rate": 7.735396105628168e-06,
+      "loss": 0.0073,
+      "step": 1740
+    },
+    {
+      "epoch": 0.4978839930296241,
+      "grad_norm": 0.10039868205785751,
+      "learning_rate": 7.779852405085801e-06,
+      "loss": 0.0064,
+      "step": 1750
+    },
+    {
+      "epoch": 0.4978839930296241,
+      "eval_loss": 0.04991479963064194,
+      "eval_runtime": 9.5702,
+      "eval_samples_per_second": 156.737,
+      "eval_steps_per_second": 9.822,
+      "eval_sts_dev_pearson_cosine": 0.7809326345755484,
+      "eval_sts_dev_pearson_dot": 0.6075613489678238,
+      "eval_sts_dev_pearson_euclidean": 0.7492249179863018,
+      "eval_sts_dev_pearson_manhattan": 0.748824124010157,
+      "eval_sts_dev_pearson_max": 0.7809326345755484,
+      "eval_sts_dev_spearman_cosine": 0.7937663255592566,
+      "eval_sts_dev_spearman_dot": 0.5920294462594152,
+      "eval_sts_dev_spearman_euclidean": 0.7488717447673626,
+      "eval_sts_dev_spearman_manhattan": 0.7486136956534813,
+      "eval_sts_dev_spearman_max": 0.7937663255592566,
+      "step": 1750
+    },
+    {
+      "epoch": 0.5007290444183649,
+      "grad_norm": 0.08260150998830795,
+      "learning_rate": 7.824308704543434e-06,
+      "loss": 0.0064,
+      "step": 1760
+    },
+    {
+      "epoch": 0.5035740958071055,
+      "grad_norm": 0.05973382294178009,
+      "learning_rate": 7.868765004001068e-06,
+      "loss": 0.0068,
+      "step": 1770
+    },
+    {
+      "epoch": 0.5064191471958462,
+      "grad_norm": 0.08796348422765732,
+      "learning_rate": 7.913221303458701e-06,
+      "loss": 0.007,
+      "step": 1780
+    },
+    {
+      "epoch": 0.5092641985845869,
+      "grad_norm": 0.10702888667583466,
+      "learning_rate": 7.957677602916333e-06,
+      "loss": 0.0065,
+      "step": 1790
+    },
+    {
+      "epoch": 0.5121092499733276,
+      "grad_norm": 0.07452105730772018,
+      "learning_rate": 8.002133902373966e-06,
+      "loss": 0.0073,
+      "step": 1800
+    },
+    {
+      "epoch": 0.5149543013620683,
+      "grad_norm": 0.07454142719507217,
+      "learning_rate": 8.0465902018316e-06,
+      "loss": 0.0061,
+      "step": 1810
+    },
+    {
+      "epoch": 0.517799352750809,
+      "grad_norm": 0.08079402148723602,
+      "learning_rate": 8.091046501289233e-06,
+      "loss": 0.0071,
+      "step": 1820
+    },
+    {
+      "epoch": 0.5206444041395498,
+      "grad_norm": 0.05563436076045036,
+      "learning_rate": 8.135502800746867e-06,
+      "loss": 0.0058,
+      "step": 1830
+    },
+    {
+      "epoch": 0.5234894555282905,
+      "grad_norm": 0.08133077621459961,
+      "learning_rate": 8.1799591002045e-06,
+      "loss": 0.0065,
+      "step": 1840
+    },
+    {
+      "epoch": 0.5263345069170312,
+      "grad_norm": 0.06776826083660126,
+      "learning_rate": 8.224415399662133e-06,
+      "loss": 0.0067,
+      "step": 1850
+    },
+    {
+      "epoch": 0.5291795583057719,
+      "grad_norm": 0.07137738913297653,
+      "learning_rate": 8.268871699119765e-06,
+      "loss": 0.0063,
+      "step": 1860
+    },
+    {
+      "epoch": 0.5320246096945126,
+      "grad_norm": 0.08924838155508041,
+      "learning_rate": 8.313327998577398e-06,
+      "loss": 0.007,
+      "step": 1870
+    },
+    {
+      "epoch": 0.5348696610832533,
+      "grad_norm": 0.10980788618326187,
+      "learning_rate": 8.357784298035032e-06,
+      "loss": 0.0069,
+      "step": 1880
+    },
+    {
+      "epoch": 0.537714712471994,
+      "grad_norm": 0.10096590220928192,
+      "learning_rate": 8.402240597492665e-06,
+      "loss": 0.0073,
+      "step": 1890
+    },
+    {
+      "epoch": 0.5405597638607348,
+      "grad_norm": 0.06633611768484116,
+      "learning_rate": 8.446696896950299e-06,
+      "loss": 0.0067,
+      "step": 1900
+    },
+    {
+      "epoch": 0.5434048152494755,
+      "grad_norm": 0.0696336105465889,
+      "learning_rate": 8.491153196407932e-06,
+      "loss": 0.0068,
+      "step": 1910
+    },
+    {
+      "epoch": 0.5462498666382162,
+      "grad_norm": 0.0785793736577034,
+      "learning_rate": 8.535609495865565e-06,
+      "loss": 0.0066,
+      "step": 1920
+    },
+    {
+      "epoch": 0.5490949180269569,
+      "grad_norm": 0.10194658488035202,
+      "learning_rate": 8.580065795323197e-06,
+      "loss": 0.007,
+      "step": 1930
+    },
+    {
+      "epoch": 0.5519399694156976,
+      "grad_norm": 0.07388205081224442,
+      "learning_rate": 8.62452209478083e-06,
+      "loss": 0.006,
+      "step": 1940
+    },
+    {
+      "epoch": 0.5547850208044383,
+      "grad_norm": 0.09130030125379562,
+      "learning_rate": 8.668978394238464e-06,
+      "loss": 0.0062,
+      "step": 1950
+    },
+    {
+      "epoch": 0.557630072193179,
+      "grad_norm": 0.06957433372735977,
+      "learning_rate": 8.713434693696097e-06,
+      "loss": 0.0062,
+      "step": 1960
+    },
+    {
+      "epoch": 0.5604751235819196,
+      "grad_norm": 0.07989591360092163,
+      "learning_rate": 8.75789099315373e-06,
+      "loss": 0.0067,
+      "step": 1970
+    },
+    {
+      "epoch": 0.5633201749706604,
+      "grad_norm": 0.06442303210496902,
+      "learning_rate": 8.802347292611364e-06,
+      "loss": 0.0063,
+      "step": 1980
+    },
+    {
+      "epoch": 0.5661652263594011,
+      "grad_norm": 0.0740804597735405,
+      "learning_rate": 8.846803592068996e-06,
+      "loss": 0.006,
+      "step": 1990
+    },
+    {
+      "epoch": 0.5690102777481418,
+      "grad_norm": 0.09938943386077881,
+      "learning_rate": 8.89125989152663e-06,
+      "loss": 0.0067,
+      "step": 2000
+    },
+    {
+      "epoch": 0.5690102777481418,
+      "eval_loss": 0.04781487584114075,
+      "eval_runtime": 9.1274,
+      "eval_samples_per_second": 164.341,
+      "eval_steps_per_second": 10.299,
+      "eval_sts_dev_pearson_cosine": 0.7835655100602283,
+      "eval_sts_dev_pearson_dot": 0.6140666203682124,
+      "eval_sts_dev_pearson_euclidean": 0.7549529684333918,
+      "eval_sts_dev_pearson_manhattan": 0.7544779011570287,
+      "eval_sts_dev_pearson_max": 0.7835655100602283,
+      "eval_sts_dev_spearman_cosine": 0.7943462202168604,
+      "eval_sts_dev_spearman_dot": 0.5976362907741744,
+      "eval_sts_dev_spearman_euclidean": 0.7553189959320907,
+      "eval_sts_dev_spearman_manhattan": 0.7548879863357982,
+      "eval_sts_dev_spearman_max": 0.7943462202168604,
+      "step": 2000
+    },
+    {
+      "epoch": 0.5718553291368825,
+      "grad_norm": 0.08203410357236862,
+      "learning_rate": 8.935716190984263e-06,
+      "loss": 0.0076,
+      "step": 2010
+    },
+    {
+      "epoch": 0.5747003805256232,
+      "grad_norm": 0.06707337498664856,
+      "learning_rate": 8.980172490441896e-06,
+      "loss": 0.0069,
+      "step": 2020
+    },
+    {
+      "epoch": 0.5775454319143639,
+      "grad_norm": 0.07895516604185104,
+      "learning_rate": 9.02462878989953e-06,
+      "loss": 0.0065,
+      "step": 2030
+    },
+    {
+      "epoch": 0.5803904833031046,
+      "grad_norm": 0.08560437709093094,
+      "learning_rate": 9.069085089357163e-06,
+      "loss": 0.007,
+      "step": 2040
+    },
+    {
+      "epoch": 0.5832355346918454,
+      "grad_norm": 0.07935027033090591,
+      "learning_rate": 9.113541388814796e-06,
+      "loss": 0.006,
+      "step": 2050
+    },
+    {
+      "epoch": 0.5860805860805861,
+      "grad_norm": 0.07242047786712646,
+      "learning_rate": 9.157997688272428e-06,
+      "loss": 0.0064,
+      "step": 2060
+    },
+    {
+      "epoch": 0.5889256374693268,
+      "grad_norm": 0.067719466984272,
+      "learning_rate": 9.202453987730062e-06,
+      "loss": 0.0063,
+      "step": 2070
+    },
+    {
+      "epoch": 0.5917706888580675,
+      "grad_norm": 0.06705283373594284,
+      "learning_rate": 9.246910287187695e-06,
+      "loss": 0.0067,
+      "step": 2080
+    },
+    {
+      "epoch": 0.5946157402468082,
+      "grad_norm": 0.07053534686565399,
+      "learning_rate": 9.291366586645328e-06,
+      "loss": 0.0064,
+      "step": 2090
+    },
+    {
+      "epoch": 0.5974607916355489,
+      "grad_norm": 0.07750871032476425,
+      "learning_rate": 9.335822886102962e-06,
+      "loss": 0.0062,
+      "step": 2100
+    },
+    {
+      "epoch": 0.6003058430242896,
+      "grad_norm": 0.0752708688378334,
+      "learning_rate": 9.380279185560595e-06,
+      "loss": 0.0063,
+      "step": 2110
+    },
+    {
+      "epoch": 0.6031508944130304,
+      "grad_norm": 0.06802825629711151,
+      "learning_rate": 9.424735485018229e-06,
+      "loss": 0.0063,
+      "step": 2120
+    },
+    {
+      "epoch": 0.6059959458017711,
+      "grad_norm": 0.08155028522014618,
+      "learning_rate": 9.46919178447586e-06,
+      "loss": 0.0074,
+      "step": 2130
+    },
+    {
+      "epoch": 0.6088409971905118,
+      "grad_norm": 0.09067688137292862,
+      "learning_rate": 9.513648083933494e-06,
+      "loss": 0.0067,
+      "step": 2140
+    },
+    {
+      "epoch": 0.6116860485792525,
+      "grad_norm": 0.06688214093446732,
+      "learning_rate": 9.558104383391127e-06,
+      "loss": 0.006,
+      "step": 2150
+    },
+    {
+      "epoch": 0.6145310999679932,
+      "grad_norm": 0.07848970592021942,
+      "learning_rate": 9.60256068284876e-06,
+      "loss": 0.0062,
+      "step": 2160
+    },
+    {
+      "epoch": 0.6173761513567338,
+      "grad_norm": 0.07614806294441223,
+      "learning_rate": 9.647016982306394e-06,
+      "loss": 0.007,
+      "step": 2170
+    },
+    {
+      "epoch": 0.6202212027454745,
+      "grad_norm": 0.07507840543985367,
+      "learning_rate": 9.691473281764027e-06,
+      "loss": 0.0069,
+      "step": 2180
+    },
+    {
+      "epoch": 0.6230662541342153,
+      "grad_norm": 0.08833298087120056,
+      "learning_rate": 9.73592958122166e-06,
+      "loss": 0.007,
+      "step": 2190
+    },
+    {
+      "epoch": 0.625911305522956,
+      "grad_norm": 0.07082614302635193,
+      "learning_rate": 9.780385880679292e-06,
+      "loss": 0.0065,
+      "step": 2200
+    },
+    {
+      "epoch": 0.6287563569116967,
+      "grad_norm": 0.0823511853814125,
+      "learning_rate": 9.824842180136926e-06,
+      "loss": 0.0071,
+      "step": 2210
+    },
+    {
+      "epoch": 0.6316014083004374,
+      "grad_norm": 0.06206200271844864,
+      "learning_rate": 9.86929847959456e-06,
+      "loss": 0.007,
+      "step": 2220
+    },
+    {
+      "epoch": 0.6344464596891781,
+      "grad_norm": 0.0808667540550232,
+      "learning_rate": 9.913754779052193e-06,
+      "loss": 0.0064,
+      "step": 2230
+    },
+    {
+      "epoch": 0.6372915110779188,
+      "grad_norm": 0.06442642211914062,
+      "learning_rate": 9.958211078509826e-06,
+      "loss": 0.0061,
+      "step": 2240
+    },
+    {
+      "epoch": 0.6401365624666595,
+      "grad_norm": 0.07416217774152756,
+      "learning_rate": 1.000266737796746e-05,
+      "loss": 0.0062,
+      "step": 2250
+    },
+    {
+      "epoch": 0.6401365624666595,
+      "eval_loss": 0.04642986133694649,
+      "eval_runtime": 9.625,
+      "eval_samples_per_second": 155.844,
+      "eval_steps_per_second": 9.766,
+      "eval_sts_dev_pearson_cosine": 0.7855623274467606,
+      "eval_sts_dev_pearson_dot": 0.5924008322873175,
+      "eval_sts_dev_pearson_euclidean": 0.7567766160311133,
+      "eval_sts_dev_pearson_manhattan": 0.7563225529205405,
+      "eval_sts_dev_pearson_max": 0.7855623274467606,
+      "eval_sts_dev_spearman_cosine": 0.7934736270810029,
+      "eval_sts_dev_spearman_dot": 0.5798196487719647,
+      "eval_sts_dev_spearman_euclidean": 0.7571185706646215,
+      "eval_sts_dev_spearman_manhattan": 0.7567665724312764,
+      "eval_sts_dev_spearman_max": 0.7934736270810029,
+      "step": 2250
+    },
+    {
+      "epoch": 0.6429816138554003,
+      "grad_norm": 0.09135396778583527,
+      "learning_rate": 1.0047123677425091e-05,
+      "loss": 0.0069,
+      "step": 2260
+    },
+    {
+      "epoch": 0.645826665244141,
+      "grad_norm": 0.08974805474281311,
+      "learning_rate": 1.0091579976882725e-05,
+      "loss": 0.0062,
+      "step": 2270
+    },
+    {
+      "epoch": 0.6486717166328817,
+      "grad_norm": 0.07147160917520523,
+      "learning_rate": 1.0136036276340358e-05,
+      "loss": 0.0063,
+      "step": 2280
+    },
+    {
+      "epoch": 0.6515167680216224,
+      "grad_norm": 0.0677226111292839,
+      "learning_rate": 1.0180492575797991e-05,
+      "loss": 0.0063,
+      "step": 2290
+    },
+    {
+      "epoch": 0.6543618194103631,
+      "grad_norm": 0.06699724495410919,
+      "learning_rate": 1.0224948875255625e-05,
+      "loss": 0.006,
+      "step": 2300
+    },
+    {
+      "epoch": 0.6572068707991038,
+      "grad_norm": 0.07739193737506866,
+      "learning_rate": 1.0269405174713258e-05,
+      "loss": 0.0064,
+      "step": 2310
+    },
+    {
+      "epoch": 0.6600519221878445,
+      "grad_norm": 0.04876289144158363,
+      "learning_rate": 1.0313861474170892e-05,
+      "loss": 0.0061,
+      "step": 2320
+    },
+    {
+      "epoch": 0.6628969735765853,
+      "grad_norm": 0.0824015662074089,
+      "learning_rate": 1.0358317773628523e-05,
+      "loss": 0.0065,
+      "step": 2330
+    },
+    {
+      "epoch": 0.665742024965326,
+      "grad_norm": 0.06627832353115082,
+      "learning_rate": 1.0402774073086157e-05,
+      "loss": 0.0061,
+      "step": 2340
+    },
+    {
+      "epoch": 0.6685870763540667,
+      "grad_norm": 0.0926406979560852,
+      "learning_rate": 1.044723037254379e-05,
+      "loss": 0.0067,
+      "step": 2350
+    },
+    {
+      "epoch": 0.6714321277428074,
+      "grad_norm": 0.06944271177053452,
+      "learning_rate": 1.0491686672001424e-05,
+      "loss": 0.0066,
+      "step": 2360
+    },
+    {
+      "epoch": 0.674277179131548,
+      "grad_norm": 0.07248842716217041,
+      "learning_rate": 1.0536142971459057e-05,
+      "loss": 0.0068,
+      "step": 2370
+    },
+    {
+      "epoch": 0.6771222305202887,
+      "grad_norm": 0.07560228556394577,
+      "learning_rate": 1.058059927091669e-05,
+      "loss": 0.0071,
+      "step": 2380
+    },
+    {
+      "epoch": 0.6799672819090294,
+      "grad_norm": 0.07752121984958649,
+      "learning_rate": 1.0625055570374324e-05,
+      "loss": 0.0064,
+      "step": 2390
+    },
+    {
+      "epoch": 0.6828123332977702,
+      "grad_norm": 0.091914102435112,
+      "learning_rate": 1.0669511869831956e-05,
+      "loss": 0.0064,
+      "step": 2400
+    },
+    {
+      "epoch": 0.6856573846865109,
+      "grad_norm": 0.08635730296373367,
+      "learning_rate": 1.0713968169289589e-05,
+      "loss": 0.0064,
+      "step": 2410
+    },
+    {
+      "epoch": 0.6885024360752516,
+      "grad_norm": 0.06619139015674591,
+      "learning_rate": 1.0758424468747222e-05,
+      "loss": 0.0064,
+      "step": 2420
+    },
+    {
+      "epoch": 0.6913474874639923,
+      "grad_norm": 0.08025142550468445,
+      "learning_rate": 1.0802880768204856e-05,
+      "loss": 0.0062,
+      "step": 2430
+    },
+    {
+      "epoch": 0.694192538852733,
+      "grad_norm": 0.07627130299806595,
+      "learning_rate": 1.084733706766249e-05,
+      "loss": 0.0067,
+      "step": 2440
+    },
+    {
+      "epoch": 0.6970375902414737,
+      "grad_norm": 0.0778108760714531,
+      "learning_rate": 1.0891793367120123e-05,
+      "loss": 0.0062,
+      "step": 2450
+    },
+    {
+      "epoch": 0.6998826416302144,
+      "grad_norm": 0.06801705807447433,
+      "learning_rate": 1.0936249666577754e-05,
+      "loss": 0.0059,
+      "step": 2460
+    },
+    {
+      "epoch": 0.7027276930189551,
+      "grad_norm": 0.06568338721990585,
+      "learning_rate": 1.0980705966035388e-05,
+      "loss": 0.0063,
+      "step": 2470
+    },
+    {
+      "epoch": 0.7055727444076959,
+      "grad_norm": 0.07497888058423996,
+      "learning_rate": 1.1025162265493021e-05,
+      "loss": 0.0055,
+      "step": 2480
+    },
+    {
+      "epoch": 0.7084177957964366,
+      "grad_norm": 0.0807732567191124,
+      "learning_rate": 1.1069618564950655e-05,
+      "loss": 0.0074,
+      "step": 2490
+    },
+    {
+      "epoch": 0.7112628471851773,
+      "grad_norm": 0.070890873670578,
+      "learning_rate": 1.1114074864408288e-05,
+      "loss": 0.0064,
+      "step": 2500
+    },
+    {
+      "epoch": 0.7112628471851773,
+      "eval_loss": 0.04881654307246208,
+      "eval_runtime": 9.0782,
+      "eval_samples_per_second": 165.231,
+      "eval_steps_per_second": 10.354,
+      "eval_sts_dev_pearson_cosine": 0.7833801585647451,
+      "eval_sts_dev_pearson_dot": 0.6099273388035109,
+      "eval_sts_dev_pearson_euclidean": 0.7555637908097597,
+      "eval_sts_dev_pearson_manhattan": 0.755336974018684,
+      "eval_sts_dev_pearson_max": 0.7833801585647451,
+      "eval_sts_dev_spearman_cosine": 0.7938690229216494,
+      "eval_sts_dev_spearman_dot": 0.5921480251455608,
+      "eval_sts_dev_spearman_euclidean": 0.7551889084831053,
+      "eval_sts_dev_spearman_manhattan": 0.7549767859265626,
+      "eval_sts_dev_spearman_max": 0.7938690229216494,
+      "step": 2500
+    },
+    {
+      "epoch": 0.714107898573918,
+      "grad_norm": 0.06863918155431747,
+      "learning_rate": 1.1158531163865921e-05,
+      "loss": 0.006,
+      "step": 2510
+    },
+    {
+      "epoch": 0.7169529499626587,
+      "grad_norm": 0.07372977584600449,
+      "learning_rate": 1.1202987463323555e-05,
+      "loss": 0.0061,
+      "step": 2520
+    },
+    {
+      "epoch": 0.7197980013513994,
+      "grad_norm": 0.06173211336135864,
+      "learning_rate": 1.1247443762781187e-05,
+      "loss": 0.0064,
+      "step": 2530
+    },
+    {
+      "epoch": 0.7226430527401401,
+      "grad_norm": 0.0746549442410469,
+      "learning_rate": 1.129190006223882e-05,
+      "loss": 0.0059,
+      "step": 2540
+    },
+    {
+      "epoch": 0.7254881041288809,
+      "grad_norm": 0.07160431891679764,
+      "learning_rate": 1.1336356361696453e-05,
+      "loss": 0.0064,
+      "step": 2550
+    },
+    {
+      "epoch": 0.7283331555176216,
+      "grad_norm": 0.07895245403051376,
+      "learning_rate": 1.1380812661154087e-05,
+      "loss": 0.0061,
+      "step": 2560
+    },
+    {
+      "epoch": 0.7311782069063623,
+      "grad_norm": 0.06770409643650055,
+      "learning_rate": 1.142526896061172e-05,
+      "loss": 0.0062,
+      "step": 2570
+    },
+    {
+      "epoch": 0.734023258295103,
+      "grad_norm": 0.07894620299339294,
+      "learning_rate": 1.1469725260069354e-05,
+      "loss": 0.0068,
+      "step": 2580
+    },
+    {
+      "epoch": 0.7368683096838436,
+      "grad_norm": 0.07674427330493927,
+      "learning_rate": 1.1514181559526985e-05,
+      "loss": 0.0061,
+      "step": 2590
+    },
+    {
+      "epoch": 0.7397133610725843,
+      "grad_norm": 0.09392740577459335,
+      "learning_rate": 1.1558637858984619e-05,
+      "loss": 0.0065,
+      "step": 2600
+    },
+    {
+      "epoch": 0.742558412461325,
+      "grad_norm": 0.06918327510356903,
+      "learning_rate": 1.1603094158442252e-05,
+      "loss": 0.0055,
+      "step": 2610
+    },
+    {
+      "epoch": 0.7454034638500658,
+      "grad_norm": 0.0910848081111908,
+      "learning_rate": 1.1647550457899886e-05,
+      "loss": 0.0057,
+      "step": 2620
+    },
+    {
+      "epoch": 0.7482485152388065,
+      "grad_norm": 0.06995029747486115,
+      "learning_rate": 1.1692006757357519e-05,
+      "loss": 0.0064,
+      "step": 2630
+    },
+    {
+      "epoch": 0.7510935666275472,
+      "grad_norm": 0.06808440387248993,
+      "learning_rate": 1.1736463056815152e-05,
+      "loss": 0.0056,
+      "step": 2640
+    },
+    {
+      "epoch": 0.7539386180162879,
+      "grad_norm": 0.06680841743946075,
+      "learning_rate": 1.1780919356272784e-05,
+      "loss": 0.0059,
+      "step": 2650
+    },
+    {
+      "epoch": 0.7567836694050286,
+      "grad_norm": 0.07254023104906082,
+      "learning_rate": 1.1825375655730417e-05,
+      "loss": 0.0059,
+      "step": 2660
+    },
+    {
+      "epoch": 0.7596287207937693,
+      "grad_norm": 0.06777305901050568,
+      "learning_rate": 1.1869831955188051e-05,
+      "loss": 0.0064,
+      "step": 2670
+    },
+    {
+      "epoch": 0.76247377218251,
+      "grad_norm": 0.07384537160396576,
+      "learning_rate": 1.1914288254645684e-05,
+      "loss": 0.0067,
+      "step": 2680
+    },
+    {
+      "epoch": 0.7653188235712508,
+      "grad_norm": 0.07408931851387024,
+      "learning_rate": 1.1958744554103318e-05,
+      "loss": 0.0062,
+      "step": 2690
+    },
+    {
+      "epoch": 0.7681638749599915,
+      "grad_norm": 0.0712476596236229,
+      "learning_rate": 1.200320085356095e-05,
+      "loss": 0.0056,
+      "step": 2700
+    },
+    {
+      "epoch": 0.7710089263487322,
+      "grad_norm": 0.07831274718046188,
+      "learning_rate": 1.2047657153018583e-05,
+      "loss": 0.0063,
+      "step": 2710
+    },
+    {
+      "epoch": 0.7738539777374729,
+      "grad_norm": 0.08914618194103241,
+      "learning_rate": 1.2092113452476216e-05,
+      "loss": 0.0064,
+      "step": 2720
+    },
+    {
+      "epoch": 0.7766990291262136,
+      "grad_norm": 0.05413926765322685,
+      "learning_rate": 1.213656975193385e-05,
+      "loss": 0.0063,
+      "step": 2730
+    },
+    {
+      "epoch": 0.7795440805149543,
+      "grad_norm": 0.07458141446113586,
+      "learning_rate": 1.2181026051391483e-05,
+      "loss": 0.0062,
+      "step": 2740
+    },
+    {
+      "epoch": 0.782389131903695,
+      "grad_norm": 0.06790082156658173,
+      "learning_rate": 1.2225482350849116e-05,
+      "loss": 0.0058,
+      "step": 2750
+    },
+    {
+      "epoch": 0.782389131903695,
+      "eval_loss": 0.047921594232320786,
+      "eval_runtime": 9.1598,
+      "eval_samples_per_second": 163.759,
+      "eval_steps_per_second": 10.262,
+      "eval_sts_dev_pearson_cosine": 0.7890077878318924,
+      "eval_sts_dev_pearson_dot": 0.6145569918459842,
+      "eval_sts_dev_pearson_euclidean": 0.7622940567617165,
+      "eval_sts_dev_pearson_manhattan": 0.7619872460429895,
+      "eval_sts_dev_pearson_max": 0.7890077878318924,
+      "eval_sts_dev_spearman_cosine": 0.7986960426836712,
+      "eval_sts_dev_spearman_dot": 0.5995859874295417,
+      "eval_sts_dev_spearman_euclidean": 0.7620718720832805,
+      "eval_sts_dev_spearman_manhattan": 0.7619175969681303,
+      "eval_sts_dev_spearman_max": 0.7986960426836712,
+      "step": 2750
+    },
+    {
+      "epoch": 0.7852341832924358,
+      "grad_norm": 0.07279020547866821,
+      "learning_rate": 1.2269938650306748e-05,
+      "loss": 0.0063,
+      "step": 2760
+    },
+    {
+      "epoch": 0.7880792346811765,
+      "grad_norm": 0.07638818025588989,
+      "learning_rate": 1.2314394949764382e-05,
+      "loss": 0.0061,
+      "step": 2770
+    },
+    {
+      "epoch": 0.7909242860699172,
+      "grad_norm": 0.07741549611091614,
+      "learning_rate": 1.2358851249222015e-05,
+      "loss": 0.0059,
+      "step": 2780
+    },
+    {
+      "epoch": 0.7937693374586579,
+      "grad_norm": 0.07450976967811584,
+      "learning_rate": 1.2403307548679648e-05,
+      "loss": 0.0061,
+      "step": 2790
+    },
+    {
+      "epoch": 0.7966143888473985,
+      "grad_norm": 0.07069046795368195,
+      "learning_rate": 1.2447763848137282e-05,
+      "loss": 0.0059,
+      "step": 2800
+    },
+    {
+      "epoch": 0.7994594402361392,
+      "grad_norm": 0.07043927907943726,
+      "learning_rate": 1.2492220147594915e-05,
+      "loss": 0.0058,
+      "step": 2810
+    },
+    {
+      "epoch": 0.8023044916248799,
+      "grad_norm": 0.06363347172737122,
+      "learning_rate": 1.2536676447052547e-05,
+      "loss": 0.0057,
+      "step": 2820
+    },
+    {
+      "epoch": 0.8051495430136207,
+      "grad_norm": 0.0633586049079895,
+      "learning_rate": 1.2581132746510182e-05,
+      "loss": 0.0059,
+      "step": 2830
+    },
+    {
+      "epoch": 0.8079945944023614,
+      "grad_norm": 0.0656353160738945,
+      "learning_rate": 1.2625589045967814e-05,
+      "loss": 0.0058,
+      "step": 2840
+    },
+    {
+      "epoch": 0.8108396457911021,
+      "grad_norm": 0.05306802690029144,
+      "learning_rate": 1.2670045345425447e-05,
+      "loss": 0.0068,
+      "step": 2850
+    },
+    {
+      "epoch": 0.8136846971798428,
+      "grad_norm": 0.06369970738887787,
+      "learning_rate": 1.271450164488308e-05,
+      "loss": 0.006,
+      "step": 2860
+    },
+    {
+      "epoch": 0.8165297485685835,
+      "grad_norm": 0.059002261608839035,
+      "learning_rate": 1.2758957944340712e-05,
+      "loss": 0.0058,
+      "step": 2870
+    },
+    {
+      "epoch": 0.8193747999573242,
+      "grad_norm": 0.06396970897912979,
+      "learning_rate": 1.2803414243798347e-05,
+      "loss": 0.0061,
+      "step": 2880
+    },
+    {
+      "epoch": 0.8222198513460649,
+      "grad_norm": 0.06522241234779358,
+      "learning_rate": 1.2847870543255979e-05,
+      "loss": 0.0058,
+      "step": 2890
+    },
+    {
+      "epoch": 0.8250649027348056,
+      "grad_norm": 0.07420309633016586,
+      "learning_rate": 1.2892326842713613e-05,
+      "loss": 0.0055,
+      "step": 2900
+    },
+    {
+      "epoch": 0.8279099541235464,
+      "grad_norm": 0.0638047531247139,
+      "learning_rate": 1.2936783142171246e-05,
+      "loss": 0.006,
+      "step": 2910
+    },
+    {
+      "epoch": 0.8307550055122871,
+      "grad_norm": 0.06835135072469711,
+      "learning_rate": 1.298123944162888e-05,
+      "loss": 0.0063,
+      "step": 2920
+    },
+    {
+      "epoch": 0.8336000569010278,
+      "grad_norm": 0.07669410854578018,
+      "learning_rate": 1.3025695741086513e-05,
+      "loss": 0.0066,
+      "step": 2930
+    },
+    {
+      "epoch": 0.8364451082897685,
+      "grad_norm": 0.0799371749162674,
+      "learning_rate": 1.3070152040544144e-05,
+      "loss": 0.0059,
+      "step": 2940
+    },
+    {
+      "epoch": 0.8392901596785092,
+      "grad_norm": 0.06452161073684692,
+      "learning_rate": 1.311460834000178e-05,
+      "loss": 0.0056,
+      "step": 2950
+    },
+    {
+      "epoch": 0.8421352110672499,
+      "grad_norm": 0.08192815631628036,
+      "learning_rate": 1.3159064639459411e-05,
+      "loss": 0.006,
+      "step": 2960
+    },
+    {
+      "epoch": 0.8449802624559906,
+      "grad_norm": 0.08512122929096222,
+      "learning_rate": 1.3203520938917045e-05,
+      "loss": 0.0058,
+      "step": 2970
+    },
+    {
+      "epoch": 0.8478253138447314,
+      "grad_norm": 0.05736185237765312,
+      "learning_rate": 1.3247977238374678e-05,
+      "loss": 0.006,
+      "step": 2980
+    },
+    {
+      "epoch": 0.8506703652334721,
+      "grad_norm": 0.056353483349084854,
+      "learning_rate": 1.329243353783231e-05,
+      "loss": 0.0056,
+      "step": 2990
+    },
+    {
+      "epoch": 0.8535154166222128,
+      "grad_norm": 0.07176116853952408,
+      "learning_rate": 1.3336889837289945e-05,
+      "loss": 0.0062,
+      "step": 3000
+    },
+    {
+      "epoch": 0.8535154166222128,
+      "eval_loss": 0.05112998187541962,
+      "eval_runtime": 8.8822,
+      "eval_samples_per_second": 168.877,
+      "eval_steps_per_second": 10.583,
+      "eval_sts_dev_pearson_cosine": 0.7899474758938174,
+      "eval_sts_dev_pearson_dot": 0.601477379904751,
+      "eval_sts_dev_pearson_euclidean": 0.7631393093085868,
+      "eval_sts_dev_pearson_manhattan": 0.762660331965514,
+      "eval_sts_dev_pearson_max": 0.7899474758938174,
+      "eval_sts_dev_spearman_cosine": 0.7996334507041041,
+      "eval_sts_dev_spearman_dot": 0.5888332407477099,
+      "eval_sts_dev_spearman_euclidean": 0.7632786368314162,
+      "eval_sts_dev_spearman_manhattan": 0.7624223584443209,
+      "eval_sts_dev_spearman_max": 0.7996334507041041,
+      "step": 3000
+    },
+    {
+      "epoch": 0.8563604680109534,
+      "grad_norm": 0.07095087319612503,
+      "learning_rate": 1.3381346136747577e-05,
+      "loss": 0.0059,
+      "step": 3010
+    },
+    {
+      "epoch": 0.8592055193996941,
+      "grad_norm": 0.06647340953350067,
+      "learning_rate": 1.3425802436205212e-05,
+      "loss": 0.0064,
+      "step": 3020
+    },
+    {
+      "epoch": 0.8620505707884348,
+      "grad_norm": 0.069040946662426,
+      "learning_rate": 1.3470258735662843e-05,
+      "loss": 0.0064,
+      "step": 3030
+    },
+    {
+      "epoch": 0.8648956221771755,
+      "grad_norm": 0.05208707973361015,
+      "learning_rate": 1.3514715035120475e-05,
+      "loss": 0.006,
+      "step": 3040
+    },
+    {
+      "epoch": 0.8677406735659163,
+      "grad_norm": 0.058777816593647,
+      "learning_rate": 1.355917133457811e-05,
+      "loss": 0.0059,
+      "step": 3050
+    },
+    {
+      "epoch": 0.870585724954657,
+      "grad_norm": 0.0731450691819191,
+      "learning_rate": 1.3603627634035742e-05,
+      "loss": 0.0055,
+      "step": 3060
+    },
+    {
+      "epoch": 0.8734307763433977,
+      "grad_norm": 0.08497700840234756,
+      "learning_rate": 1.3648083933493377e-05,
+      "loss": 0.0056,
+      "step": 3070
+    },
+    {
+      "epoch": 0.8762758277321384,
+      "grad_norm": 0.06076115369796753,
+      "learning_rate": 1.3692540232951009e-05,
+      "loss": 0.0058,
+      "step": 3080
+    },
+    {
+      "epoch": 0.8791208791208791,
+      "grad_norm": 0.05804910510778427,
+      "learning_rate": 1.3736996532408644e-05,
+      "loss": 0.0057,
+      "step": 3090
+    },
+    {
+      "epoch": 0.8819659305096198,
+      "grad_norm": 0.06687209010124207,
+      "learning_rate": 1.3781452831866276e-05,
+      "loss": 0.0058,
+      "step": 3100
+    },
+    {
+      "epoch": 0.8848109818983605,
+      "grad_norm": 0.06274023652076721,
+      "learning_rate": 1.3825909131323907e-05,
+      "loss": 0.0062,
+      "step": 3110
+    },
+    {
+      "epoch": 0.8876560332871013,
+      "grad_norm": 0.06041685864329338,
+      "learning_rate": 1.3870365430781542e-05,
+      "loss": 0.0058,
+      "step": 3120
+    },
+    {
+      "epoch": 0.890501084675842,
+      "grad_norm": 0.07634738087654114,
+      "learning_rate": 1.3914821730239174e-05,
+      "loss": 0.0058,
+      "step": 3130
+    },
+    {
+      "epoch": 0.8933461360645827,
+      "grad_norm": 0.05441463738679886,
+      "learning_rate": 1.395927802969681e-05,
+      "loss": 0.0055,
+      "step": 3140
+    },
+    {
+      "epoch": 0.8961911874533234,
+      "grad_norm": 0.08034619688987732,
+      "learning_rate": 1.4003734329154441e-05,
+      "loss": 0.0056,
+      "step": 3150
+    },
+    {
+      "epoch": 0.8990362388420641,
+      "grad_norm": 0.05668621510267258,
+      "learning_rate": 1.4048190628612076e-05,
+      "loss": 0.0055,
+      "step": 3160
+    },
+    {
+      "epoch": 0.9018812902308048,
+      "grad_norm": 0.0647374764084816,
+      "learning_rate": 1.4092646928069708e-05,
+      "loss": 0.0054,
+      "step": 3170
+    },
+    {
+      "epoch": 0.9047263416195455,
+      "grad_norm": 0.06342489272356033,
+      "learning_rate": 1.413710322752734e-05,
+      "loss": 0.0059,
+      "step": 3180
+    },
+    {
+      "epoch": 0.9075713930082863,
+      "grad_norm": 0.05244365334510803,
+      "learning_rate": 1.4181559526984975e-05,
+      "loss": 0.0056,
+      "step": 3190
+    },
+    {
+      "epoch": 0.910416444397027,
+      "grad_norm": 0.06412120163440704,
+      "learning_rate": 1.4226015826442606e-05,
+      "loss": 0.0057,
+      "step": 3200
+    },
+    {
+      "epoch": 0.9132614957857677,
+      "grad_norm": 0.06194351613521576,
+      "learning_rate": 1.4270472125900241e-05,
+      "loss": 0.0055,
+      "step": 3210
+    },
+    {
+      "epoch": 0.9161065471745083,
+      "grad_norm": 0.06416637450456619,
+      "learning_rate": 1.4314928425357873e-05,
+      "loss": 0.0061,
+      "step": 3220
+    },
+    {
+      "epoch": 0.918951598563249,
+      "grad_norm": 0.05956665053963661,
+      "learning_rate": 1.4359384724815508e-05,
+      "loss": 0.0055,
+      "step": 3230
+    },
+    {
+      "epoch": 0.9217966499519897,
+      "grad_norm": 0.055366478860378265,
+      "learning_rate": 1.440384102427314e-05,
+      "loss": 0.0062,
+      "step": 3240
+    },
+    {
+      "epoch": 0.9246417013407304,
+      "grad_norm": 0.065581776201725,
+      "learning_rate": 1.4448297323730772e-05,
+      "loss": 0.006,
+      "step": 3250
+    },
+    {
+      "epoch": 0.9246417013407304,
+      "eval_loss": 0.050821732729673386,
+      "eval_runtime": 9.0965,
+      "eval_samples_per_second": 164.898,
+      "eval_steps_per_second": 10.334,
+      "eval_sts_dev_pearson_cosine": 0.7909511151327651,
+      "eval_sts_dev_pearson_dot": 0.6053394742010075,
+      "eval_sts_dev_pearson_euclidean": 0.7648214403760886,
+      "eval_sts_dev_pearson_manhattan": 0.7646742286120747,
+      "eval_sts_dev_pearson_max": 0.7909511151327651,
+      "eval_sts_dev_spearman_cosine": 0.7988801832352788,
+      "eval_sts_dev_spearman_dot": 0.5877204065924448,
+      "eval_sts_dev_spearman_euclidean": 0.7657836560350672,
+      "eval_sts_dev_spearman_manhattan": 0.7656908568548414,
+      "eval_sts_dev_spearman_max": 0.7988801832352788,
+      "step": 3250
+    },
+    {
+      "epoch": 0.9274867527294712,
+      "grad_norm": 0.07194357365369797,
+      "learning_rate": 1.4492753623188407e-05,
+      "loss": 0.0058,
+      "step": 3260
+    },
+    {
+      "epoch": 0.9303318041182119,
+      "grad_norm": 0.07017621397972107,
+      "learning_rate": 1.4537209922646039e-05,
+      "loss": 0.0053,
+      "step": 3270
+    },
+    {
+      "epoch": 0.9331768555069526,
+      "grad_norm": 0.0723465159535408,
+      "learning_rate": 1.4581666222103674e-05,
+      "loss": 0.0064,
+      "step": 3280
+    },
+    {
+      "epoch": 0.9360219068956933,
+      "grad_norm": 0.07010278105735779,
+      "learning_rate": 1.4626122521561305e-05,
+      "loss": 0.006,
+      "step": 3290
+    },
+    {
+      "epoch": 0.938866958284434,
+      "grad_norm": 0.06073617562651634,
+      "learning_rate": 1.467057882101894e-05,
+      "loss": 0.0057,
+      "step": 3300
+    },
+    {
+      "epoch": 0.9417120096731747,
+      "grad_norm": 0.05822043493390083,
+      "learning_rate": 1.4715035120476572e-05,
+      "loss": 0.0059,
+      "step": 3310
+    },
+    {
+      "epoch": 0.9445570610619154,
+      "grad_norm": 0.05081350356340408,
+      "learning_rate": 1.4759491419934204e-05,
+      "loss": 0.0057,
+      "step": 3320
+    },
+    {
+      "epoch": 0.9474021124506561,
+      "grad_norm": 0.0619865357875824,
+      "learning_rate": 1.4803947719391839e-05,
+      "loss": 0.0056,
+      "step": 3330
+    },
+    {
+      "epoch": 0.9502471638393969,
+      "grad_norm": 0.06353365629911423,
+      "learning_rate": 1.484840401884947e-05,
+      "loss": 0.0056,
+      "step": 3340
+    },
+    {
+      "epoch": 0.9530922152281376,
+      "grad_norm": 0.07539238780736923,
+      "learning_rate": 1.4892860318307106e-05,
+      "loss": 0.0061,
+      "step": 3350
+    },
+    {
+      "epoch": 0.9559372666168783,
+      "grad_norm": 0.06994681805372238,
+      "learning_rate": 1.4937316617764737e-05,
+      "loss": 0.0053,
+      "step": 3360
+    },
+    {
+      "epoch": 0.958782318005619,
+      "grad_norm": 0.05879216268658638,
+      "learning_rate": 1.498177291722237e-05,
+      "loss": 0.0056,
+      "step": 3370
+    },
+    {
+      "epoch": 0.9616273693943597,
+      "grad_norm": 0.0628664419054985,
+      "learning_rate": 1.5026229216680004e-05,
+      "loss": 0.006,
+      "step": 3380
+    },
+    {
+      "epoch": 0.9644724207831004,
+      "grad_norm": 0.07300525158643723,
+      "learning_rate": 1.5070685516137636e-05,
+      "loss": 0.0066,
+      "step": 3390
+    },
+    {
+      "epoch": 0.967317472171841,
+      "grad_norm": 0.09258022159337997,
+      "learning_rate": 1.5115141815595271e-05,
+      "loss": 0.0062,
+      "step": 3400
+    },
+    {
+      "epoch": 0.9701625235605819,
+      "grad_norm": 0.05411943048238754,
+      "learning_rate": 1.5159598115052903e-05,
+      "loss": 0.0053,
+      "step": 3410
+    },
+    {
+      "epoch": 0.9730075749493226,
+      "grad_norm": 0.06785852462053299,
+      "learning_rate": 1.5204054414510538e-05,
+      "loss": 0.0062,
+      "step": 3420
+    },
+    {
+      "epoch": 0.9758526263380632,
+      "grad_norm": 0.04845158010721207,
+      "learning_rate": 1.524851071396817e-05,
+      "loss": 0.0057,
+      "step": 3430
+    },
+    {
+      "epoch": 0.9786976777268039,
+      "grad_norm": 0.07520108669996262,
+      "learning_rate": 1.52929670134258e-05,
+      "loss": 0.0059,
+      "step": 3440
+    },
+    {
+      "epoch": 0.9815427291155446,
+      "grad_norm": 0.0539853572845459,
+      "learning_rate": 1.5337423312883436e-05,
+      "loss": 0.0061,
+      "step": 3450
+    },
+    {
+      "epoch": 0.9843877805042853,
+      "grad_norm": 0.0653446614742279,
+      "learning_rate": 1.5381879612341068e-05,
+      "loss": 0.0057,
+      "step": 3460
+    },
+    {
+      "epoch": 0.987232831893026,
+      "grad_norm": 0.0627661645412445,
+      "learning_rate": 1.5426335911798703e-05,
+      "loss": 0.0054,
+      "step": 3470
+    },
+    {
+      "epoch": 0.9900778832817668,
+      "grad_norm": 0.05211208760738373,
+      "learning_rate": 1.5470792211256335e-05,
+      "loss": 0.0054,
+      "step": 3480
+    },
+    {
+      "epoch": 0.9929229346705075,
+      "grad_norm": 0.06061340123414993,
+      "learning_rate": 1.551524851071397e-05,
+      "loss": 0.0057,
+      "step": 3490
+    },
+    {
+      "epoch": 0.9957679860592482,
+      "grad_norm": 0.06133125722408295,
+      "learning_rate": 1.5559704810171602e-05,
+      "loss": 0.0056,
+      "step": 3500
+    },
+    {
+      "epoch": 0.9957679860592482,
+      "eval_loss": 0.04846322536468506,
+      "eval_runtime": 9.4808,
+      "eval_samples_per_second": 158.214,
+      "eval_steps_per_second": 9.915,
+      "eval_sts_dev_pearson_cosine": 0.7873863893875668,
+      "eval_sts_dev_pearson_dot": 0.590451098760234,
+      "eval_sts_dev_pearson_euclidean": 0.7578432355248655,
+      "eval_sts_dev_pearson_manhattan": 0.7576106133017201,
+      "eval_sts_dev_pearson_max": 0.7873863893875668,
+      "eval_sts_dev_spearman_cosine": 0.7957546813665759,
+      "eval_sts_dev_spearman_dot": 0.5781029293331612,
+      "eval_sts_dev_spearman_euclidean": 0.7559256690552184,
+      "eval_sts_dev_spearman_manhattan": 0.7557621348675576,
+      "eval_sts_dev_spearman_max": 0.7957546813665759,
+      "step": 3500
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 7028,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 3500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 16,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8602f8d64d4fcda8a8f2a56cff94a6dbda3bf7003e1e1f28d9fe03e20fd49727
+size 5688