Training in progress, step 266, checkpoint

checkpoint-266/1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

checkpoint-266/README.md

@@ -0,0 +1,795 @@
+---
+base_model: bobox/DeBERTa-small-ST-v1-test-step3
+datasets: []
+language: []
+library_name: sentence-transformers
+metrics:
+- pearson_cosine
+- spearman_cosine
+- pearson_manhattan
+- spearman_manhattan
+- pearson_euclidean
+- spearman_euclidean
+- pearson_dot
+- spearman_dot
+- pearson_max
+- spearman_max
+pipeline_tag: sentence-similarity
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:120849
+- loss:CachedGISTEmbedLoss
+widget:
+- source_sentence: 'Brian Cummins, who was in his early 60s, was refereeing an Under-16s
+    final in Elburton, Devon, on Sunday when he fell to the ground.
+
+    He was taken to Derriford Hospital in Plymouth but died shortly after, the Devon
+    Junior Minor League said.
+
+    A spokesman said: "Our thoughts and condolences at this time are with his family.
+    The league has lost a very loyal referee."
+
+    The game between Woolwell Juniors and Tavistock was abandoned after Mr Cummins
+    collapsed.
+
+    His daughter Sarah said: "He was a loving father, father in law, granddad, husband
+    and friend to all who knew him. He loved his football and refereeing.
+
+    "He will be greatly missed by all and will forever be in our hearts."
+
+    Brian Rimes, general secretary of the league, said former Devonport dockyard worker
+    Mr Cummins had been a referee for the league for about 20 years.
+
+    "He was a very good referee, a man well respected by youngsters and the referee
+    fraternity," he said.
+
+    Mark Davies wrote on the league''s Facebook page: "Very sad news indeed and our
+    thoughts go out to Brian''s family and friends. It''s a shame that it takes such
+    awful circumstances to unite the local footballing community but in Brian we know
+    we have lost a true gent. RIP Brian."
+
+    Mark Evans wrote: "RIP Brian. Grassroots football has lost an amazing guy and
+    great referee. Deepest sympathies from all Devon FA referees."
+
+    Michael Davies tweeted: "#RIP Brian Cummins, such sad news! Top Ref, top neighbour
+    but most of all a top bloke! Will be sadly missed."'
+  sentences:
+  - 'Ptosis (Sagging Eyelids): Check Your Symptoms and Signs Watery Eye A drooping
+    or sagging of the eyelid is medically known as ptosis or blepharoptosis . Drooping
+    eyelids may occur on both sides (bilateral) or on one side only (unilateral),
+    in which case it is more easily noticed. Congenital ptosis is eyelid drooping
+    that is present at birth; when it develops later, it is referred to as acquired
+    ptosis. Depending upon the severity of the condition, drooping eyelids may be
+    barely noticeable or quite prominent. Some sagging of the skin and connective
+    tissues occurs during the normal aging process, potentially leading to drooping
+    of the eyelids. Other causes include conditions that affect the muscles and nerves
+    of the eyelid as well as conditions that affect the skin and connective tissues
+    of the eyelid. Rarely, tumors of the brain or eye area are the cause of drooping
+    eyelids. Medically Reviewed by a Doctor on 3/6/2012 Health concern on your mind?
+    Visit the Symptom Checker. REFERENCE: Fauci, Anthony S., et al. Harrison''s Principles
+    of Internal Medicine. 17th ed. United States: McGraw-Hill Professional, 2008.
+    Causes of Ptosis Allergy (Allergies) An allergy refers to a misguided reaction
+    by our immune system in response to bodily contact with certain foreign substances.
+    ... learn more » Botulism Botulism is an illness caused by a neurotoxin produced
+    by the bacterium Clostridium botulinum. There are three types of botulism:...
+    learn more » In This Article'
+  - A football referee has died after collapsing during a boys' cup final.
+  - The car is at the intersection while the sun is setting.
+- source_sentence: sparge water temperature
+  sentences:
+  - This is easy to translate to gallons and degrees F. (for example, suggested sparge
+    water temperature is 167° F., which is 75° C.). It also features a stay warm
+    feature - after the target water temperature is hit, it will keep it at the desired
+    temperature as long as it is on.
+  - Arsenal playmaker Mesut Ozil says he will put talks over his future at the club
+    on hold until the summer.
+  - a greenhouse is used to protect plants by keeping them warm
+- source_sentence: What does sunlight create for plants?
+  sentences:
+  - "a plant requires sunlight for photosynthesis. Photosynthesis occurs using the\
+    \ suns energy to create the plants own energy. \n sunlight creates energy for\
+    \ plants"
+  - His references in electronic music are Todd Terry , Armand Van Helden , Roger
+    Sanchez , Tiesto and the Epic Sax Guy.
+  - "if a neutral atom loses an electron then an atom with a negative charge will\
+    \ be formed. Ions are neutral atoms. \n ions can have a negative charge if they\
+    \ lose an electron"
+- source_sentence: Metals, metalloids, and nonmetals are the different classes of
+    what?
+  sentences:
+  - when an animal sheds its fur , its fur becomes less dense
+  - Though there's no limit to how much you can keep in a savings account, you should
+    know the rules surrounding large deposits to savings accounts. When it comes to
+    making deposits to a bank account, $10,000 is the magic number.
+  - The classes of elements are metals, metalloids, and nonmetals. They are color-coded
+    in the table. Blue stands for metals, orange for metalloids, and green for nonmetals.
+    You can read about each of these three classes of elements later in the chapter,
+    in the lesson "Classes of Elements. ".
+- source_sentence: More than 336,000 COVID-19 cases have been reported in over 190
+    countries .
+  sentences:
+  - Birds are four-limbed, endothermic vertebrates with wings and feathers. They produce
+    amniotic eggs and are the most numerous class of vertebrates.
+  - As of 23 March , more than 337,000 cases of COVID-19 have been reported in over
+    190 countries and territories , resulting in more than 14,600 deaths and 97,000
+    recoveries .
+  - Apocalypticism Apocalypticism is the religious belief that there will be an apocalypse,
+    a term which originally referred to a revelation of God's will, but now usually
+    refers to the belief that the end of the world is imminent, even within one's
+    own lifetime. This belief is usually accompanied by the idea that civilization
+    will soon come to a tumultuous end due to some sort of catastrophic global event.
+model-index:
+- name: SentenceTransformer based on bobox/DeBERTa-small-ST-v1-test-step3
+  results:
+  - task:
+      type: semantic-similarity
+      name: Semantic Similarity
+    dataset:
+      name: sts test
+      type: sts-test
+    metrics:
+    - type: pearson_cosine
+      value: 0.8842448913937776
+      name: Pearson Cosine
+    - type: spearman_cosine
+      value: 0.9093488983153196
+      name: Spearman Cosine
+    - type: pearson_manhattan
+      value: 0.9125853937525384
+      name: Pearson Manhattan
+    - type: spearman_manhattan
+      value: 0.9085058740532286
+      name: Spearman Manhattan
+    - type: pearson_euclidean
+      value: 0.9135135876297823
+      name: Pearson Euclidean
+    - type: spearman_euclidean
+      value: 0.9094515129576901
+      name: Spearman Euclidean
+    - type: pearson_dot
+      value: 0.8618584457426092
+      name: Pearson Dot
+    - type: spearman_dot
+      value: 0.8592465943455663
+      name: Spearman Dot
+    - type: pearson_max
+      value: 0.9135135876297823
+      name: Pearson Max
+    - type: spearman_max
+      value: 0.9094515129576901
+      name: Spearman Max
+---
+
+# SentenceTransformer based on bobox/DeBERTa-small-ST-v1-test-step3
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [bobox/DeBERTa-small-ST-v1-test-step3](https://huggingface.co/bobox/DeBERTa-small-ST-v1-test-step3) on the bobox/enhanced_nli-50_k dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [bobox/DeBERTa-small-ST-v1-test-step3](https://huggingface.co/bobox/DeBERTa-small-ST-v1-test-step3) <!-- at revision df9aaa75fe0c2791e5ed35ff33de1689d9a5f5ff -->
+- **Maximum Sequence Length:** 512 tokens
+- **Output Dimensionality:** 768 tokens
+- **Similarity Function:** Cosine Similarity
+- **Training Dataset:**
+    - bobox/enhanced_nli-50_k
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: DebertaV2Model 
+  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("bobox/DeBERTa-small-ST-v1-test-UnifiedDatasets-bis-checkpoints-tmp")
+# Run inference
+sentences = [
+    'More than 336,000 COVID-19 cases have been reported in over 190 countries .',
+    'As of 23 March , more than 337,000 cases of COVID-19 have been reported in over 190 countries and territories , resulting in more than 14,600 deaths and 97,000 recoveries .',
+    "Apocalypticism Apocalypticism is the religious belief that there will be an apocalypse, a term which originally referred to a revelation of God's will, but now usually refers to the belief that the end of the world is imminent, even within one's own lifetime. This belief is usually accompanied by the idea that civilization will soon come to a tumultuous end due to some sort of catastrophic global event.",
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 768]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Semantic Similarity
+* Dataset: `sts-test`
+* Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
+
+| Metric              | Value      |
+|:--------------------|:-----------|
+| pearson_cosine      | 0.8842     |
+| **spearman_cosine** | **0.9093** |
+| pearson_manhattan   | 0.9126     |
+| spearman_manhattan  | 0.9085     |
+| pearson_euclidean   | 0.9135     |
+| spearman_euclidean  | 0.9095     |
+| pearson_dot         | 0.8619     |
+| spearman_dot        | 0.8592     |
+| pearson_max         | 0.9135     |
+| spearman_max        | 0.9095     |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### bobox/enhanced_nli-50_k
+
+* Dataset: bobox/enhanced_nli-50_k
+* Size: 120,849 training samples
+* Columns: <code>sentence1</code> and <code>sentence2</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | sentence1                                                                          | sentence2                                                                          |
+  |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
+  | type    | string                                                                             | string                                                                             |
+  | details | <ul><li>min: 4 tokens</li><li>mean: 33.98 tokens</li><li>max: 358 tokens</li></ul> | <ul><li>min: 2 tokens</li><li>mean: 63.13 tokens</li><li>max: 414 tokens</li></ul> |
+* Samples:
+  | sentence1                                                                        | sentence2                                                                                                                                                                                                                                                                                                                                                                                                               |
+  |:---------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>A lady working in a kitchen with several different types of dishes.</code> | <code>A woman is cooking and cleaning in her kitchen.</code>                                                                                                                                                                                                                                                                                                                                                            |
+  | <code>is it possible to get pregnant after delivery?</code>                      | <code>How soon can you get pregnant after giving birth? It's possible to get pregnant before you even have your first postpartum period, which can occur as early as four weeks after giving birth or as late as 24 weeks after baby arrives (or later), depending on whether you're breastfeeding exclusively or not.</code>                                                                                           |
+  | <code>how long does corn take to grill in foil</code>                            | <code>Place each corn on top of one piece the heavy-duty foil. Brush each ear generously with soft butter. Season lightly with seasoned salt or white salt and black pepper. Wrap the corn then seal the foil loosely but leave room for expansion, then cut a very small hole in the foil to allow steam to escape. Grill over medium coals for about 15-20 minutes (the larger ears may take a little longer).</code> |
+* Loss: [<code>CachedGISTEmbedLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedgistembedloss) with these parameters:
+  ```json
+  {'guide': SentenceTransformer(
+    (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
+    (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+    (2): Normalize()
+  ), 'temperature': 0.025}
+  ```
+
+### Evaluation Dataset
+
+#### bobox/enhanced_nli-50_k
+
+* Dataset: bobox/enhanced_nli-50_k
+* Size: 3,052 evaluation samples
+* Columns: <code>sentence1</code> and <code>sentence2</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | sentence1                                                                          | sentence2                                                                          |
+  |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
+  | type    | string                                                                             | string                                                                             |
+  | details | <ul><li>min: 4 tokens</li><li>mean: 33.63 tokens</li><li>max: 328 tokens</li></ul> | <ul><li>min: 2 tokens</li><li>mean: 60.36 tokens</li><li>max: 501 tokens</li></ul> |
+* Samples:
+  | sentence1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | sentence2                                                                                                                                                                                                                                                                                                                   |
+  |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>The 17-year-old asked not to be named but said he lost control of his silver hatchback when he swerved to avoid a cat in Parkway, Chellaston, Derbyshire.<br>He estimated he was travelling at about 30mph when he smashed into the garage of a residential home on Friday night.<br>The owners were away at the time and the crash was reported to police.<br>The driver told BBC News: "I swerved to the right and the back of the car went out.<br>"I then swerved to the left and lost control - I couldn't bring it back.<br>"I just missed two parked cars and a tree and ended up in the wall - it was a big impact.<br>"I smelt burning, I thought it was the car, so I got out and laid down - I was in shock."<br>Neighbours reported hearing a "loud screeching" followed by a "massive bang".<br>The driver said: "It was a natural reaction to swerve to miss the cat, but I went into a state of shock and panic."<br>Derbyshire Police said the driver was due to appear before magistrates at a later date charged with driving without due care or attention.</code> | <code>A new driver who ploughed into a house, having swerved around two parked cars and a tree to avoid hitting a cat in the road, faces court.</code>                                                                                                                                                                      |
+  | <code>what requirements are needed to be a psychologist?</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | <code>To become a clinician, you must apply for registration with the College of Psychologists of Ontario, a process which requires 4 years of work experience and one year of supervised practice. In some provinces, you can be registered to practice as a psychologist with either a masters or doctoral degree.</code> |
+  | <code>What does sunlight create for plants?</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | <code>a plant requires sunlight for photosynthesis. Photosynthesis occurs using the suns energy to create the plants own energy. <br> sunlight creates energy for plants</code>                                                                                                                                             |
+* Loss: [<code>CachedGISTEmbedLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedgistembedloss) with these parameters:
+  ```json
+  {'guide': SentenceTransformer(
+    (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
+    (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+    (2): Normalize()
+  ), 'temperature': 0.025}
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 640
+- `per_device_eval_batch_size`: 128
+- `learning_rate`: 3.5e-05
+- `weight_decay`: 0.0001
+- `num_train_epochs`: 2
+- `lr_scheduler_type`: cosine_with_min_lr
+- `lr_scheduler_kwargs`: {'num_cycles': 0.5, 'min_lr': 5.833333333333333e-06}
+- `warmup_ratio`: 0.25
+- `save_safetensors`: False
+- `fp16`: True
+- `push_to_hub`: True
+- `hub_model_id`: bobox/DeBERTa-small-ST-v1-test-UnifiedDatasets-bis-checkpoints-tmp
+- `hub_strategy`: all_checkpoints
+- `batch_sampler`: no_duplicates
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 640
+- `per_device_eval_batch_size`: 128
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 3.5e-05
+- `weight_decay`: 0.0001
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 2
+- `max_steps`: -1
+- `lr_scheduler_type`: cosine_with_min_lr
+- `lr_scheduler_kwargs`: {'num_cycles': 0.5, 'min_lr': 5.833333333333333e-06}
+- `warmup_ratio`: 0.25
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: False
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: True
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: True
+- `resume_from_checkpoint`: None
+- `hub_model_id`: bobox/DeBERTa-small-ST-v1-test-UnifiedDatasets-bis-checkpoints-tmp
+- `hub_strategy`: all_checkpoints
+- `hub_private_repo`: False
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `eval_use_gather_object`: False
+- `batch_sampler`: no_duplicates
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+<details><summary>Click to expand</summary>
+
+| Epoch  | Step | Training Loss | loss   | sts-test_spearman_cosine |
+|:------:|:----:|:-------------:|:------:|:------------------------:|
+| 0.0053 | 1    | 0.3768        | -      | -                        |
+| 0.0106 | 2    | 0.3162        | -      | -                        |
+| 0.0159 | 3    | 0.275         | -      | -                        |
+| 0.0212 | 4    | 0.293         | -      | -                        |
+| 0.0265 | 5    | 0.2437        | 0.2190 | 0.9079                   |
+| 0.0317 | 6    | 0.3681        | -      | -                        |
+| 0.0370 | 7    | 0.2314        | -      | -                        |
+| 0.0423 | 8    | 0.2481        | -      | -                        |
+| 0.0476 | 9    | 0.2403        | -      | -                        |
+| 0.0529 | 10   | 0.2966        | 0.2125 | 0.9079                   |
+| 0.0582 | 11   | 0.2867        | -      | -                        |
+| 0.0635 | 12   | 0.3413        | -      | -                        |
+| 0.0688 | 13   | 0.4119        | -      | -                        |
+| 0.0741 | 14   | 0.3118        | -      | -                        |
+| 0.0794 | 15   | 0.327         | 0.2031 | 0.9082                   |
+| 0.0847 | 16   | 0.3389        | -      | -                        |
+| 0.0899 | 17   | 0.2018        | -      | -                        |
+| 0.0952 | 18   | 0.2861        | -      | -                        |
+| 0.1005 | 19   | 0.2848        | -      | -                        |
+| 0.1058 | 20   | 0.2563        | 0.1943 | 0.9082                   |
+| 0.1111 | 21   | 0.3058        | -      | -                        |
+| 0.1164 | 22   | 0.285         | -      | -                        |
+| 0.1217 | 23   | 0.3151        | -      | -                        |
+| 0.1270 | 24   | 0.2716        | -      | -                        |
+| 0.1323 | 25   | 0.2422        | 0.1794 | 0.9082                   |
+| 0.1376 | 26   | 0.2858        | -      | -                        |
+| 0.1429 | 27   | 0.3211        | -      | -                        |
+| 0.1481 | 28   | 0.2158        | -      | -                        |
+| 0.1534 | 29   | 0.2811        | -      | -                        |
+| 0.1587 | 30   | 0.2063        | 0.1636 | 0.9077                   |
+| 0.1640 | 31   | 0.2492        | -      | -                        |
+| 0.1693 | 32   | 0.3096        | -      | -                        |
+| 0.1746 | 33   | 0.2914        | -      | -                        |
+| 0.1799 | 34   | 0.2888        | -      | -                        |
+| 0.1852 | 35   | 0.223         | 0.1532 | 0.9072                   |
+| 0.1905 | 36   | 0.2595        | -      | -                        |
+| 0.1958 | 37   | 0.3122        | -      | -                        |
+| 0.2011 | 38   | 0.2327        | -      | -                        |
+| 0.2063 | 39   | 0.1718        | -      | -                        |
+| 0.2116 | 40   | 0.3162        | 0.1443 | 0.9067                   |
+| 0.2169 | 41   | 0.296         | -      | -                        |
+| 0.2222 | 42   | 0.2821        | -      | -                        |
+| 0.2275 | 43   | 0.2069        | -      | -                        |
+| 0.2328 | 44   | 0.2573        | -      | -                        |
+| 0.2381 | 45   | 0.3119        | 0.1343 | 0.9064                   |
+| 0.2434 | 46   | 0.2743        | -      | -                        |
+| 0.2487 | 47   | 0.2666        | -      | -                        |
+| 0.2540 | 48   | 0.2414        | -      | -                        |
+| 0.2593 | 49   | 0.2793        | -      | -                        |
+| 0.2646 | 50   | 0.2212        | 0.1251 | 0.9068                   |
+| 0.2698 | 51   | 0.2071        | -      | -                        |
+| 0.2751 | 52   | 0.296         | -      | -                        |
+| 0.2804 | 53   | 0.2061        | -      | -                        |
+| 0.2857 | 54   | 0.2164        | -      | -                        |
+| 0.2910 | 55   | 0.188         | 0.1197 | 0.9072                   |
+| 0.2963 | 56   | 0.2411        | -      | -                        |
+| 0.3016 | 57   | 0.2031        | -      | -                        |
+| 0.3069 | 58   | 0.2438        | -      | -                        |
+| 0.3122 | 59   | 0.2417        | -      | -                        |
+| 0.3175 | 60   | 0.1515        | 0.1233 | 0.9066                   |
+| 0.3228 | 61   | 0.21          | -      | -                        |
+| 0.3280 | 62   | 0.21          | -      | -                        |
+| 0.3333 | 63   | 0.2157        | -      | -                        |
+| 0.3386 | 64   | 0.2138        | -      | -                        |
+| 0.3439 | 65   | 0.2403        | 0.1273 | 0.9058                   |
+| 0.3492 | 66   | 0.2808        | -      | -                        |
+| 0.3545 | 67   | 0.1891        | -      | -                        |
+| 0.3598 | 68   | 0.1991        | -      | -                        |
+| 0.3651 | 69   | 0.2121        | -      | -                        |
+| 0.3704 | 70   | 0.2039        | 0.1311 | 0.9066                   |
+| 0.3757 | 71   | 0.1986        | -      | -                        |
+| 0.3810 | 72   | 0.2925        | -      | -                        |
+| 0.3862 | 73   | 0.2527        | -      | -                        |
+| 0.3915 | 74   | 0.279         | -      | -                        |
+| 0.3968 | 75   | 0.2419        | 0.1315 | 0.9066                   |
+| 0.4021 | 76   | 0.2228        | -      | -                        |
+| 0.4074 | 77   | 0.2242        | -      | -                        |
+| 0.4127 | 78   | 0.2737        | -      | -                        |
+| 0.4180 | 79   | 0.2328        | -      | -                        |
+| 0.4233 | 80   | 0.2802        | 0.1262 | 0.9058                   |
+| 0.4286 | 81   | 0.2044        | -      | -                        |
+| 0.4339 | 82   | 0.1828        | -      | -                        |
+| 0.4392 | 83   | 0.2372        | -      | -                        |
+| 0.4444 | 84   | 0.2241        | -      | -                        |
+| 0.4497 | 85   | 0.2782        | 0.1207 | 0.9063                   |
+| 0.4550 | 86   | 0.3244        | -      | -                        |
+| 0.4603 | 87   | 0.2102        | -      | -                        |
+| 0.4656 | 88   | 0.2265        | -      | -                        |
+| 0.4709 | 89   | 0.2666        | -      | -                        |
+| 0.4762 | 90   | 0.23          | 0.1186 | 0.9078                   |
+| 0.4815 | 91   | 0.2358        | -      | -                        |
+| 0.4868 | 92   | 0.2896        | -      | -                        |
+| 0.4921 | 93   | 0.2126        | -      | -                        |
+| 0.4974 | 94   | 0.2669        | -      | -                        |
+| 0.5026 | 95   | 0.2375        | 0.1128 | 0.9087                   |
+| 0.5079 | 96   | 0.1903        | -      | -                        |
+| 0.5132 | 97   | 0.2507        | -      | -                        |
+| 0.5185 | 98   | 0.1897        | -      | -                        |
+| 0.5238 | 99   | 0.2775        | -      | -                        |
+| 0.5291 | 100  | 0.2098        | 0.1168 | 0.9064                   |
+| 0.5344 | 101  | 0.1628        | -      | -                        |
+| 0.5397 | 102  | 0.2158        | -      | -                        |
+| 0.5450 | 103  | 0.1552        | -      | -                        |
+| 0.5503 | 104  | 0.2364        | -      | -                        |
+| 0.5556 | 105  | 0.272         | 0.1178 | 0.9056                   |
+| 0.5608 | 106  | 0.2271        | -      | -                        |
+| 0.5661 | 107  | 0.2132        | -      | -                        |
+| 0.5714 | 108  | 0.1782        | -      | -                        |
+| 0.5767 | 109  | 0.1598        | -      | -                        |
+| 0.5820 | 110  | 0.2472        | 0.1269 | 0.9050                   |
+| 0.5873 | 111  | 0.2041        | -      | -                        |
+| 0.5926 | 112  | 0.2426        | -      | -                        |
+| 0.5979 | 113  | 0.2105        | -      | -                        |
+| 0.6032 | 114  | 0.1923        | -      | -                        |
+| 0.6085 | 115  | 0.2271        | 0.1233 | 0.9061                   |
+| 0.6138 | 116  | 0.3029        | -      | -                        |
+| 0.6190 | 117  | 0.2554        | -      | -                        |
+| 0.6243 | 118  | 0.2182        | -      | -                        |
+| 0.6296 | 119  | 0.2852        | -      | -                        |
+| 0.6349 | 120  | 0.2285        | 0.1280 | 0.9053                   |
+| 0.6402 | 121  | 0.218         | -      | -                        |
+| 0.6455 | 122  | 0.1841        | -      | -                        |
+| 0.6508 | 123  | 0.2629        | -      | -                        |
+| 0.6561 | 124  | 0.1749        | -      | -                        |
+| 0.6614 | 125  | 0.2417        | 0.1415 | 0.9057                   |
+| 0.6667 | 126  | 0.2305        | -      | -                        |
+| 0.6720 | 127  | 0.2841        | -      | -                        |
+| 0.6772 | 128  | 0.1785        | -      | -                        |
+| 0.6825 | 129  | 0.2153        | -      | -                        |
+| 0.6878 | 130  | 0.2548        | 0.1413 | 0.9079                   |
+| 0.6931 | 131  | 0.2059        | -      | -                        |
+| 0.6984 | 132  | 0.2073        | -      | -                        |
+| 0.7037 | 133  | 0.191         | -      | -                        |
+| 0.7090 | 134  | 0.1633        | -      | -                        |
+| 0.7143 | 135  | 0.2627        | 0.1333 | 0.9077                   |
+| 0.7196 | 136  | 0.2451        | -      | -                        |
+| 0.7249 | 137  | 0.1441        | -      | -                        |
+| 0.7302 | 138  | 0.2138        | -      | -                        |
+| 0.7354 | 139  | 0.2564        | -      | -                        |
+| 0.7407 | 140  | 0.1524        | 0.1323 | 0.9049                   |
+| 0.7460 | 141  | 0.1786        | -      | -                        |
+| 0.7513 | 142  | 0.2104        | -      | -                        |
+| 0.7566 | 143  | 0.2512        | -      | -                        |
+| 0.7619 | 144  | 0.1889        | -      | -                        |
+| 0.7672 | 145  | 0.2127        | 0.1291 | 0.9015                   |
+| 0.7725 | 146  | 0.2115        | -      | -                        |
+| 0.7778 | 147  | 0.179         | -      | -                        |
+| 0.7831 | 148  | 0.2188        | -      | -                        |
+| 0.7884 | 149  | 0.1687        | -      | -                        |
+| 0.7937 | 150  | 0.2265        | 0.1145 | 0.9014                   |
+| 0.7989 | 151  | 0.182         | -      | -                        |
+| 0.8042 | 152  | 0.1789        | -      | -                        |
+| 0.8095 | 153  | 0.225         | -      | -                        |
+| 0.8148 | 154  | 0.3015        | -      | -                        |
+| 0.8201 | 155  | 0.1656        | 0.1009 | 0.9045                   |
+| 0.8254 | 156  | 0.2321        | -      | -                        |
+| 0.8307 | 157  | 0.2514        | -      | -                        |
+| 0.8360 | 158  | 0.2348        | -      | -                        |
+| 0.8413 | 159  | 0.2003        | -      | -                        |
+| 0.8466 | 160  | 0.2487        | 0.0983 | 0.9057                   |
+| 0.8519 | 161  | 0.1674        | -      | -                        |
+| 0.8571 | 162  | 0.1968        | -      | -                        |
+| 0.8624 | 163  | 0.2237        | -      | -                        |
+| 0.8677 | 164  | 0.2636        | -      | -                        |
+| 0.8730 | 165  | 0.1858        | 0.1015 | 0.9043                   |
+| 0.8783 | 166  | 0.198         | -      | -                        |
+| 0.8836 | 167  | 0.2011        | -      | -                        |
+| 0.8889 | 168  | 0.2887        | -      | -                        |
+| 0.8942 | 169  | 0.2014        | -      | -                        |
+| 0.8995 | 170  | 0.2122        | 0.1002 | 0.9025                   |
+| 0.9048 | 171  | 0.2537        | -      | -                        |
+| 0.9101 | 172  | 0.2104        | -      | -                        |
+| 0.9153 | 173  | 0.1291        | -      | -                        |
+| 0.9206 | 174  | 0.1794        | -      | -                        |
+| 0.9259 | 175  | 0.2569        | 0.0948 | 0.9027                   |
+| 0.9312 | 176  | 0.26          | -      | -                        |
+| 0.9365 | 177  | 0.1665        | -      | -                        |
+| 0.9418 | 178  | 0.2012        | -      | -                        |
+| 0.9471 | 179  | 0.2171        | -      | -                        |
+| 0.9524 | 180  | 0.1554        | 0.0874 | 0.9055                   |
+| 0.9577 | 181  | 0.2177        | -      | -                        |
+| 0.9630 | 182  | 0.1919        | -      | -                        |
+| 0.9683 | 183  | 0.1686        | -      | -                        |
+| 0.9735 | 184  | 0.2277        | -      | -                        |
+| 0.9788 | 185  | 0.2169        | 0.0847 | 0.9089                   |
+| 0.9841 | 186  | 0.1991        | -      | -                        |
+| 0.9894 | 187  | 0.2373        | -      | -                        |
+| 0.9947 | 188  | 0.1636        | -      | -                        |
+| 1.0    | 189  | 0.0           | -      | -                        |
+| 1.0053 | 190  | 0.0           | 0.0846 | 0.9103                   |
+| 1.0106 | 191  | 0.0           | -      | -                        |
+| 1.0159 | 192  | 0.0           | -      | -                        |
+| 1.0212 | 193  | 0.0           | -      | -                        |
+| 1.0265 | 194  | 0.0           | -      | -                        |
+| 1.0317 | 195  | 0.0           | 0.0859 | 0.9098                   |
+| 1.0370 | 196  | 0.0           | -      | -                        |
+| 1.0423 | 197  | 0.0           | -      | -                        |
+| 1.0476 | 198  | 0.0           | -      | -                        |
+| 1.0529 | 199  | 0.0           | -      | -                        |
+| 1.0582 | 200  | 0.0           | 0.0870 | 0.9101                   |
+| 1.0635 | 201  | 0.0           | -      | -                        |
+| 1.0688 | 202  | 0.0           | -      | -                        |
+| 1.0741 | 203  | 0.0           | -      | -                        |
+| 1.0794 | 204  | 0.0           | -      | -                        |
+| 1.0847 | 205  | 0.0           | 0.0879 | 0.9100                   |
+| 1.0899 | 206  | 0.0           | -      | -                        |
+| 1.0952 | 207  | 0.0           | -      | -                        |
+| 1.1005 | 208  | 0.0           | -      | -                        |
+| 1.1058 | 209  | 0.0           | -      | -                        |
+| 1.1111 | 210  | 0.0           | 0.0884 | 0.9097                   |
+| 1.1164 | 211  | 0.0           | -      | -                        |
+| 1.1217 | 212  | 0.0           | -      | -                        |
+| 1.1270 | 213  | 0.0           | -      | -                        |
+| 1.1323 | 214  | 0.0           | -      | -                        |
+| 1.1376 | 215  | 0.0           | 0.0888 | 0.9096                   |
+| 1.1429 | 216  | 0.0           | -      | -                        |
+| 1.1481 | 217  | 0.0           | -      | -                        |
+| 1.1534 | 218  | 0.0           | -      | -                        |
+| 1.1587 | 219  | 0.0           | -      | -                        |
+| 1.1640 | 220  | 0.0           | 0.0890 | 0.9095                   |
+| 1.1693 | 221  | 0.0           | -      | -                        |
+| 1.1746 | 222  | 0.0           | -      | -                        |
+| 1.1799 | 223  | 0.0           | -      | -                        |
+| 1.1852 | 224  | 0.0           | -      | -                        |
+| 1.1905 | 225  | 0.0           | 0.0891 | 0.9095                   |
+| 1.1958 | 226  | 0.0           | -      | -                        |
+| 1.2011 | 227  | 0.0           | -      | -                        |
+| 1.2063 | 228  | 0.0           | -      | -                        |
+| 1.2116 | 229  | 0.0           | -      | -                        |
+| 1.2169 | 230  | 0.0           | 0.0892 | 0.9095                   |
+| 1.2222 | 231  | 0.0           | -      | -                        |
+| 1.2275 | 232  | 0.0           | -      | -                        |
+| 1.2328 | 233  | 0.0           | -      | -                        |
+| 1.2381 | 234  | 0.0           | -      | -                        |
+| 1.2434 | 235  | 0.0           | 0.0892 | 0.9094                   |
+| 1.2487 | 236  | 0.0           | -      | -                        |
+| 1.2540 | 237  | 0.0           | -      | -                        |
+| 1.2593 | 238  | 0.0           | -      | -                        |
+| 1.2646 | 239  | 0.0           | -      | -                        |
+| 1.2698 | 240  | 0.0           | 0.0892 | 0.9093                   |
+| 1.2751 | 241  | 0.0           | -      | -                        |
+| 1.2804 | 242  | 0.0           | -      | -                        |
+| 1.2857 | 243  | 0.0           | -      | -                        |
+| 1.2910 | 244  | 0.0           | -      | -                        |
+| 1.2963 | 245  | 0.0           | 0.0892 | 0.9094                   |
+| 1.3016 | 246  | 0.0           | -      | -                        |
+| 1.3069 | 247  | 0.0           | -      | -                        |
+| 1.3122 | 248  | 0.0           | -      | -                        |
+| 1.3175 | 249  | 0.0           | -      | -                        |
+| 1.3228 | 250  | 0.0           | 0.0892 | 0.9094                   |
+| 1.3280 | 251  | 0.0           | -      | -                        |
+| 1.3333 | 252  | 0.0           | -      | -                        |
+| 1.3386 | 253  | 0.0           | -      | -                        |
+| 1.3439 | 254  | 0.0           | -      | -                        |
+| 1.3492 | 255  | 0.0           | 0.0893 | 0.9093                   |
+| 1.3545 | 256  | 0.0           | -      | -                        |
+| 1.3598 | 257  | 0.0           | -      | -                        |
+| 1.3651 | 258  | 0.0           | -      | -                        |
+| 1.3704 | 259  | 0.0           | -      | -                        |
+| 1.3757 | 260  | 0.0           | 0.0893 | 0.9093                   |
+| 1.3810 | 261  | 0.0           | -      | -                        |
+| 1.3862 | 262  | 0.0           | -      | -                        |
+| 1.3915 | 263  | 0.0           | -      | -                        |
+| 1.3968 | 264  | 0.0           | -      | -                        |
+| 1.4021 | 265  | 0.0           | 0.0893 | 0.9093                   |
+| 1.4074 | 266  | 0.0           | -      | -                        |
+
+</details>
+
+### Framework Versions
+- Python: 3.10.14
+- Sentence Transformers: 3.0.1
+- Transformers: 4.44.0
+- PyTorch: 2.4.0
+- Accelerate: 0.33.0
+- Datasets: 2.21.0
+- Tokenizers: 0.19.1
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

checkpoint-266/added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "[MASK]": 128000
+}

checkpoint-266/config.json

@@ -0,0 +1,35 @@
+{
+  "_name_or_path": "bobox/DeBERTa-small-ST-v1-test-step3",
+  "architectures": [
+    "DebertaV2Model"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-07,
+  "max_position_embeddings": 512,
+  "max_relative_positions": -1,
+  "model_type": "deberta-v2",
+  "norm_rel_ebd": "layer_norm",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 6,
+  "pad_token_id": 0,
+  "pooler_dropout": 0,
+  "pooler_hidden_act": "gelu",
+  "pooler_hidden_size": 768,
+  "pos_att_type": [
+    "p2c",
+    "c2p"
+  ],
+  "position_biased_input": false,
+  "position_buckets": 256,
+  "relative_attention": true,
+  "share_att_key": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.44.0",
+  "type_vocab_size": 0,
+  "vocab_size": 128100
+}

checkpoint-266/config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.0.1",
+    "transformers": "4.44.0",
+    "pytorch": "2.4.0"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": null
+}

checkpoint-266/modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

checkpoint-266/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55877634e8de730af43d7e4902e254c731066aeef13fc17bdf3f50ac67da9e10
+size 1130520122

checkpoint-266/pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7da3426fc032212e000b4ae956eeb9aa3f498fdf55441989b82bc00297421f2
+size 565251810

checkpoint-266/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b6bdf8a249c880772b1497cf2dc30f93e166453d3143f24ca63fa5c5a93ff6cd
+size 14244

checkpoint-266/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cee8c80a5d1f31d666e6ba0087d5c118b6162ad83d34133340503d52e83df8b9
+size 1064

checkpoint-266/sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": false
+}

checkpoint-266/special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-266/spm.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c679fbf93643d19aab7ee10c0b99e460bdbc02fedf34b92b05af343b4af586fd
+size 2464616

checkpoint-266/tokenizer_config.json

@@ -0,0 +1,65 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128000": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "[CLS]",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "eos_token": "[SEP]",
+  "mask_token": "[MASK]",
+  "max_length": 512,
+  "model_max_length": 512,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "sp_model_kwargs": {},
+  "split_by_punct": false,
+  "stride": 0,
+  "tokenizer_class": "DebertaV2Tokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]",
+  "vocab_type": "spm"
+}

checkpoint-266/trainer_state.json

@@ -0,0 +1,2849 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.4074074074074074,
+  "eval_steps": 5,
+  "global_step": 266,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.005291005291005291,
+      "grad_norm": 4.4978814125061035,
+      "learning_rate": 3.684210526315789e-07,
+      "loss": 0.3768,
+      "step": 1
+    },
+    {
+      "epoch": 0.010582010582010581,
+      "grad_norm": 4.62701416015625,
+      "learning_rate": 7.368421052631578e-07,
+      "loss": 0.3162,
+      "step": 2
+    },
+    {
+      "epoch": 0.015873015873015872,
+      "grad_norm": 4.304025650024414,
+      "learning_rate": 1.1052631578947367e-06,
+      "loss": 0.275,
+      "step": 3
+    },
+    {
+      "epoch": 0.021164021164021163,
+      "grad_norm": 4.237359046936035,
+      "learning_rate": 1.4736842105263156e-06,
+      "loss": 0.293,
+      "step": 4
+    },
+    {
+      "epoch": 0.026455026455026454,
+      "grad_norm": 3.979759693145752,
+      "learning_rate": 1.8421052631578944e-06,
+      "loss": 0.2437,
+      "step": 5
+    },
+    {
+      "epoch": 0.026455026455026454,
+      "eval_loss": 0.2190357893705368,
+      "eval_runtime": 116.0092,
+      "eval_samples_per_second": 26.308,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8862434303901607,
+      "eval_sts-test_pearson_dot": 0.8769941125545049,
+      "eval_sts-test_pearson_euclidean": 0.9081330515608292,
+      "eval_sts-test_pearson_manhattan": 0.9088207208262792,
+      "eval_sts-test_pearson_max": 0.9088207208262792,
+      "eval_sts-test_spearman_cosine": 0.9078630669346023,
+      "eval_sts-test_spearman_dot": 0.8793787995564538,
+      "eval_sts-test_spearman_euclidean": 0.9040411749579927,
+      "eval_sts-test_spearman_manhattan": 0.9044944604846262,
+      "eval_sts-test_spearman_max": 0.9078630669346023,
+      "step": 5
+    },
+    {
+      "epoch": 0.031746031746031744,
+      "grad_norm": 4.23664665222168,
+      "learning_rate": 2.2105263157894734e-06,
+      "loss": 0.3681,
+      "step": 6
+    },
+    {
+      "epoch": 0.037037037037037035,
+      "grad_norm": 3.7219350337982178,
+      "learning_rate": 2.5789473684210523e-06,
+      "loss": 0.2314,
+      "step": 7
+    },
+    {
+      "epoch": 0.042328042328042326,
+      "grad_norm": 4.135527610778809,
+      "learning_rate": 2.9473684210526313e-06,
+      "loss": 0.2481,
+      "step": 8
+    },
+    {
+      "epoch": 0.047619047619047616,
+      "grad_norm": 3.6268699169158936,
+      "learning_rate": 3.3157894736842103e-06,
+      "loss": 0.2403,
+      "step": 9
+    },
+    {
+      "epoch": 0.05291005291005291,
+      "grad_norm": 4.4451069831848145,
+      "learning_rate": 3.684210526315789e-06,
+      "loss": 0.2966,
+      "step": 10
+    },
+    {
+      "epoch": 0.05291005291005291,
+      "eval_loss": 0.21254141628742218,
+      "eval_runtime": 115.9559,
+      "eval_samples_per_second": 26.32,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8863175059264351,
+      "eval_sts-test_pearson_dot": 0.8768417645238664,
+      "eval_sts-test_pearson_euclidean": 0.908487750463006,
+      "eval_sts-test_pearson_manhattan": 0.9091691821321777,
+      "eval_sts-test_pearson_max": 0.9091691821321777,
+      "eval_sts-test_spearman_cosine": 0.9079303280804736,
+      "eval_sts-test_spearman_dot": 0.8797622731153167,
+      "eval_sts-test_spearman_euclidean": 0.9046558335332037,
+      "eval_sts-test_spearman_manhattan": 0.905077211410465,
+      "eval_sts-test_spearman_max": 0.9079303280804736,
+      "step": 10
+    },
+    {
+      "epoch": 0.0582010582010582,
+      "grad_norm": 4.20427942276001,
+      "learning_rate": 4.052631578947368e-06,
+      "loss": 0.2867,
+      "step": 11
+    },
+    {
+      "epoch": 0.06349206349206349,
+      "grad_norm": 4.593740463256836,
+      "learning_rate": 4.421052631578947e-06,
+      "loss": 0.3413,
+      "step": 12
+    },
+    {
+      "epoch": 0.06878306878306878,
+      "grad_norm": 4.476904392242432,
+      "learning_rate": 4.789473684210526e-06,
+      "loss": 0.4119,
+      "step": 13
+    },
+    {
+      "epoch": 0.07407407407407407,
+      "grad_norm": 4.580070495605469,
+      "learning_rate": 5.157894736842105e-06,
+      "loss": 0.3118,
+      "step": 14
+    },
+    {
+      "epoch": 0.07936507936507936,
+      "grad_norm": 4.245151042938232,
+      "learning_rate": 5.526315789473683e-06,
+      "loss": 0.327,
+      "step": 15
+    },
+    {
+      "epoch": 0.07936507936507936,
+      "eval_loss": 0.20313723385334015,
+      "eval_runtime": 115.9643,
+      "eval_samples_per_second": 26.318,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.886401463284574,
+      "eval_sts-test_pearson_dot": 0.8764131384801311,
+      "eval_sts-test_pearson_euclidean": 0.9089963447148983,
+      "eval_sts-test_pearson_manhattan": 0.9096769425767649,
+      "eval_sts-test_pearson_max": 0.9096769425767649,
+      "eval_sts-test_spearman_cosine": 0.9081527866008503,
+      "eval_sts-test_spearman_dot": 0.8798801031865603,
+      "eval_sts-test_spearman_euclidean": 0.9051157869977845,
+      "eval_sts-test_spearman_manhattan": 0.9058301961346367,
+      "eval_sts-test_spearman_max": 0.9081527866008503,
+      "step": 15
+    },
+    {
+      "epoch": 0.08465608465608465,
+      "grad_norm": 4.45465087890625,
+      "learning_rate": 5.894736842105263e-06,
+      "loss": 0.3389,
+      "step": 16
+    },
+    {
+      "epoch": 0.08994708994708994,
+      "grad_norm": 3.7058451175689697,
+      "learning_rate": 6.263157894736842e-06,
+      "loss": 0.2018,
+      "step": 17
+    },
+    {
+      "epoch": 0.09523809523809523,
+      "grad_norm": 4.276317119598389,
+      "learning_rate": 6.6315789473684205e-06,
+      "loss": 0.2861,
+      "step": 18
+    },
+    {
+      "epoch": 0.10052910052910052,
+      "grad_norm": 4.1567511558532715,
+      "learning_rate": 7e-06,
+      "loss": 0.2848,
+      "step": 19
+    },
+    {
+      "epoch": 0.10582010582010581,
+      "grad_norm": 3.727539300918579,
+      "learning_rate": 7.368421052631578e-06,
+      "loss": 0.2563,
+      "step": 20
+    },
+    {
+      "epoch": 0.10582010582010581,
+      "eval_loss": 0.19427762925624847,
+      "eval_runtime": 115.9622,
+      "eval_samples_per_second": 26.319,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8859728422141003,
+      "eval_sts-test_pearson_dot": 0.8747321172801832,
+      "eval_sts-test_pearson_euclidean": 0.9094621598863318,
+      "eval_sts-test_pearson_manhattan": 0.9101781902466943,
+      "eval_sts-test_pearson_max": 0.9101781902466943,
+      "eval_sts-test_spearman_cosine": 0.9082097102053401,
+      "eval_sts-test_spearman_dot": 0.8789482924218687,
+      "eval_sts-test_spearman_euclidean": 0.9059236815196834,
+      "eval_sts-test_spearman_manhattan": 0.9063636311704222,
+      "eval_sts-test_spearman_max": 0.9082097102053401,
+      "step": 20
+    },
+    {
+      "epoch": 0.1111111111111111,
+      "grad_norm": 4.300179958343506,
+      "learning_rate": 7.736842105263158e-06,
+      "loss": 0.3058,
+      "step": 21
+    },
+    {
+      "epoch": 0.1164021164021164,
+      "grad_norm": 4.133388042449951,
+      "learning_rate": 8.105263157894736e-06,
+      "loss": 0.285,
+      "step": 22
+    },
+    {
+      "epoch": 0.12169312169312169,
+      "grad_norm": 4.270442962646484,
+      "learning_rate": 8.473684210526315e-06,
+      "loss": 0.3151,
+      "step": 23
+    },
+    {
+      "epoch": 0.12698412698412698,
+      "grad_norm": 3.83665132522583,
+      "learning_rate": 8.842105263157893e-06,
+      "loss": 0.2716,
+      "step": 24
+    },
+    {
+      "epoch": 0.13227513227513227,
+      "grad_norm": 3.7526190280914307,
+      "learning_rate": 9.210526315789472e-06,
+      "loss": 0.2422,
+      "step": 25
+    },
+    {
+      "epoch": 0.13227513227513227,
+      "eval_loss": 0.17938196659088135,
+      "eval_runtime": 115.932,
+      "eval_samples_per_second": 26.326,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8855121875537549,
+      "eval_sts-test_pearson_dot": 0.8729669422265561,
+      "eval_sts-test_pearson_euclidean": 0.9099991783491139,
+      "eval_sts-test_pearson_manhattan": 0.9106856714841502,
+      "eval_sts-test_pearson_max": 0.9106856714841502,
+      "eval_sts-test_spearman_cosine": 0.9081626318783566,
+      "eval_sts-test_spearman_dot": 0.8777849833819992,
+      "eval_sts-test_spearman_euclidean": 0.9059934934874541,
+      "eval_sts-test_spearman_manhattan": 0.9070222354843194,
+      "eval_sts-test_spearman_max": 0.9081626318783566,
+      "step": 25
+    },
+    {
+      "epoch": 0.13756613756613756,
+      "grad_norm": 4.082998275756836,
+      "learning_rate": 9.578947368421052e-06,
+      "loss": 0.2858,
+      "step": 26
+    },
+    {
+      "epoch": 0.14285714285714285,
+      "grad_norm": 4.099833965301514,
+      "learning_rate": 9.94736842105263e-06,
+      "loss": 0.3211,
+      "step": 27
+    },
+    {
+      "epoch": 0.14814814814814814,
+      "grad_norm": 3.2643215656280518,
+      "learning_rate": 1.031578947368421e-05,
+      "loss": 0.2158,
+      "step": 28
+    },
+    {
+      "epoch": 0.15343915343915343,
+      "grad_norm": 3.7243423461914062,
+      "learning_rate": 1.068421052631579e-05,
+      "loss": 0.2811,
+      "step": 29
+    },
+    {
+      "epoch": 0.15873015873015872,
+      "grad_norm": 3.2406320571899414,
+      "learning_rate": 1.1052631578947366e-05,
+      "loss": 0.2063,
+      "step": 30
+    },
+    {
+      "epoch": 0.15873015873015872,
+      "eval_loss": 0.16362369060516357,
+      "eval_runtime": 115.8583,
+      "eval_samples_per_second": 26.343,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.884331218823937,
+      "eval_sts-test_pearson_dot": 0.8704189891244698,
+      "eval_sts-test_pearson_euclidean": 0.9099375497606286,
+      "eval_sts-test_pearson_manhattan": 0.9105955358876667,
+      "eval_sts-test_pearson_max": 0.9105955358876667,
+      "eval_sts-test_spearman_cosine": 0.907708540829018,
+      "eval_sts-test_spearman_dot": 0.8752376073298165,
+      "eval_sts-test_spearman_euclidean": 0.9062101343438497,
+      "eval_sts-test_spearman_manhattan": 0.9067411975627813,
+      "eval_sts-test_spearman_max": 0.907708540829018,
+      "step": 30
+    },
+    {
+      "epoch": 0.164021164021164,
+      "grad_norm": 3.6572649478912354,
+      "learning_rate": 1.1421052631578947e-05,
+      "loss": 0.2492,
+      "step": 31
+    },
+    {
+      "epoch": 0.1693121693121693,
+      "grad_norm": 3.982050657272339,
+      "learning_rate": 1.1789473684210525e-05,
+      "loss": 0.3096,
+      "step": 32
+    },
+    {
+      "epoch": 0.1746031746031746,
+      "grad_norm": 3.694265365600586,
+      "learning_rate": 1.2157894736842105e-05,
+      "loss": 0.2914,
+      "step": 33
+    },
+    {
+      "epoch": 0.17989417989417988,
+      "grad_norm": 3.955336809158325,
+      "learning_rate": 1.2526315789473684e-05,
+      "loss": 0.2888,
+      "step": 34
+    },
+    {
+      "epoch": 0.18518518518518517,
+      "grad_norm": 3.3689701557159424,
+      "learning_rate": 1.289473684210526e-05,
+      "loss": 0.223,
+      "step": 35
+    },
+    {
+      "epoch": 0.18518518518518517,
+      "eval_loss": 0.15322057902812958,
+      "eval_runtime": 115.8758,
+      "eval_samples_per_second": 26.339,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8831736003329786,
+      "eval_sts-test_pearson_dot": 0.8675011167098496,
+      "eval_sts-test_pearson_euclidean": 0.9098679645586549,
+      "eval_sts-test_pearson_manhattan": 0.9104851057704236,
+      "eval_sts-test_pearson_max": 0.9104851057704236,
+      "eval_sts-test_spearman_cosine": 0.9071846378119088,
+      "eval_sts-test_spearman_dot": 0.8722759240996724,
+      "eval_sts-test_spearman_euclidean": 0.9062134214414588,
+      "eval_sts-test_spearman_manhattan": 0.9068589381315022,
+      "eval_sts-test_spearman_max": 0.9071846378119088,
+      "step": 35
+    },
+    {
+      "epoch": 0.19047619047619047,
+      "grad_norm": 3.665268898010254,
+      "learning_rate": 1.3263157894736841e-05,
+      "loss": 0.2595,
+      "step": 36
+    },
+    {
+      "epoch": 0.19576719576719576,
+      "grad_norm": 3.816418409347534,
+      "learning_rate": 1.363157894736842e-05,
+      "loss": 0.3122,
+      "step": 37
+    },
+    {
+      "epoch": 0.20105820105820105,
+      "grad_norm": 3.2825276851654053,
+      "learning_rate": 1.4e-05,
+      "loss": 0.2327,
+      "step": 38
+    },
+    {
+      "epoch": 0.20634920634920634,
+      "grad_norm": 2.9109766483306885,
+      "learning_rate": 1.4368421052631578e-05,
+      "loss": 0.1718,
+      "step": 39
+    },
+    {
+      "epoch": 0.21164021164021163,
+      "grad_norm": 3.6622438430786133,
+      "learning_rate": 1.4736842105263155e-05,
+      "loss": 0.3162,
+      "step": 40
+    },
+    {
+      "epoch": 0.21164021164021163,
+      "eval_loss": 0.14427155256271362,
+      "eval_runtime": 115.8732,
+      "eval_samples_per_second": 26.339,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8815373372232074,
+      "eval_sts-test_pearson_dot": 0.8638584960095411,
+      "eval_sts-test_pearson_euclidean": 0.909867067075244,
+      "eval_sts-test_pearson_manhattan": 0.9104161328856638,
+      "eval_sts-test_pearson_max": 0.9104161328856638,
+      "eval_sts-test_spearman_cosine": 0.9066622115864256,
+      "eval_sts-test_spearman_dot": 0.8693618114603872,
+      "eval_sts-test_spearman_euclidean": 0.9060806241933829,
+      "eval_sts-test_spearman_manhattan": 0.907040852009058,
+      "eval_sts-test_spearman_max": 0.907040852009058,
+      "step": 40
+    },
+    {
+      "epoch": 0.21693121693121692,
+      "grad_norm": 3.7417047023773193,
+      "learning_rate": 1.5105263157894735e-05,
+      "loss": 0.296,
+      "step": 41
+    },
+    {
+      "epoch": 0.2222222222222222,
+      "grad_norm": 3.4067814350128174,
+      "learning_rate": 1.5473684210526316e-05,
+      "loss": 0.2821,
+      "step": 42
+    },
+    {
+      "epoch": 0.2275132275132275,
+      "grad_norm": 3.0177342891693115,
+      "learning_rate": 1.5842105263157892e-05,
+      "loss": 0.2069,
+      "step": 43
+    },
+    {
+      "epoch": 0.2328042328042328,
+      "grad_norm": 3.4185049533843994,
+      "learning_rate": 1.6210526315789473e-05,
+      "loss": 0.2573,
+      "step": 44
+    },
+    {
+      "epoch": 0.23809523809523808,
+      "grad_norm": 3.918159246444702,
+      "learning_rate": 1.657894736842105e-05,
+      "loss": 0.3119,
+      "step": 45
+    },
+    {
+      "epoch": 0.23809523809523808,
+      "eval_loss": 0.13426049053668976,
+      "eval_runtime": 115.9898,
+      "eval_samples_per_second": 26.313,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8805625986040617,
+      "eval_sts-test_pearson_dot": 0.8616132264481751,
+      "eval_sts-test_pearson_euclidean": 0.9101879546714963,
+      "eval_sts-test_pearson_manhattan": 0.9107296923568013,
+      "eval_sts-test_pearson_max": 0.9107296923568013,
+      "eval_sts-test_spearman_cosine": 0.9064269094540296,
+      "eval_sts-test_spearman_dot": 0.8677769455382106,
+      "eval_sts-test_spearman_euclidean": 0.9069313904236947,
+      "eval_sts-test_spearman_manhattan": 0.9076995905767397,
+      "eval_sts-test_spearman_max": 0.9076995905767397,
+      "step": 45
+    },
+    {
+      "epoch": 0.24338624338624337,
+      "grad_norm": 3.0733871459960938,
+      "learning_rate": 1.694736842105263e-05,
+      "loss": 0.2743,
+      "step": 46
+    },
+    {
+      "epoch": 0.24867724867724866,
+      "grad_norm": 3.3528106212615967,
+      "learning_rate": 1.731578947368421e-05,
+      "loss": 0.2666,
+      "step": 47
+    },
+    {
+      "epoch": 0.25396825396825395,
+      "grad_norm": 3.0437304973602295,
+      "learning_rate": 1.7684210526315787e-05,
+      "loss": 0.2414,
+      "step": 48
+    },
+    {
+      "epoch": 0.25925925925925924,
+      "grad_norm": 3.451265573501587,
+      "learning_rate": 1.8052631578947367e-05,
+      "loss": 0.2793,
+      "step": 49
+    },
+    {
+      "epoch": 0.26455026455026454,
+      "grad_norm": 2.9664905071258545,
+      "learning_rate": 1.8421052631578944e-05,
+      "loss": 0.2212,
+      "step": 50
+    },
+    {
+      "epoch": 0.26455026455026454,
+      "eval_loss": 0.12510398030281067,
+      "eval_runtime": 116.2617,
+      "eval_samples_per_second": 26.251,
+      "eval_steps_per_second": 0.206,
+      "eval_sts-test_pearson_cosine": 0.8803749979605222,
+      "eval_sts-test_pearson_dot": 0.8597944232348109,
+      "eval_sts-test_pearson_euclidean": 0.9105906609339514,
+      "eval_sts-test_pearson_manhattan": 0.9111264139452835,
+      "eval_sts-test_pearson_max": 0.9111264139452835,
+      "eval_sts-test_spearman_cosine": 0.9067783858609977,
+      "eval_sts-test_spearman_dot": 0.8664887804790594,
+      "eval_sts-test_spearman_euclidean": 0.9077306479521453,
+      "eval_sts-test_spearman_manhattan": 0.9082951851145977,
+      "eval_sts-test_spearman_max": 0.9082951851145977,
+      "step": 50
+    },
+    {
+      "epoch": 0.2698412698412698,
+      "grad_norm": 3.051281690597534,
+      "learning_rate": 1.8789473684210524e-05,
+      "loss": 0.2071,
+      "step": 51
+    },
+    {
+      "epoch": 0.2751322751322751,
+      "grad_norm": 3.539724826812744,
+      "learning_rate": 1.9157894736842104e-05,
+      "loss": 0.296,
+      "step": 52
+    },
+    {
+      "epoch": 0.2804232804232804,
+      "grad_norm": 3.101149559020996,
+      "learning_rate": 1.952631578947368e-05,
+      "loss": 0.2061,
+      "step": 53
+    },
+    {
+      "epoch": 0.2857142857142857,
+      "grad_norm": 3.113757848739624,
+      "learning_rate": 1.989473684210526e-05,
+      "loss": 0.2164,
+      "step": 54
+    },
+    {
+      "epoch": 0.291005291005291,
+      "grad_norm": 2.9952962398529053,
+      "learning_rate": 2.0263157894736842e-05,
+      "loss": 0.188,
+      "step": 55
+    },
+    {
+      "epoch": 0.291005291005291,
+      "eval_loss": 0.11972539871931076,
+      "eval_runtime": 115.8621,
+      "eval_samples_per_second": 26.342,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.881021505301097,
+      "eval_sts-test_pearson_dot": 0.8597228411065384,
+      "eval_sts-test_pearson_euclidean": 0.9108125110302233,
+      "eval_sts-test_pearson_manhattan": 0.9112746469255748,
+      "eval_sts-test_pearson_max": 0.9112746469255748,
+      "eval_sts-test_spearman_cosine": 0.9072042388643983,
+      "eval_sts-test_spearman_dot": 0.8659804956521757,
+      "eval_sts-test_spearman_euclidean": 0.9079662185921096,
+      "eval_sts-test_spearman_manhattan": 0.9082669470686597,
+      "eval_sts-test_spearman_max": 0.9082669470686597,
+      "step": 55
+    },
+    {
+      "epoch": 0.2962962962962963,
+      "grad_norm": 3.065542697906494,
+      "learning_rate": 2.063157894736842e-05,
+      "loss": 0.2411,
+      "step": 56
+    },
+    {
+      "epoch": 0.30158730158730157,
+      "grad_norm": 2.8689684867858887,
+      "learning_rate": 2.1e-05,
+      "loss": 0.2031,
+      "step": 57
+    },
+    {
+      "epoch": 0.30687830687830686,
+      "grad_norm": 2.9906833171844482,
+      "learning_rate": 2.136842105263158e-05,
+      "loss": 0.2438,
+      "step": 58
+    },
+    {
+      "epoch": 0.31216931216931215,
+      "grad_norm": 3.040048837661743,
+      "learning_rate": 2.1736842105263156e-05,
+      "loss": 0.2417,
+      "step": 59
+    },
+    {
+      "epoch": 0.31746031746031744,
+      "grad_norm": 2.7314910888671875,
+      "learning_rate": 2.2105263157894733e-05,
+      "loss": 0.1515,
+      "step": 60
+    },
+    {
+      "epoch": 0.31746031746031744,
+      "eval_loss": 0.12329312413930893,
+      "eval_runtime": 115.9779,
+      "eval_samples_per_second": 26.315,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8813346129633393,
+      "eval_sts-test_pearson_dot": 0.8610768249798144,
+      "eval_sts-test_pearson_euclidean": 0.9107487658386748,
+      "eval_sts-test_pearson_manhattan": 0.9112378269812353,
+      "eval_sts-test_pearson_max": 0.9112378269812353,
+      "eval_sts-test_spearman_cosine": 0.9065531975136762,
+      "eval_sts-test_spearman_dot": 0.867737743433232,
+      "eval_sts-test_spearman_euclidean": 0.9073249330163708,
+      "eval_sts-test_spearman_manhattan": 0.9077088988391091,
+      "eval_sts-test_spearman_max": 0.9077088988391091,
+      "step": 60
+    },
+    {
+      "epoch": 0.32275132275132273,
+      "grad_norm": 3.0552306175231934,
+      "learning_rate": 2.2473684210526313e-05,
+      "loss": 0.21,
+      "step": 61
+    },
+    {
+      "epoch": 0.328042328042328,
+      "grad_norm": 3.1015548706054688,
+      "learning_rate": 2.2842105263157893e-05,
+      "loss": 0.21,
+      "step": 62
+    },
+    {
+      "epoch": 0.3333333333333333,
+      "grad_norm": 3.154890537261963,
+      "learning_rate": 2.321052631578947e-05,
+      "loss": 0.2157,
+      "step": 63
+    },
+    {
+      "epoch": 0.3386243386243386,
+      "grad_norm": 3.384970188140869,
+      "learning_rate": 2.357894736842105e-05,
+      "loss": 0.2138,
+      "step": 64
+    },
+    {
+      "epoch": 0.3439153439153439,
+      "grad_norm": 3.58601713180542,
+      "learning_rate": 2.394736842105263e-05,
+      "loss": 0.2403,
+      "step": 65
+    },
+    {
+      "epoch": 0.3439153439153439,
+      "eval_loss": 0.12730959057807922,
+      "eval_runtime": 115.9959,
+      "eval_samples_per_second": 26.311,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8797954361206015,
+      "eval_sts-test_pearson_dot": 0.8587071220533238,
+      "eval_sts-test_pearson_euclidean": 0.9093975785151563,
+      "eval_sts-test_pearson_manhattan": 0.9102944590146977,
+      "eval_sts-test_pearson_max": 0.9102944590146977,
+      "eval_sts-test_spearman_cosine": 0.9058099238132263,
+      "eval_sts-test_spearman_dot": 0.8649206962799048,
+      "eval_sts-test_spearman_euclidean": 0.906062455181258,
+      "eval_sts-test_spearman_manhattan": 0.9070602293052407,
+      "eval_sts-test_spearman_max": 0.9070602293052407,
+      "step": 65
+    },
+    {
+      "epoch": 0.3492063492063492,
+      "grad_norm": 3.397090435028076,
+      "learning_rate": 2.431578947368421e-05,
+      "loss": 0.2808,
+      "step": 66
+    },
+    {
+      "epoch": 0.3544973544973545,
+      "grad_norm": 2.786862373352051,
+      "learning_rate": 2.4684210526315788e-05,
+      "loss": 0.1891,
+      "step": 67
+    },
+    {
+      "epoch": 0.35978835978835977,
+      "grad_norm": 3.0845346450805664,
+      "learning_rate": 2.5052631578947368e-05,
+      "loss": 0.1991,
+      "step": 68
+    },
+    {
+      "epoch": 0.36507936507936506,
+      "grad_norm": 3.1166787147521973,
+      "learning_rate": 2.5421052631578948e-05,
+      "loss": 0.2121,
+      "step": 69
+    },
+    {
+      "epoch": 0.37037037037037035,
+      "grad_norm": 3.181523323059082,
+      "learning_rate": 2.578947368421052e-05,
+      "loss": 0.2039,
+      "step": 70
+    },
+    {
+      "epoch": 0.37037037037037035,
+      "eval_loss": 0.13109122216701508,
+      "eval_runtime": 115.9695,
+      "eval_samples_per_second": 26.317,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8802169762825238,
+      "eval_sts-test_pearson_dot": 0.8598037708245074,
+      "eval_sts-test_pearson_euclidean": 0.9094245832802006,
+      "eval_sts-test_pearson_manhattan": 0.9105289688155764,
+      "eval_sts-test_pearson_max": 0.9105289688155764,
+      "eval_sts-test_spearman_cosine": 0.906595934968305,
+      "eval_sts-test_spearman_dot": 0.8654662141562657,
+      "eval_sts-test_spearman_euclidean": 0.906532611933436,
+      "eval_sts-test_spearman_manhattan": 0.9073325407308075,
+      "eval_sts-test_spearman_max": 0.9073325407308075,
+      "step": 70
+    },
+    {
+      "epoch": 0.37566137566137564,
+      "grad_norm": 3.032498598098755,
+      "learning_rate": 2.6157894736842102e-05,
+      "loss": 0.1986,
+      "step": 71
+    },
+    {
+      "epoch": 0.38095238095238093,
+      "grad_norm": 3.486334800720215,
+      "learning_rate": 2.6526315789473682e-05,
+      "loss": 0.2925,
+      "step": 72
+    },
+    {
+      "epoch": 0.3862433862433862,
+      "grad_norm": 3.424903631210327,
+      "learning_rate": 2.689473684210526e-05,
+      "loss": 0.2527,
+      "step": 73
+    },
+    {
+      "epoch": 0.3915343915343915,
+      "grad_norm": 3.587599992752075,
+      "learning_rate": 2.726315789473684e-05,
+      "loss": 0.279,
+      "step": 74
+    },
+    {
+      "epoch": 0.3968253968253968,
+      "grad_norm": 3.3220415115356445,
+      "learning_rate": 2.763157894736842e-05,
+      "loss": 0.2419,
+      "step": 75
+    },
+    {
+      "epoch": 0.3968253968253968,
+      "eval_loss": 0.13154849410057068,
+      "eval_runtime": 115.989,
+      "eval_samples_per_second": 26.313,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.879906095889138,
+      "eval_sts-test_pearson_dot": 0.8607649105164801,
+      "eval_sts-test_pearson_euclidean": 0.9088378228597751,
+      "eval_sts-test_pearson_manhattan": 0.9102620649137054,
+      "eval_sts-test_pearson_max": 0.9102620649137054,
+      "eval_sts-test_spearman_cosine": 0.9066359425959888,
+      "eval_sts-test_spearman_dot": 0.8668292928269864,
+      "eval_sts-test_spearman_euclidean": 0.9061907122964057,
+      "eval_sts-test_spearman_manhattan": 0.9072107277972998,
+      "eval_sts-test_spearman_max": 0.9072107277972998,
+      "step": 75
+    },
+    {
+      "epoch": 0.4021164021164021,
+      "grad_norm": 2.994913101196289,
+      "learning_rate": 2.8e-05,
+      "loss": 0.2228,
+      "step": 76
+    },
+    {
+      "epoch": 0.4074074074074074,
+      "grad_norm": 3.613633632659912,
+      "learning_rate": 2.8368421052631576e-05,
+      "loss": 0.2242,
+      "step": 77
+    },
+    {
+      "epoch": 0.4126984126984127,
+      "grad_norm": 3.4389431476593018,
+      "learning_rate": 2.8736842105263157e-05,
+      "loss": 0.2737,
+      "step": 78
+    },
+    {
+      "epoch": 0.41798941798941797,
+      "grad_norm": 3.348423957824707,
+      "learning_rate": 2.9105263157894737e-05,
+      "loss": 0.2328,
+      "step": 79
+    },
+    {
+      "epoch": 0.42328042328042326,
+      "grad_norm": 3.3796963691711426,
+      "learning_rate": 2.947368421052631e-05,
+      "loss": 0.2802,
+      "step": 80
+    },
+    {
+      "epoch": 0.42328042328042326,
+      "eval_loss": 0.12620440125465393,
+      "eval_runtime": 115.88,
+      "eval_samples_per_second": 26.338,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8786083890603608,
+      "eval_sts-test_pearson_dot": 0.8607044568869332,
+      "eval_sts-test_pearson_euclidean": 0.9083064997932102,
+      "eval_sts-test_pearson_manhattan": 0.909785902193573,
+      "eval_sts-test_pearson_max": 0.909785902193573,
+      "eval_sts-test_spearman_cosine": 0.9058450088021573,
+      "eval_sts-test_spearman_dot": 0.8683225529170947,
+      "eval_sts-test_spearman_euclidean": 0.9058583894293133,
+      "eval_sts-test_spearman_manhattan": 0.9067757007853141,
+      "eval_sts-test_spearman_max": 0.9067757007853141,
+      "step": 80
+    },
+    {
+      "epoch": 0.42857142857142855,
+      "grad_norm": 2.940751314163208,
+      "learning_rate": 2.984210526315789e-05,
+      "loss": 0.2044,
+      "step": 81
+    },
+    {
+      "epoch": 0.43386243386243384,
+      "grad_norm": 2.742959499359131,
+      "learning_rate": 3.021052631578947e-05,
+      "loss": 0.1828,
+      "step": 82
+    },
+    {
+      "epoch": 0.43915343915343913,
+      "grad_norm": 3.1117758750915527,
+      "learning_rate": 3.0578947368421054e-05,
+      "loss": 0.2372,
+      "step": 83
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "grad_norm": 3.11297869682312,
+      "learning_rate": 3.094736842105263e-05,
+      "loss": 0.2241,
+      "step": 84
+    },
+    {
+      "epoch": 0.4497354497354497,
+      "grad_norm": 3.2359349727630615,
+      "learning_rate": 3.131578947368421e-05,
+      "loss": 0.2782,
+      "step": 85
+    },
+    {
+      "epoch": 0.4497354497354497,
+      "eval_loss": 0.12065601348876953,
+      "eval_runtime": 115.9874,
+      "eval_samples_per_second": 26.313,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8792266747813421,
+      "eval_sts-test_pearson_dot": 0.8635914020857025,
+      "eval_sts-test_pearson_euclidean": 0.909050966414346,
+      "eval_sts-test_pearson_manhattan": 0.9104391562008682,
+      "eval_sts-test_pearson_max": 0.9104391562008682,
+      "eval_sts-test_spearman_cosine": 0.9062812888494619,
+      "eval_sts-test_spearman_dot": 0.8713958905444104,
+      "eval_sts-test_spearman_euclidean": 0.9064880396770901,
+      "eval_sts-test_spearman_manhattan": 0.9073885693100694,
+      "eval_sts-test_spearman_max": 0.9073885693100694,
+      "step": 85
+    },
+    {
+      "epoch": 0.455026455026455,
+      "grad_norm": 3.3813822269439697,
+      "learning_rate": 3.1684210526315785e-05,
+      "loss": 0.3244,
+      "step": 86
+    },
+    {
+      "epoch": 0.4603174603174603,
+      "grad_norm": 2.657428503036499,
+      "learning_rate": 3.205263157894736e-05,
+      "loss": 0.2102,
+      "step": 87
+    },
+    {
+      "epoch": 0.4656084656084656,
+      "grad_norm": 2.7600059509277344,
+      "learning_rate": 3.2421052631578945e-05,
+      "loss": 0.2265,
+      "step": 88
+    },
+    {
+      "epoch": 0.4708994708994709,
+      "grad_norm": 3.10657000541687,
+      "learning_rate": 3.278947368421052e-05,
+      "loss": 0.2666,
+      "step": 89
+    },
+    {
+      "epoch": 0.47619047619047616,
+      "grad_norm": 3.0764777660369873,
+      "learning_rate": 3.31578947368421e-05,
+      "loss": 0.23,
+      "step": 90
+    },
+    {
+      "epoch": 0.47619047619047616,
+      "eval_loss": 0.11863212287425995,
+      "eval_runtime": 115.9856,
+      "eval_samples_per_second": 26.314,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8796178782254354,
+      "eval_sts-test_pearson_dot": 0.8632853604720685,
+      "eval_sts-test_pearson_euclidean": 0.9106195949839817,
+      "eval_sts-test_pearson_manhattan": 0.9115246839048733,
+      "eval_sts-test_pearson_max": 0.9115246839048733,
+      "eval_sts-test_spearman_cosine": 0.9078120057453548,
+      "eval_sts-test_spearman_dot": 0.8720862682538955,
+      "eval_sts-test_spearman_euclidean": 0.9083378778179652,
+      "eval_sts-test_spearman_manhattan": 0.9089643954774449,
+      "eval_sts-test_spearman_max": 0.9089643954774449,
+      "step": 90
+    },
+    {
+      "epoch": 0.48148148148148145,
+      "grad_norm": 3.020212411880493,
+      "learning_rate": 3.352631578947368e-05,
+      "loss": 0.2358,
+      "step": 91
+    },
+    {
+      "epoch": 0.48677248677248675,
+      "grad_norm": 3.419471502304077,
+      "learning_rate": 3.389473684210526e-05,
+      "loss": 0.2896,
+      "step": 92
+    },
+    {
+      "epoch": 0.49206349206349204,
+      "grad_norm": 3.062520980834961,
+      "learning_rate": 3.426315789473684e-05,
+      "loss": 0.2126,
+      "step": 93
+    },
+    {
+      "epoch": 0.4973544973544973,
+      "grad_norm": 3.3430864810943604,
+      "learning_rate": 3.463157894736842e-05,
+      "loss": 0.2669,
+      "step": 94
+    },
+    {
+      "epoch": 0.5026455026455027,
+      "grad_norm": 3.1061036586761475,
+      "learning_rate": 3.5e-05,
+      "loss": 0.2375,
+      "step": 95
+    },
+    {
+      "epoch": 0.5026455026455027,
+      "eval_loss": 0.11278577148914337,
+      "eval_runtime": 115.9371,
+      "eval_samples_per_second": 26.325,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8790475705877597,
+      "eval_sts-test_pearson_dot": 0.8631845510913274,
+      "eval_sts-test_pearson_euclidean": 0.9107267591418424,
+      "eval_sts-test_pearson_manhattan": 0.911801845501315,
+      "eval_sts-test_pearson_max": 0.911801845501315,
+      "eval_sts-test_spearman_cosine": 0.9087454723067182,
+      "eval_sts-test_spearman_dot": 0.8721337493422319,
+      "eval_sts-test_spearman_euclidean": 0.9085557269584185,
+      "eval_sts-test_spearman_manhattan": 0.9095949855017109,
+      "eval_sts-test_spearman_max": 0.9095949855017109,
+      "step": 95
+    },
+    {
+      "epoch": 0.5079365079365079,
+      "grad_norm": 2.760911703109741,
+      "learning_rate": 3.4999101435573165e-05,
+      "loss": 0.1903,
+      "step": 96
+    },
+    {
+      "epoch": 0.5132275132275133,
+      "grad_norm": 2.9554224014282227,
+      "learning_rate": 3.499640585302429e-05,
+      "loss": 0.2507,
+      "step": 97
+    },
+    {
+      "epoch": 0.5185185185185185,
+      "grad_norm": 2.9563987255096436,
+      "learning_rate": 3.4991913584534566e-05,
+      "loss": 0.1897,
+      "step": 98
+    },
+    {
+      "epoch": 0.5238095238095238,
+      "grad_norm": 3.0752642154693604,
+      "learning_rate": 3.4985625183693854e-05,
+      "loss": 0.2775,
+      "step": 99
+    },
+    {
+      "epoch": 0.5291005291005291,
+      "grad_norm": 2.8446006774902344,
+      "learning_rate": 3.4977541425432415e-05,
+      "loss": 0.2098,
+      "step": 100
+    },
+    {
+      "epoch": 0.5291005291005291,
+      "eval_loss": 0.11681391298770905,
+      "eval_runtime": 115.9468,
+      "eval_samples_per_second": 26.322,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8772591964347106,
+      "eval_sts-test_pearson_dot": 0.8585096145266702,
+      "eval_sts-test_pearson_euclidean": 0.9091316759212603,
+      "eval_sts-test_pearson_manhattan": 0.9099210072486108,
+      "eval_sts-test_pearson_max": 0.9099210072486108,
+      "eval_sts-test_spearman_cosine": 0.9064237321144708,
+      "eval_sts-test_spearman_dot": 0.8648877146002593,
+      "eval_sts-test_spearman_euclidean": 0.9063920482214057,
+      "eval_sts-test_spearman_manhattan": 0.9075959914066186,
+      "eval_sts-test_spearman_max": 0.9075959914066186,
+      "step": 100
+    },
+    {
+      "epoch": 0.5343915343915344,
+      "grad_norm": 2.7542779445648193,
+      "learning_rate": 3.496766330592549e-05,
+      "loss": 0.1628,
+      "step": 101
+    },
+    {
+      "epoch": 0.5396825396825397,
+      "grad_norm": 2.7565367221832275,
+      "learning_rate": 3.4955992042470456e-05,
+      "loss": 0.2158,
+      "step": 102
+    },
+    {
+      "epoch": 0.544973544973545,
+      "grad_norm": 2.4343347549438477,
+      "learning_rate": 3.494252907333686e-05,
+      "loss": 0.1552,
+      "step": 103
+    },
+    {
+      "epoch": 0.5502645502645502,
+      "grad_norm": 3.1311893463134766,
+      "learning_rate": 3.492727605758918e-05,
+      "loss": 0.2364,
+      "step": 104
+    },
+    {
+      "epoch": 0.5555555555555556,
+      "grad_norm": 3.080811023712158,
+      "learning_rate": 3.491023487488238e-05,
+      "loss": 0.272,
+      "step": 105
+    },
+    {
+      "epoch": 0.5555555555555556,
+      "eval_loss": 0.1177554726600647,
+      "eval_runtime": 115.9887,
+      "eval_samples_per_second": 26.313,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8775298085625183,
+      "eval_sts-test_pearson_dot": 0.8593352032357447,
+      "eval_sts-test_pearson_euclidean": 0.9089120653468052,
+      "eval_sts-test_pearson_manhattan": 0.908955609090125,
+      "eval_sts-test_pearson_max": 0.908955609090125,
+      "eval_sts-test_spearman_cosine": 0.9056424645930997,
+      "eval_sts-test_spearman_dot": 0.8631139983550111,
+      "eval_sts-test_spearman_euclidean": 0.9054400993890878,
+      "eval_sts-test_spearman_manhattan": 0.9059316024929496,
+      "eval_sts-test_spearman_max": 0.9059316024929496,
+      "step": 105
+    },
+    {
+      "epoch": 0.5608465608465608,
+      "grad_norm": 2.8664119243621826,
+      "learning_rate": 3.489140762523023e-05,
+      "loss": 0.2271,
+      "step": 106
+    },
+    {
+      "epoch": 0.5661375661375662,
+      "grad_norm": 2.7945520877838135,
+      "learning_rate": 3.4870796628746594e-05,
+      "loss": 0.2132,
+      "step": 107
+    },
+    {
+      "epoch": 0.5714285714285714,
+      "grad_norm": 2.8966565132141113,
+      "learning_rate": 3.4848404425359456e-05,
+      "loss": 0.1782,
+      "step": 108
+    },
+    {
+      "epoch": 0.5767195767195767,
+      "grad_norm": 2.6960151195526123,
+      "learning_rate": 3.4824233774497946e-05,
+      "loss": 0.1598,
+      "step": 109
+    },
+    {
+      "epoch": 0.582010582010582,
+      "grad_norm": 3.1113312244415283,
+      "learning_rate": 3.4798287654752294e-05,
+      "loss": 0.2472,
+      "step": 110
+    },
+    {
+      "epoch": 0.582010582010582,
+      "eval_loss": 0.12694795429706573,
+      "eval_runtime": 115.9427,
+      "eval_samples_per_second": 26.323,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8781162735100789,
+      "eval_sts-test_pearson_dot": 0.8634318341628767,
+      "eval_sts-test_pearson_euclidean": 0.9086688422059872,
+      "eval_sts-test_pearson_manhattan": 0.9085119625190576,
+      "eval_sts-test_pearson_max": 0.9086688422059872,
+      "eval_sts-test_spearman_cosine": 0.9050255684548193,
+      "eval_sts-test_spearman_dot": 0.8664220563483247,
+      "eval_sts-test_spearman_euclidean": 0.9042466279990407,
+      "eval_sts-test_spearman_manhattan": 0.9047099825594873,
+      "eval_sts-test_spearman_max": 0.9050255684548193,
+      "step": 110
+    },
+    {
+      "epoch": 0.5873015873015873,
+      "grad_norm": 3.0204665660858154,
+      "learning_rate": 3.477056926350673e-05,
+      "loss": 0.2041,
+      "step": 111
+    },
+    {
+      "epoch": 0.5925925925925926,
+      "grad_norm": 3.3232171535491943,
+      "learning_rate": 3.474108201654556e-05,
+      "loss": 0.2426,
+      "step": 112
+    },
+    {
+      "epoch": 0.5978835978835979,
+      "grad_norm": 3.1148412227630615,
+      "learning_rate": 3.4709829547632134e-05,
+      "loss": 0.2105,
+      "step": 113
+    },
+    {
+      "epoch": 0.6031746031746031,
+      "grad_norm": 2.886481523513794,
+      "learning_rate": 3.467681570806109e-05,
+      "loss": 0.1923,
+      "step": 114
+    },
+    {
+      "epoch": 0.6084656084656085,
+      "grad_norm": 3.2116329669952393,
+      "learning_rate": 3.464204456618378e-05,
+      "loss": 0.2271,
+      "step": 115
+    },
+    {
+      "epoch": 0.6084656084656085,
+      "eval_loss": 0.12330599874258041,
+      "eval_runtime": 115.9423,
+      "eval_samples_per_second": 26.323,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8798272374408064,
+      "eval_sts-test_pearson_dot": 0.867285673702976,
+      "eval_sts-test_pearson_euclidean": 0.909363327246087,
+      "eval_sts-test_pearson_manhattan": 0.9091246101967716,
+      "eval_sts-test_pearson_max": 0.909363327246087,
+      "eval_sts-test_spearman_cosine": 0.9060899772070137,
+      "eval_sts-test_spearman_dot": 0.8704560245526682,
+      "eval_sts-test_spearman_euclidean": 0.9054406364042247,
+      "eval_sts-test_spearman_manhattan": 0.9051852857067252,
+      "eval_sts-test_spearman_max": 0.9060899772070137,
+      "step": 115
+    },
+    {
+      "epoch": 0.6137566137566137,
+      "grad_norm": 3.385223627090454,
+      "learning_rate": 3.460552040690687e-05,
+      "loss": 0.3029,
+      "step": 116
+    },
+    {
+      "epoch": 0.6190476190476191,
+      "grad_norm": 3.150688409805298,
+      "learning_rate": 3.456724773116434e-05,
+      "loss": 0.2554,
+      "step": 117
+    },
+    {
+      "epoch": 0.6243386243386243,
+      "grad_norm": 3.0149803161621094,
+      "learning_rate": 3.452723125536281e-05,
+      "loss": 0.2182,
+      "step": 118
+    },
+    {
+      "epoch": 0.6296296296296297,
+      "grad_norm": 3.19419002532959,
+      "learning_rate": 3.448547591080033e-05,
+      "loss": 0.2852,
+      "step": 119
+    },
+    {
+      "epoch": 0.6349206349206349,
+      "grad_norm": 2.99879789352417,
+      "learning_rate": 3.444198684305871e-05,
+      "loss": 0.2285,
+      "step": 120
+    },
+    {
+      "epoch": 0.6349206349206349,
+      "eval_loss": 0.12802907824516296,
+      "eval_runtime": 115.8977,
+      "eval_samples_per_second": 26.334,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8798175425010222,
+      "eval_sts-test_pearson_dot": 0.863763595764963,
+      "eval_sts-test_pearson_euclidean": 0.9094261191010624,
+      "eval_sts-test_pearson_manhattan": 0.9090772657963855,
+      "eval_sts-test_pearson_max": 0.9094261191010624,
+      "eval_sts-test_spearman_cosine": 0.9053311300675998,
+      "eval_sts-test_spearman_dot": 0.8654653191310379,
+      "eval_sts-test_spearman_euclidean": 0.9053121555327698,
+      "eval_sts-test_spearman_manhattan": 0.9052343330892101,
+      "eval_sts-test_spearman_max": 0.9053311300675998,
+      "step": 120
+    },
+    {
+      "epoch": 0.6402116402116402,
+      "grad_norm": 3.01249361038208,
+      "learning_rate": 3.439676941136937e-05,
+      "loss": 0.218,
+      "step": 121
+    },
+    {
+      "epoch": 0.6455026455026455,
+      "grad_norm": 2.7809016704559326,
+      "learning_rate": 3.4349829187952966e-05,
+      "loss": 0.1841,
+      "step": 122
+    },
+    {
+      "epoch": 0.6507936507936508,
+      "grad_norm": 3.183194398880005,
+      "learning_rate": 3.4301171957332694e-05,
+      "loss": 0.2629,
+      "step": 123
+    },
+    {
+      "epoch": 0.656084656084656,
+      "grad_norm": 3.006030797958374,
+      "learning_rate": 3.425080371562145e-05,
+      "loss": 0.1749,
+      "step": 124
+    },
+    {
+      "epoch": 0.6613756613756614,
+      "grad_norm": 2.9949681758880615,
+      "learning_rate": 3.4198730669782925e-05,
+      "loss": 0.2417,
+      "step": 125
+    },
+    {
+      "epoch": 0.6613756613756614,
+      "eval_loss": 0.14149585366249084,
+      "eval_runtime": 115.9037,
+      "eval_samples_per_second": 26.332,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8801193496474387,
+      "eval_sts-test_pearson_dot": 0.8614489114996757,
+      "eval_sts-test_pearson_euclidean": 0.9096775446763254,
+      "eval_sts-test_pearson_manhattan": 0.909541166275706,
+      "eval_sts-test_pearson_max": 0.9096775446763254,
+      "eval_sts-test_spearman_cosine": 0.9056506540739345,
+      "eval_sts-test_spearman_dot": 0.8627393855459037,
+      "eval_sts-test_spearman_euclidean": 0.905229410450457,
+      "eval_sts-test_spearman_manhattan": 0.9051265720517796,
+      "eval_sts-test_spearman_max": 0.9056506540739345,
+      "step": 125
+    },
+    {
+      "epoch": 0.6666666666666666,
+      "grad_norm": 2.9404184818267822,
+      "learning_rate": 3.414495923686671e-05,
+      "loss": 0.2305,
+      "step": 126
+    },
+    {
+      "epoch": 0.671957671957672,
+      "grad_norm": 3.515538215637207,
+      "learning_rate": 3.408949604321751e-05,
+      "loss": 0.2841,
+      "step": 127
+    },
+    {
+      "epoch": 0.6772486772486772,
+      "grad_norm": 2.7193102836608887,
+      "learning_rate": 3.4032347923658554e-05,
+      "loss": 0.1785,
+      "step": 128
+    },
+    {
+      "epoch": 0.6825396825396826,
+      "grad_norm": 2.949784755706787,
+      "learning_rate": 3.397352192064938e-05,
+      "loss": 0.2153,
+      "step": 129
+    },
+    {
+      "epoch": 0.6878306878306878,
+      "grad_norm": 3.263798475265503,
+      "learning_rate": 3.39130252834179e-05,
+      "loss": 0.2548,
+      "step": 130
+    },
+    {
+      "epoch": 0.6878306878306878,
+      "eval_loss": 0.1412901133298874,
+      "eval_runtime": 115.8261,
+      "eval_samples_per_second": 26.35,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8817491042172992,
+      "eval_sts-test_pearson_dot": 0.8653158778803413,
+      "eval_sts-test_pearson_euclidean": 0.9109017540975314,
+      "eval_sts-test_pearson_manhattan": 0.9110420593534589,
+      "eval_sts-test_pearson_max": 0.9110420593534589,
+      "eval_sts-test_spearman_cosine": 0.9079463490320518,
+      "eval_sts-test_spearman_dot": 0.8671617499478633,
+      "eval_sts-test_spearman_euclidean": 0.9076926541312239,
+      "eval_sts-test_spearman_manhattan": 0.9074005626481224,
+      "eval_sts-test_spearman_max": 0.9079463490320518,
+      "step": 130
+    },
+    {
+      "epoch": 0.6931216931216931,
+      "grad_norm": 2.7200613021850586,
+      "learning_rate": 3.385086546706714e-05,
+      "loss": 0.2059,
+      "step": 131
+    },
+    {
+      "epoch": 0.6984126984126984,
+      "grad_norm": 2.916797399520874,
+      "learning_rate": 3.3787050131656496e-05,
+      "loss": 0.2073,
+      "step": 132
+    },
+    {
+      "epoch": 0.7037037037037037,
+      "grad_norm": 3.0047614574432373,
+      "learning_rate": 3.372158714125777e-05,
+      "loss": 0.191,
+      "step": 133
+    },
+    {
+      "epoch": 0.708994708994709,
+      "grad_norm": 2.702082633972168,
+      "learning_rate": 3.36544845629861e-05,
+      "loss": 0.1633,
+      "step": 134
+    },
+    {
+      "epoch": 0.7142857142857143,
+      "grad_norm": 3.3000283241271973,
+      "learning_rate": 3.358575066600578e-05,
+      "loss": 0.2627,
+      "step": 135
+    },
+    {
+      "epoch": 0.7142857142857143,
+      "eval_loss": 0.13328148424625397,
+      "eval_runtime": 115.7947,
+      "eval_samples_per_second": 26.357,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8814270028194036,
+      "eval_sts-test_pearson_dot": 0.8654755263554079,
+      "eval_sts-test_pearson_euclidean": 0.9100944689124159,
+      "eval_sts-test_pearson_manhattan": 0.9103746959208892,
+      "eval_sts-test_pearson_max": 0.9103746959208892,
+      "eval_sts-test_spearman_cosine": 0.9076914905984277,
+      "eval_sts-test_spearman_dot": 0.8685219197865933,
+      "eval_sts-test_spearman_euclidean": 0.9077337805404427,
+      "eval_sts-test_spearman_manhattan": 0.9075425136492559,
+      "eval_sts-test_spearman_max": 0.9077337805404427,
+      "step": 135
+    },
+    {
+      "epoch": 0.7195767195767195,
+      "grad_norm": 3.1306369304656982,
+      "learning_rate": 3.351539392051131e-05,
+      "loss": 0.2451,
+      "step": 136
+    },
+    {
+      "epoch": 0.7248677248677249,
+      "grad_norm": 2.528568983078003,
+      "learning_rate": 3.344342299668353e-05,
+      "loss": 0.1441,
+      "step": 137
+    },
+    {
+      "epoch": 0.7301587301587301,
+      "grad_norm": 2.893615484237671,
+      "learning_rate": 3.33698467636212e-05,
+      "loss": 0.2138,
+      "step": 138
+    },
+    {
+      "epoch": 0.7354497354497355,
+      "grad_norm": 3.235039472579956,
+      "learning_rate": 3.32946742882481e-05,
+      "loss": 0.2564,
+      "step": 139
+    },
+    {
+      "epoch": 0.7407407407407407,
+      "grad_norm": 2.8678057193756104,
+      "learning_rate": 3.321791483419561e-05,
+      "loss": 0.1524,
+      "step": 140
+    },
+    {
+      "epoch": 0.7407407407407407,
+      "eval_loss": 0.13225606083869934,
+      "eval_runtime": 115.8232,
+      "eval_samples_per_second": 26.351,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8781919392624582,
+      "eval_sts-test_pearson_dot": 0.8608413217193998,
+      "eval_sts-test_pearson_euclidean": 0.9068541902117857,
+      "eval_sts-test_pearson_manhattan": 0.9075012201786494,
+      "eval_sts-test_pearson_max": 0.9075012201786494,
+      "eval_sts-test_spearman_cosine": 0.9049418835960173,
+      "eval_sts-test_spearman_dot": 0.8634087749137961,
+      "eval_sts-test_spearman_euclidean": 0.9045396592586316,
+      "eval_sts-test_spearman_manhattan": 0.9048969533295804,
+      "eval_sts-test_spearman_max": 0.9049418835960173,
+      "step": 140
+    },
+    {
+      "epoch": 0.746031746031746,
+      "grad_norm": 2.5006282329559326,
+      "learning_rate": 3.313957786066119e-05,
+      "loss": 0.1786,
+      "step": 141
+    },
+    {
+      "epoch": 0.7513227513227513,
+      "grad_norm": 3.081282377243042,
+      "learning_rate": 3.3059673021242716e-05,
+      "loss": 0.2104,
+      "step": 142
+    },
+    {
+      "epoch": 0.7566137566137566,
+      "grad_norm": 3.044879198074341,
+      "learning_rate": 3.297821016274879e-05,
+      "loss": 0.2512,
+      "step": 143
+    },
+    {
+      "epoch": 0.7619047619047619,
+      "grad_norm": 2.87981915473938,
+      "learning_rate": 3.2895199323985386e-05,
+      "loss": 0.1889,
+      "step": 144
+    },
+    {
+      "epoch": 0.7671957671957672,
+      "grad_norm": 2.799271821975708,
+      "learning_rate": 3.281065073451867e-05,
+      "loss": 0.2127,
+      "step": 145
+    },
+    {
+      "epoch": 0.7671957671957672,
+      "eval_loss": 0.12909488379955292,
+      "eval_runtime": 115.7825,
+      "eval_samples_per_second": 26.36,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8735553111785674,
+      "eval_sts-test_pearson_dot": 0.8567353468688738,
+      "eval_sts-test_pearson_euclidean": 0.9031851032382168,
+      "eval_sts-test_pearson_manhattan": 0.9046152117713773,
+      "eval_sts-test_pearson_max": 0.9046152117713773,
+      "eval_sts-test_spearman_cosine": 0.9015403849689184,
+      "eval_sts-test_spearman_dot": 0.8596978213141758,
+      "eval_sts-test_spearman_euclidean": 0.9003828040905072,
+      "eval_sts-test_spearman_manhattan": 0.9019434148290095,
+      "eval_sts-test_spearman_max": 0.9019434148290095,
+      "step": 145
+    },
+    {
+      "epoch": 0.7724867724867724,
+      "grad_norm": 2.8767826557159424,
+      "learning_rate": 3.2724574813414464e-05,
+      "loss": 0.2115,
+      "step": 146
+    },
+    {
+      "epoch": 0.7777777777777778,
+      "grad_norm": 2.861135244369507,
+      "learning_rate": 3.2636982167954236e-05,
+      "loss": 0.179,
+      "step": 147
+    },
+    {
+      "epoch": 0.783068783068783,
+      "grad_norm": 2.881589651107788,
+      "learning_rate": 3.254788359232795e-05,
+      "loss": 0.2188,
+      "step": 148
+    },
+    {
+      "epoch": 0.7883597883597884,
+      "grad_norm": 2.7440359592437744,
+      "learning_rate": 3.245729006630393e-05,
+      "loss": 0.1687,
+      "step": 149
+    },
+    {
+      "epoch": 0.7936507936507936,
+      "grad_norm": 2.965336561203003,
+      "learning_rate": 3.236521275387573e-05,
+      "loss": 0.2265,
+      "step": 150
+    },
+    {
+      "epoch": 0.7936507936507936,
+      "eval_loss": 0.11447698622941971,
+      "eval_runtime": 115.8332,
+      "eval_samples_per_second": 26.348,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.874015974599682,
+      "eval_sts-test_pearson_dot": 0.8572630785995417,
+      "eval_sts-test_pearson_euclidean": 0.9037766701587291,
+      "eval_sts-test_pearson_manhattan": 0.905395978358323,
+      "eval_sts-test_pearson_max": 0.905395978358323,
+      "eval_sts-test_spearman_cosine": 0.9014261797498476,
+      "eval_sts-test_spearman_dot": 0.8600332320183043,
+      "eval_sts-test_spearman_euclidean": 0.9009870356218139,
+      "eval_sts-test_spearman_manhattan": 0.9018416504606055,
+      "eval_sts-test_spearman_max": 0.9018416504606055,
+      "step": 150
+    },
+    {
+      "epoch": 0.798941798941799,
+      "grad_norm": 2.718717098236084,
+      "learning_rate": 3.227166300188643e-05,
+      "loss": 0.182,
+      "step": 151
+    },
+    {
+      "epoch": 0.8042328042328042,
+      "grad_norm": 2.84299373626709,
+      "learning_rate": 3.2176652338630344e-05,
+      "loss": 0.1789,
+      "step": 152
+    },
+    {
+      "epoch": 0.8095238095238095,
+      "grad_norm": 2.6284101009368896,
+      "learning_rate": 3.2080192472432324e-05,
+      "loss": 0.225,
+      "step": 153
+    },
+    {
+      "epoch": 0.8148148148148148,
+      "grad_norm": 3.4302072525024414,
+      "learning_rate": 3.198229529020499e-05,
+      "loss": 0.3015,
+      "step": 154
+    },
+    {
+      "epoch": 0.8201058201058201,
+      "grad_norm": 2.625488758087158,
+      "learning_rate": 3.188297285598381e-05,
+      "loss": 0.1656,
+      "step": 155
+    },
+    {
+      "epoch": 0.8201058201058201,
+      "eval_loss": 0.10088476538658142,
+      "eval_runtime": 115.7342,
+      "eval_samples_per_second": 26.371,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8778735515080995,
+      "eval_sts-test_pearson_dot": 0.8592183485478384,
+      "eval_sts-test_pearson_euclidean": 0.9077661642183765,
+      "eval_sts-test_pearson_manhattan": 0.9087255199218803,
+      "eval_sts-test_pearson_max": 0.9087255199218803,
+      "eval_sts-test_spearman_cosine": 0.9044667147025635,
+      "eval_sts-test_spearman_dot": 0.8617503379178922,
+      "eval_sts-test_spearman_euclidean": 0.9049174941585592,
+      "eval_sts-test_spearman_manhattan": 0.9053862188703726,
+      "eval_sts-test_spearman_max": 0.9053862188703726,
+      "step": 155
+    },
+    {
+      "epoch": 0.8253968253968254,
+      "grad_norm": 2.843688726425171,
+      "learning_rate": 3.178223740944051e-05,
+      "loss": 0.2321,
+      "step": 156
+    },
+    {
+      "epoch": 0.8306878306878307,
+      "grad_norm": 2.9048900604248047,
+      "learning_rate": 3.168010136437467e-05,
+      "loss": 0.2514,
+      "step": 157
+    },
+    {
+      "epoch": 0.8359788359788359,
+      "grad_norm": 2.917659282684326,
+      "learning_rate": 3.1576577307184056e-05,
+      "loss": 0.2348,
+      "step": 158
+    },
+    {
+      "epoch": 0.8412698412698413,
+      "grad_norm": 2.9148318767547607,
+      "learning_rate": 3.1471677995313453e-05,
+      "loss": 0.2003,
+      "step": 159
+    },
+    {
+      "epoch": 0.8465608465608465,
+      "grad_norm": 2.801607131958008,
+      "learning_rate": 3.136541635568265e-05,
+      "loss": 0.2487,
+      "step": 160
+    },
+    {
+      "epoch": 0.8465608465608465,
+      "eval_loss": 0.09830425679683685,
+      "eval_runtime": 116.0864,
+      "eval_samples_per_second": 26.291,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8783875552844034,
+      "eval_sts-test_pearson_dot": 0.85858622106775,
+      "eval_sts-test_pearson_euclidean": 0.9083724543344728,
+      "eval_sts-test_pearson_manhattan": 0.9090357666425739,
+      "eval_sts-test_pearson_max": 0.9090357666425739,
+      "eval_sts-test_spearman_cosine": 0.9056794738862703,
+      "eval_sts-test_spearman_dot": 0.8620214410594015,
+      "eval_sts-test_spearman_euclidean": 0.906014079067694,
+      "eval_sts-test_spearman_manhattan": 0.9057808802445833,
+      "eval_sts-test_spearman_max": 0.906014079067694,
+      "step": 160
+    },
+    {
+      "epoch": 0.8518518518518519,
+      "grad_norm": 2.867021322250366,
+      "learning_rate": 3.125780548309338e-05,
+      "loss": 0.1674,
+      "step": 161
+    },
+    {
+      "epoch": 0.8571428571428571,
+      "grad_norm": 2.5584158897399902,
+      "learning_rate": 3.114885863861564e-05,
+      "loss": 0.1968,
+      "step": 162
+    },
+    {
+      "epoch": 0.8624338624338624,
+      "grad_norm": 2.951962471008301,
+      "learning_rate": 3.10385892479535e-05,
+      "loss": 0.2237,
+      "step": 163
+    },
+    {
+      "epoch": 0.8677248677248677,
+      "grad_norm": 3.1288952827453613,
+      "learning_rate": 3.0927010899790636e-05,
+      "loss": 0.2636,
+      "step": 164
+    },
+    {
+      "epoch": 0.873015873015873,
+      "grad_norm": 2.768833875656128,
+      "learning_rate": 3.081413734411578e-05,
+      "loss": 0.1858,
+      "step": 165
+    },
+    {
+      "epoch": 0.873015873015873,
+      "eval_loss": 0.10154492408037186,
+      "eval_runtime": 116.0209,
+      "eval_samples_per_second": 26.306,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8755774094051667,
+      "eval_sts-test_pearson_dot": 0.8587982677526502,
+      "eval_sts-test_pearson_euclidean": 0.9058079612884306,
+      "eval_sts-test_pearson_manhattan": 0.9067791571071063,
+      "eval_sts-test_pearson_max": 0.9067791571071063,
+      "eval_sts-test_spearman_cosine": 0.9043415901757131,
+      "eval_sts-test_spearman_dot": 0.8629658716798055,
+      "eval_sts-test_spearman_euclidean": 0.904651761168417,
+      "eval_sts-test_spearman_manhattan": 0.9046320258621435,
+      "eval_sts-test_spearman_max": 0.904651761168417,
+      "step": 165
+    },
+    {
+      "epoch": 0.8783068783068783,
+      "grad_norm": 2.7155349254608154,
+      "learning_rate": 3.069998249052827e-05,
+      "loss": 0.198,
+      "step": 166
+    },
+    {
+      "epoch": 0.8835978835978836,
+      "grad_norm": 2.870551586151123,
+      "learning_rate": 3.058456040652396e-05,
+      "loss": 0.2011,
+      "step": 167
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 3.004664182662964,
+      "learning_rate": 3.0467885315761637e-05,
+      "loss": 0.2887,
+      "step": 168
+    },
+    {
+      "epoch": 0.8941798941798942,
+      "grad_norm": 2.8544065952301025,
+      "learning_rate": 3.0349971596310263e-05,
+      "loss": 0.2014,
+      "step": 169
+    },
+    {
+      "epoch": 0.8994708994708994,
+      "grad_norm": 2.9045169353485107,
+      "learning_rate": 3.023083377887709e-05,
+      "loss": 0.2122,
+      "step": 170
+    },
+    {
+      "epoch": 0.8994708994708994,
+      "eval_loss": 0.10024923086166382,
+      "eval_runtime": 116.0623,
+      "eval_samples_per_second": 26.296,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8730559262674971,
+      "eval_sts-test_pearson_dot": 0.8574715059734537,
+      "eval_sts-test_pearson_euclidean": 0.903257721171822,
+      "eval_sts-test_pearson_manhattan": 0.9045779112782556,
+      "eval_sts-test_pearson_max": 0.9045779112782556,
+      "eval_sts-test_spearman_cosine": 0.902514709431932,
+      "eval_sts-test_spearman_dot": 0.8610834993718991,
+      "eval_sts-test_spearman_euclidean": 0.9021505684179902,
+      "eval_sts-test_spearman_manhattan": 0.9024851288481524,
+      "eval_sts-test_spearman_max": 0.902514709431932,
+      "step": 170
+    },
+    {
+      "epoch": 0.9047619047619048,
+      "grad_norm": 2.916578531265259,
+      "learning_rate": 3.0110486545017035e-05,
+      "loss": 0.2537,
+      "step": 171
+    },
+    {
+      "epoch": 0.91005291005291,
+      "grad_norm": 2.8374223709106445,
+      "learning_rate": 2.998894472532346e-05,
+      "loss": 0.2104,
+      "step": 172
+    },
+    {
+      "epoch": 0.9153439153439153,
+      "grad_norm": 2.1109468936920166,
+      "learning_rate": 2.986622329760054e-05,
+      "loss": 0.1291,
+      "step": 173
+    },
+    {
+      "epoch": 0.9206349206349206,
+      "grad_norm": 2.7503883838653564,
+      "learning_rate": 2.9742337385017572e-05,
+      "loss": 0.1794,
+      "step": 174
+    },
+    {
+      "epoch": 0.9259259259259259,
+      "grad_norm": 3.0585219860076904,
+      "learning_rate": 2.9617302254245285e-05,
+      "loss": 0.2569,
+      "step": 175
+    },
+    {
+      "epoch": 0.9259259259259259,
+      "eval_loss": 0.09481428563594818,
+      "eval_runtime": 116.1072,
+      "eval_samples_per_second": 26.286,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8728242331324298,
+      "eval_sts-test_pearson_dot": 0.8567724380659857,
+      "eval_sts-test_pearson_euclidean": 0.9033940339950373,
+      "eval_sts-test_pearson_manhattan": 0.904631801418599,
+      "eval_sts-test_pearson_max": 0.904631801418599,
+      "eval_sts-test_spearman_cosine": 0.9026927747010085,
+      "eval_sts-test_spearman_dot": 0.859896919676106,
+      "eval_sts-test_spearman_euclidean": 0.9024781476513751,
+      "eval_sts-test_spearman_manhattan": 0.9026701753140057,
+      "eval_sts-test_spearman_max": 0.9026927747010085,
+      "step": 175
+    },
+    {
+      "epoch": 0.9312169312169312,
+      "grad_norm": 2.9827516078948975,
+      "learning_rate": 2.9491133313574487e-05,
+      "loss": 0.26,
+      "step": 176
+    },
+    {
+      "epoch": 0.9365079365079365,
+      "grad_norm": 2.5629608631134033,
+      "learning_rate": 2.9363846111017323e-05,
+      "loss": 0.1665,
+      "step": 177
+    },
+    {
+      "epoch": 0.9417989417989417,
+      "grad_norm": 2.6901652812957764,
+      "learning_rate": 2.923545633239126e-05,
+      "loss": 0.2012,
+      "step": 178
+    },
+    {
+      "epoch": 0.9470899470899471,
+      "grad_norm": 2.883577823638916,
+      "learning_rate": 2.9105979799386034e-05,
+      "loss": 0.2171,
+      "step": 179
+    },
+    {
+      "epoch": 0.9523809523809523,
+      "grad_norm": 2.6022555828094482,
+      "learning_rate": 2.8975432467614006e-05,
+      "loss": 0.1554,
+      "step": 180
+    },
+    {
+      "epoch": 0.9523809523809523,
+      "eval_loss": 0.08742579072713852,
+      "eval_runtime": 116.1318,
+      "eval_samples_per_second": 26.28,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8764850626067987,
+      "eval_sts-test_pearson_dot": 0.8594281799824091,
+      "eval_sts-test_pearson_euclidean": 0.906763287295881,
+      "eval_sts-test_pearson_manhattan": 0.9077192925473501,
+      "eval_sts-test_pearson_max": 0.9077192925473501,
+      "eval_sts-test_spearman_cosine": 0.9055436538079475,
+      "eval_sts-test_spearman_dot": 0.8633867125419303,
+      "eval_sts-test_spearman_euclidean": 0.9054898627917551,
+      "eval_sts-test_spearman_manhattan": 0.9062817811133373,
+      "eval_sts-test_spearman_max": 0.9062817811133373,
+      "step": 180
+    },
+    {
+      "epoch": 0.9576719576719577,
+      "grad_norm": 3.0509660243988037,
+      "learning_rate": 2.884383042464385e-05,
+      "loss": 0.2177,
+      "step": 181
+    },
+    {
+      "epoch": 0.9629629629629629,
+      "grad_norm": 2.853330373764038,
+      "learning_rate": 2.8711189888018114e-05,
+      "loss": 0.1919,
+      "step": 182
+    },
+    {
+      "epoch": 0.9682539682539683,
+      "grad_norm": 2.4763054847717285,
+      "learning_rate": 2.8577527203254652e-05,
+      "loss": 0.1686,
+      "step": 183
+    },
+    {
+      "epoch": 0.9735449735449735,
+      "grad_norm": 3.1543056964874268,
+      "learning_rate": 2.8442858841832375e-05,
+      "loss": 0.2277,
+      "step": 184
+    },
+    {
+      "epoch": 0.9788359788359788,
+      "grad_norm": 2.843949317932129,
+      "learning_rate": 2.830720139916143e-05,
+      "loss": 0.2169,
+      "step": 185
+    },
+    {
+      "epoch": 0.9788359788359788,
+      "eval_loss": 0.08474321663379669,
+      "eval_runtime": 116.0513,
+      "eval_samples_per_second": 26.299,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8822620169500903,
+      "eval_sts-test_pearson_dot": 0.8647722217227573,
+      "eval_sts-test_pearson_euclidean": 0.9113549300983457,
+      "eval_sts-test_pearson_manhattan": 0.9116172627252676,
+      "eval_sts-test_pearson_max": 0.9116172627252676,
+      "eval_sts-test_spearman_cosine": 0.9088902426373191,
+      "eval_sts-test_spearman_dot": 0.8667347781629278,
+      "eval_sts-test_spearman_euclidean": 0.9089773285919869,
+      "eval_sts-test_spearman_manhattan": 0.9093677385963655,
+      "eval_sts-test_spearman_max": 0.9093677385963655,
+      "step": 185
+    },
+    {
+      "epoch": 0.9841269841269841,
+      "grad_norm": 2.996716022491455,
+      "learning_rate": 2.817057159253813e-05,
+      "loss": 0.1991,
+      "step": 186
+    },
+    {
+      "epoch": 0.9894179894179894,
+      "grad_norm": 3.041013479232788,
+      "learning_rate": 2.803298625908483e-05,
+      "loss": 0.2373,
+      "step": 187
+    },
+    {
+      "epoch": 0.9947089947089947,
+      "grad_norm": 3.310819625854492,
+      "learning_rate": 2.789446235367508e-05,
+      "loss": 0.1636,
+      "step": 188
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 6.908136356287287e-07,
+      "learning_rate": 2.775501694684424e-05,
+      "loss": 0.0,
+      "step": 189
+    },
+    {
+      "epoch": 1.0052910052910053,
+      "grad_norm": 0.0,
+      "learning_rate": 2.7614667222685848e-05,
+      "loss": 0.0,
+      "step": 190
+    },
+    {
+      "epoch": 1.0052910052910053,
+      "eval_loss": 0.08459059149026871,
+      "eval_runtime": 116.0926,
+      "eval_samples_per_second": 26.289,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8845727889510453,
+      "eval_sts-test_pearson_dot": 0.8665715810539829,
+      "eval_sts-test_pearson_euclidean": 0.9131377744832304,
+      "eval_sts-test_pearson_manhattan": 0.912934056355235,
+      "eval_sts-test_pearson_max": 0.9131377744832304,
+      "eval_sts-test_spearman_cosine": 0.9102842891809226,
+      "eval_sts-test_spearman_dot": 0.8672268182819262,
+      "eval_sts-test_spearman_euclidean": 0.9097165746789112,
+      "eval_sts-test_spearman_manhattan": 0.9097905485139913,
+      "eval_sts-test_spearman_max": 0.9102842891809226,
+      "step": 190
+    },
+    {
+      "epoch": 1.0105820105820107,
+      "grad_norm": 0.0,
+      "learning_rate": 2.7473430476733998e-05,
+      "loss": 0.0,
+      "step": 191
+    },
+    {
+      "epoch": 1.0158730158730158,
+      "grad_norm": 0.0,
+      "learning_rate": 2.7331324113831995e-05,
+      "loss": 0.0,
+      "step": 192
+    },
+    {
+      "epoch": 1.0211640211640212,
+      "grad_norm": 0.0,
+      "learning_rate": 2.718836564598748e-05,
+      "loss": 0.0,
+      "step": 193
+    },
+    {
+      "epoch": 1.0264550264550265,
+      "grad_norm": 0.0,
+      "learning_rate": 2.7044572690214462e-05,
+      "loss": 0.0,
+      "step": 194
+    },
+    {
+      "epoch": 1.0317460317460319,
+      "grad_norm": 0.0,
+      "learning_rate": 2.6899962966362294e-05,
+      "loss": 0.0,
+      "step": 195
+    },
+    {
+      "epoch": 1.0317460317460319,
+      "eval_loss": 0.08586417883634567,
+      "eval_runtime": 116.4251,
+      "eval_samples_per_second": 26.214,
+      "eval_steps_per_second": 0.206,
+      "eval_sts-test_pearson_cosine": 0.8849634882429999,
+      "eval_sts-test_pearson_dot": 0.8655804570166644,
+      "eval_sts-test_pearson_euclidean": 0.9136398497040992,
+      "eval_sts-test_pearson_manhattan": 0.9131045787916059,
+      "eval_sts-test_pearson_max": 0.9136398497040992,
+      "eval_sts-test_spearman_cosine": 0.90978889271732,
+      "eval_sts-test_spearman_dot": 0.8655711111129673,
+      "eval_sts-test_spearman_euclidean": 0.9096662742611075,
+      "eval_sts-test_spearman_manhattan": 0.9096195539442148,
+      "eval_sts-test_spearman_max": 0.90978889271732,
+      "step": 195
+    },
+    {
+      "epoch": 1.037037037037037,
+      "grad_norm": 0.0,
+      "learning_rate": 2.6754554294932045e-05,
+      "loss": 0.0,
+      "step": 196
+    },
+    {
+      "epoch": 1.0423280423280423,
+      "grad_norm": 0.0,
+      "learning_rate": 2.660836459488044e-05,
+      "loss": 0.0,
+      "step": 197
+    },
+    {
+      "epoch": 1.0476190476190477,
+      "grad_norm": 0.0,
+      "learning_rate": 2.6461411881411704e-05,
+      "loss": 0.0,
+      "step": 198
+    },
+    {
+      "epoch": 1.052910052910053,
+      "grad_norm": 0.0,
+      "learning_rate": 2.631371426375747e-05,
+      "loss": 0.0,
+      "step": 199
+    },
+    {
+      "epoch": 1.0582010582010581,
+      "grad_norm": 0.0,
+      "learning_rate": 2.6165289942945193e-05,
+      "loss": 0.0,
+      "step": 200
+    },
+    {
+      "epoch": 1.0582010582010581,
+      "eval_loss": 0.08703548461198807,
+      "eval_runtime": 116.1136,
+      "eval_samples_per_second": 26.285,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8848263754836961,
+      "eval_sts-test_pearson_dot": 0.8643466881998988,
+      "eval_sts-test_pearson_euclidean": 0.9136955795688737,
+      "eval_sts-test_pearson_manhattan": 0.9129865343740124,
+      "eval_sts-test_pearson_max": 0.9136955795688737,
+      "eval_sts-test_spearman_cosine": 0.9101009432630021,
+      "eval_sts-test_spearman_dot": 0.8633947677689806,
+      "eval_sts-test_spearman_euclidean": 0.9096585322928866,
+      "eval_sts-test_spearman_manhattan": 0.9092393472274335,
+      "eval_sts-test_spearman_max": 0.9101009432630021,
+      "step": 200
+    },
+    {
+      "epoch": 1.0634920634920635,
+      "grad_norm": 0.0,
+      "learning_rate": 2.6016157209555184e-05,
+      "loss": 0.0,
+      "step": 201
+    },
+    {
+      "epoch": 1.0687830687830688,
+      "grad_norm": 0.0,
+      "learning_rate": 2.5866334441466633e-05,
+      "loss": 0.0,
+      "step": 202
+    },
+    {
+      "epoch": 1.0740740740740742,
+      "grad_norm": 0.0,
+      "learning_rate": 2.571584010159287e-05,
+      "loss": 0.0,
+      "step": 203
+    },
+    {
+      "epoch": 1.0793650793650793,
+      "grad_norm": 0.0,
+      "learning_rate": 2.556469273560616e-05,
+      "loss": 0.0,
+      "step": 204
+    },
+    {
+      "epoch": 1.0846560846560847,
+      "grad_norm": 0.0,
+      "learning_rate": 2.5412910969652282e-05,
+      "loss": 0.0,
+      "step": 205
+    },
+    {
+      "epoch": 1.0846560846560847,
+      "eval_loss": 0.08789028227329254,
+      "eval_runtime": 116.1451,
+      "eval_samples_per_second": 26.277,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8846570187287662,
+      "eval_sts-test_pearson_dot": 0.86341314971663,
+      "eval_sts-test_pearson_euclidean": 0.9136708583259868,
+      "eval_sts-test_pearson_manhattan": 0.9128654159614628,
+      "eval_sts-test_pearson_max": 0.9136708583259868,
+      "eval_sts-test_spearman_cosine": 0.9099815468976099,
+      "eval_sts-test_spearman_dot": 0.8617482793598683,
+      "eval_sts-test_spearman_euclidean": 0.9096507008221433,
+      "eval_sts-test_spearman_manhattan": 0.9089463159678426,
+      "eval_sts-test_spearman_max": 0.9099815468976099,
+      "step": 205
+    },
+    {
+      "epoch": 1.08994708994709,
+      "grad_norm": 0.0,
+      "learning_rate": 2.526051350805519e-05,
+      "loss": 0.0,
+      "step": 206
+    },
+    {
+      "epoch": 1.0952380952380953,
+      "grad_norm": 0.0,
+      "learning_rate": 2.510751913101206e-05,
+      "loss": 0.0,
+      "step": 207
+    },
+    {
+      "epoch": 1.1005291005291005,
+      "grad_norm": 0.0,
+      "learning_rate": 2.4953946692278975e-05,
+      "loss": 0.0,
+      "step": 208
+    },
+    {
+      "epoch": 1.1058201058201058,
+      "grad_norm": 0.0,
+      "learning_rate": 2.4799815116847518e-05,
+      "loss": 0.0,
+      "step": 209
+    },
+    {
+      "epoch": 1.1111111111111112,
+      "grad_norm": 0.0,
+      "learning_rate": 2.4645143398612655e-05,
+      "loss": 0.0,
+      "step": 210
+    },
+    {
+      "epoch": 1.1111111111111112,
+      "eval_loss": 0.08843279629945755,
+      "eval_runtime": 116.1092,
+      "eval_samples_per_second": 26.286,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8844939921994073,
+      "eval_sts-test_pearson_dot": 0.8627844351363139,
+      "eval_sts-test_pearson_euclidean": 0.9136074396788918,
+      "eval_sts-test_pearson_manhattan": 0.9127440134857732,
+      "eval_sts-test_pearson_max": 0.9136074396788918,
+      "eval_sts-test_spearman_cosine": 0.9097246746572233,
+      "eval_sts-test_spearman_dot": 0.8608252398424094,
+      "eval_sts-test_spearman_euclidean": 0.9096155263306896,
+      "eval_sts-test_spearman_manhattan": 0.9087806467981717,
+      "eval_sts-test_spearman_max": 0.9097246746572233,
+      "step": 210
+    },
+    {
+      "epoch": 1.1164021164021165,
+      "grad_norm": 0.0,
+      "learning_rate": 2.4489950598032055e-05,
+      "loss": 0.0,
+      "step": 211
+    },
+    {
+      "epoch": 1.1216931216931216,
+      "grad_norm": 0.0,
+      "learning_rate": 2.4334255839777238e-05,
+      "loss": 0.0,
+      "step": 212
+    },
+    {
+      "epoch": 1.126984126984127,
+      "grad_norm": 0.0,
+      "learning_rate": 2.4178078310376848e-05,
+      "loss": 0.0,
+      "step": 213
+    },
+    {
+      "epoch": 1.1322751322751323,
+      "grad_norm": 0.0,
+      "learning_rate": 2.4021437255852188e-05,
+      "loss": 0.0,
+      "step": 214
+    },
+    {
+      "epoch": 1.1375661375661377,
+      "grad_norm": 0.0,
+      "learning_rate": 2.3864351979345593e-05,
+      "loss": 0.0,
+      "step": 215
+    },
+    {
+      "epoch": 1.1375661375661377,
+      "eval_loss": 0.08878105878829956,
+      "eval_runtime": 116.1463,
+      "eval_samples_per_second": 26.277,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8843928712626077,
+      "eval_sts-test_pearson_dot": 0.8624145472479743,
+      "eval_sts-test_pearson_euclidean": 0.9135729422975322,
+      "eval_sts-test_pearson_manhattan": 0.9126843456721141,
+      "eval_sts-test_pearson_max": 0.9135729422975322,
+      "eval_sts-test_spearman_cosine": 0.9095934192075623,
+      "eval_sts-test_spearman_dot": 0.8603290378561014,
+      "eval_sts-test_spearman_euclidean": 0.9094919233467266,
+      "eval_sts-test_spearman_manhattan": 0.9086817912617581,
+      "eval_sts-test_spearman_max": 0.9095934192075623,
+      "step": 215
+    },
+    {
+      "epoch": 1.1428571428571428,
+      "grad_norm": 0.0,
+      "learning_rate": 2.3706841838741606e-05,
+      "loss": 0.0,
+      "step": 216
+    },
+    {
+      "epoch": 1.1481481481481481,
+      "grad_norm": 0.0,
+      "learning_rate": 2.354892624428147e-05,
+      "loss": 0.0,
+      "step": 217
+    },
+    {
+      "epoch": 1.1534391534391535,
+      "grad_norm": 0.0,
+      "learning_rate": 2.3390624656171225e-05,
+      "loss": 0.0,
+      "step": 218
+    },
+    {
+      "epoch": 1.1587301587301588,
+      "grad_norm": 0.0,
+      "learning_rate": 2.3231956582183556e-05,
+      "loss": 0.0,
+      "step": 219
+    },
+    {
+      "epoch": 1.164021164021164,
+      "grad_norm": 0.0,
+      "learning_rate": 2.307294157525381e-05,
+      "loss": 0.0,
+      "step": 220
+    },
+    {
+      "epoch": 1.164021164021164,
+      "eval_loss": 0.08897024393081665,
+      "eval_runtime": 116.1016,
+      "eval_samples_per_second": 26.287,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8843343977370239,
+      "eval_sts-test_pearson_dot": 0.8621730154689187,
+      "eval_sts-test_pearson_euclidean": 0.9135453435855337,
+      "eval_sts-test_pearson_manhattan": 0.9126389692350095,
+      "eval_sts-test_pearson_max": 0.9135453435855337,
+      "eval_sts-test_spearman_cosine": 0.909527187340703,
+      "eval_sts-test_spearman_dot": 0.8599858851837522,
+      "eval_sts-test_spearman_euclidean": 0.9095309016953984,
+      "eval_sts-test_spearman_manhattan": 0.908583562243004,
+      "eval_sts-test_spearman_max": 0.9095309016953984,
+      "step": 220
+    },
+    {
+      "epoch": 1.1693121693121693,
+      "grad_norm": 0.0,
+      "learning_rate": 2.2913599231070477e-05,
+      "loss": 0.0,
+      "step": 221
+    },
+    {
+      "epoch": 1.1746031746031746,
+      "grad_norm": 0.0,
+      "learning_rate": 2.275394918566037e-05,
+      "loss": 0.0,
+      "step": 222
+    },
+    {
+      "epoch": 1.17989417989418,
+      "grad_norm": 0.0,
+      "learning_rate": 2.2594011112968846e-05,
+      "loss": 0.0,
+      "step": 223
+    },
+    {
+      "epoch": 1.1851851851851851,
+      "grad_norm": 0.0,
+      "learning_rate": 2.2433804722435332e-05,
+      "loss": 0.0,
+      "step": 224
+    },
+    {
+      "epoch": 1.1904761904761905,
+      "grad_norm": 0.0,
+      "learning_rate": 2.2273349756564566e-05,
+      "loss": 0.0,
+      "step": 225
+    },
+    {
+      "epoch": 1.1904761904761905,
+      "eval_loss": 0.08909132331609726,
+      "eval_runtime": 116.0808,
+      "eval_samples_per_second": 26.292,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8842900264709203,
+      "eval_sts-test_pearson_dot": 0.8620349462693992,
+      "eval_sts-test_pearson_euclidean": 0.9135303294926935,
+      "eval_sts-test_pearson_manhattan": 0.9126134584086265,
+      "eval_sts-test_pearson_max": 0.9135303294926935,
+      "eval_sts-test_spearman_cosine": 0.9095139409673312,
+      "eval_sts-test_spearman_dot": 0.859645014825734,
+      "eval_sts-test_spearman_euclidean": 0.9095683137499219,
+      "eval_sts-test_spearman_manhattan": 0.9085770285588407,
+      "eval_sts-test_spearman_max": 0.9095683137499219,
+      "step": 225
+    },
+    {
+      "epoch": 1.1957671957671958,
+      "grad_norm": 0.0,
+      "learning_rate": 2.2112665988493586e-05,
+      "loss": 0.0,
+      "step": 226
+    },
+    {
+      "epoch": 1.2010582010582012,
+      "grad_norm": 0.0,
+      "learning_rate": 2.195177321955516e-05,
+      "loss": 0.0,
+      "step": 227
+    },
+    {
+      "epoch": 1.2063492063492063,
+      "grad_norm": 0.0,
+      "learning_rate": 2.1790691276837557e-05,
+      "loss": 0.0,
+      "step": 228
+    },
+    {
+      "epoch": 1.2116402116402116,
+      "grad_norm": 0.0,
+      "learning_rate": 2.1629440010741255e-05,
+      "loss": 0.0,
+      "step": 229
+    },
+    {
+      "epoch": 1.216931216931217,
+      "grad_norm": 0.0,
+      "learning_rate": 2.1468039292532757e-05,
+      "loss": 0.0,
+      "step": 230
+    },
+    {
+      "epoch": 1.216931216931217,
+      "eval_loss": 0.08917967975139618,
+      "eval_runtime": 116.6826,
+      "eval_samples_per_second": 26.156,
+      "eval_steps_per_second": 0.206,
+      "eval_sts-test_pearson_cosine": 0.8842734972575569,
+      "eval_sts-test_pearson_dot": 0.8619555764136059,
+      "eval_sts-test_pearson_euclidean": 0.9135282273111254,
+      "eval_sts-test_pearson_manhattan": 0.9126076347207293,
+      "eval_sts-test_pearson_max": 0.9135282273111254,
+      "eval_sts-test_spearman_cosine": 0.9094565698502274,
+      "eval_sts-test_spearman_dot": 0.8594810462039959,
+      "eval_sts-test_spearman_euclidean": 0.9095245022650195,
+      "eval_sts-test_spearman_manhattan": 0.9085593070593297,
+      "eval_sts-test_spearman_max": 0.9095245022650195,
+      "step": 230
+    },
+    {
+      "epoch": 1.2222222222222223,
+      "grad_norm": 0.0,
+      "learning_rate": 2.1306509011895796e-05,
+      "loss": 0.0,
+      "step": 231
+    },
+    {
+      "epoch": 1.2275132275132274,
+      "grad_norm": 0.0,
+      "learning_rate": 2.1144869074480268e-05,
+      "loss": 0.0,
+      "step": 232
+    },
+    {
+      "epoch": 1.2328042328042328,
+      "grad_norm": 0.0,
+      "learning_rate": 2.0983139399449316e-05,
+      "loss": 0.0,
+      "step": 233
+    },
+    {
+      "epoch": 1.2380952380952381,
+      "grad_norm": 0.0,
+      "learning_rate": 2.0821339917024577e-05,
+      "loss": 0.0,
+      "step": 234
+    },
+    {
+      "epoch": 1.2433862433862435,
+      "grad_norm": 0.0,
+      "learning_rate": 2.0659490566030143e-05,
+      "loss": 0.0,
+      "step": 235
+    },
+    {
+      "epoch": 1.2433862433862435,
+      "eval_loss": 0.08920004963874817,
+      "eval_runtime": 115.9646,
+      "eval_samples_per_second": 26.318,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8842621342313581,
+      "eval_sts-test_pearson_dot": 0.8619184229683311,
+      "eval_sts-test_pearson_euclidean": 0.9135188430562959,
+      "eval_sts-test_pearson_manhattan": 0.9125953164801344,
+      "eval_sts-test_pearson_max": 0.9135188430562959,
+      "eval_sts-test_spearman_cosine": 0.909387339648855,
+      "eval_sts-test_spearman_dot": 0.859487445634375,
+      "eval_sts-test_spearman_euclidean": 0.9094381770817955,
+      "eval_sts-test_spearman_manhattan": 0.908572956194054,
+      "eval_sts-test_spearman_max": 0.9094381770817955,
+      "step": 235
+    },
+    {
+      "epoch": 1.2486772486772486,
+      "grad_norm": 0.0,
+      "learning_rate": 2.0497611291435535e-05,
+      "loss": 0.0,
+      "step": 236
+    },
+    {
+      "epoch": 1.253968253968254,
+      "grad_norm": 0.0,
+      "learning_rate": 2.0335722041897796e-05,
+      "loss": 0.0,
+      "step": 237
+    },
+    {
+      "epoch": 1.2592592592592593,
+      "grad_norm": 0.0,
+      "learning_rate": 2.0173842767303188e-05,
+      "loss": 0.0,
+      "step": 238
+    },
+    {
+      "epoch": 1.2645502645502646,
+      "grad_norm": 0.0,
+      "learning_rate": 2.001199341630876e-05,
+      "loss": 0.0,
+      "step": 239
+    },
+    {
+      "epoch": 1.2698412698412698,
+      "grad_norm": 0.0,
+      "learning_rate": 1.985019393388401e-05,
+      "loss": 0.0,
+      "step": 240
+    },
+    {
+      "epoch": 1.2698412698412698,
+      "eval_loss": 0.08922849595546722,
+      "eval_runtime": 115.9117,
+      "eval_samples_per_second": 26.33,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8842502053427308,
+      "eval_sts-test_pearson_dot": 0.8618892751278742,
+      "eval_sts-test_pearson_euclidean": 0.9135112504275256,
+      "eval_sts-test_pearson_manhattan": 0.9125869322184504,
+      "eval_sts-test_pearson_max": 0.9135112504275256,
+      "eval_sts-test_spearman_cosine": 0.9093211972845183,
+      "eval_sts-test_spearman_dot": 0.8593632161327525,
+      "eval_sts-test_spearman_euclidean": 0.9094434129793783,
+      "eval_sts-test_spearman_manhattan": 0.9085768048025339,
+      "eval_sts-test_spearman_max": 0.9094434129793783,
+      "step": 240
+    },
+    {
+      "epoch": 1.2751322751322751,
+      "grad_norm": 0.0,
+      "learning_rate": 1.9688464258853056e-05,
+      "loss": 0.0,
+      "step": 241
+    },
+    {
+      "epoch": 1.2804232804232805,
+      "grad_norm": 0.0,
+      "learning_rate": 1.9526824321437538e-05,
+      "loss": 0.0,
+      "step": 242
+    },
+    {
+      "epoch": 1.2857142857142858,
+      "grad_norm": 0.0,
+      "learning_rate": 1.936529404080057e-05,
+      "loss": 0.0,
+      "step": 243
+    },
+    {
+      "epoch": 1.291005291005291,
+      "grad_norm": 0.0,
+      "learning_rate": 1.9203893322592076e-05,
+      "loss": 0.0,
+      "step": 244
+    },
+    {
+      "epoch": 1.2962962962962963,
+      "grad_norm": 0.0,
+      "learning_rate": 1.9042642056495777e-05,
+      "loss": 0.0,
+      "step": 245
+    },
+    {
+      "epoch": 1.2962962962962963,
+      "eval_loss": 0.08923891186714172,
+      "eval_runtime": 115.9755,
+      "eval_samples_per_second": 26.316,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8842533482731549,
+      "eval_sts-test_pearson_dot": 0.8618807194255759,
+      "eval_sts-test_pearson_euclidean": 0.9135135401435963,
+      "eval_sts-test_pearson_manhattan": 0.9125845361663534,
+      "eval_sts-test_pearson_max": 0.9135135401435963,
+      "eval_sts-test_spearman_cosine": 0.9093556110045284,
+      "eval_sts-test_spearman_dot": 0.859378968576762,
+      "eval_sts-test_spearman_euclidean": 0.90941034179721,
+      "eval_sts-test_spearman_manhattan": 0.9084560658992998,
+      "eval_sts-test_spearman_max": 0.90941034179721,
+      "step": 245
+    },
+    {
+      "epoch": 1.3015873015873016,
+      "grad_norm": 0.0,
+      "learning_rate": 1.888156011377817e-05,
+      "loss": 0.0,
+      "step": 246
+    },
+    {
+      "epoch": 1.306878306878307,
+      "grad_norm": 0.0,
+      "learning_rate": 1.872066734483974e-05,
+      "loss": 0.0,
+      "step": 247
+    },
+    {
+      "epoch": 1.312169312169312,
+      "grad_norm": 0.0,
+      "learning_rate": 1.8559983576768764e-05,
+      "loss": 0.0,
+      "step": 248
+    },
+    {
+      "epoch": 1.3174603174603174,
+      "grad_norm": 0.0,
+      "learning_rate": 1.8399528610897996e-05,
+      "loss": 0.0,
+      "step": 249
+    },
+    {
+      "epoch": 1.3227513227513228,
+      "grad_norm": 0.0,
+      "learning_rate": 1.8239322220364488e-05,
+      "loss": 0.0,
+      "step": 250
+    },
+    {
+      "epoch": 1.3227513227513228,
+      "eval_loss": 0.08924794942140579,
+      "eval_runtime": 115.997,
+      "eval_samples_per_second": 26.311,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8842595476555168,
+      "eval_sts-test_pearson_dot": 0.8618754786761054,
+      "eval_sts-test_pearson_euclidean": 0.9135258665657471,
+      "eval_sts-test_pearson_manhattan": 0.9125957797983937,
+      "eval_sts-test_pearson_max": 0.9135258665657471,
+      "eval_sts-test_spearman_cosine": 0.9093802689495549,
+      "eval_sts-test_spearman_dot": 0.8592881235161376,
+      "eval_sts-test_spearman_euclidean": 0.9094869986610407,
+      "eval_sts-test_spearman_manhattan": 0.9085887086380641,
+      "eval_sts-test_spearman_max": 0.9094869986610407,
+      "step": 250
+    },
+    {
+      "epoch": 1.3280423280423281,
+      "grad_norm": 0.0,
+      "learning_rate": 1.807938414767296e-05,
+      "loss": 0.0,
+      "step": 251
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "grad_norm": 0.0,
+      "learning_rate": 1.7919734102262857e-05,
+      "loss": 0.0,
+      "step": 252
+    },
+    {
+      "epoch": 1.3386243386243386,
+      "grad_norm": 0.0,
+      "learning_rate": 1.776039175807952e-05,
+      "loss": 0.0,
+      "step": 253
+    },
+    {
+      "epoch": 1.343915343915344,
+      "grad_norm": 0.0,
+      "learning_rate": 1.760137675114977e-05,
+      "loss": 0.0,
+      "step": 254
+    },
+    {
+      "epoch": 1.3492063492063493,
+      "grad_norm": 0.0,
+      "learning_rate": 1.7442708677162103e-05,
+      "loss": 0.0,
+      "step": 255
+    },
+    {
+      "epoch": 1.3492063492063493,
+      "eval_loss": 0.08925878256559372,
+      "eval_runtime": 115.9211,
+      "eval_samples_per_second": 26.328,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8842448870820279,
+      "eval_sts-test_pearson_dot": 0.8618573092539901,
+      "eval_sts-test_pearson_euclidean": 0.9135123358042139,
+      "eval_sts-test_pearson_manhattan": 0.9125818082589108,
+      "eval_sts-test_pearson_max": 0.9135123358042139,
+      "eval_sts-test_spearman_cosine": 0.9093363679621299,
+      "eval_sts-test_spearman_dot": 0.8593020411584302,
+      "eval_sts-test_spearman_euclidean": 0.9094210373486826,
+      "eval_sts-test_spearman_manhattan": 0.908524356324183,
+      "eval_sts-test_spearman_max": 0.9094210373486826,
+      "step": 255
+    },
+    {
+      "epoch": 1.3544973544973544,
+      "grad_norm": 0.0,
+      "learning_rate": 1.728440708905186e-05,
+      "loss": 0.0,
+      "step": 256
+    },
+    {
+      "epoch": 1.3597883597883598,
+      "grad_norm": 0.0,
+      "learning_rate": 1.7126491494591728e-05,
+      "loss": 0.0,
+      "step": 257
+    },
+    {
+      "epoch": 1.3650793650793651,
+      "grad_norm": 0.0,
+      "learning_rate": 1.6968981353987735e-05,
+      "loss": 0.0,
+      "step": 258
+    },
+    {
+      "epoch": 1.3703703703703705,
+      "grad_norm": 0.0,
+      "learning_rate": 1.6811896077481136e-05,
+      "loss": 0.0,
+      "step": 259
+    },
+    {
+      "epoch": 1.3756613756613756,
+      "grad_norm": 0.0,
+      "learning_rate": 1.6655255022956487e-05,
+      "loss": 0.0,
+      "step": 260
+    },
+    {
+      "epoch": 1.3756613756613756,
+      "eval_loss": 0.08925431966781616,
+      "eval_runtime": 116.1132,
+      "eval_samples_per_second": 26.285,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.884252122217321,
+      "eval_sts-test_pearson_dot": 0.8618580027072977,
+      "eval_sts-test_pearson_euclidean": 0.9135163745669427,
+      "eval_sts-test_pearson_manhattan": 0.9125849003738024,
+      "eval_sts-test_pearson_max": 0.9135163745669427,
+      "eval_sts-test_spearman_cosine": 0.9093181541987438,
+      "eval_sts-test_spearman_dot": 0.859250219197739,
+      "eval_sts-test_spearman_euclidean": 0.9094325384228603,
+      "eval_sts-test_spearman_manhattan": 0.908558188277795,
+      "eval_sts-test_spearman_max": 0.9094325384228603,
+      "step": 260
+    },
+    {
+      "epoch": 1.380952380952381,
+      "grad_norm": 0.0,
+      "learning_rate": 1.6499077493556093e-05,
+      "loss": 0.0,
+      "step": 261
+    },
+    {
+      "epoch": 1.3862433862433863,
+      "grad_norm": 0.0,
+      "learning_rate": 1.634338273530128e-05,
+      "loss": 0.0,
+      "step": 262
+    },
+    {
+      "epoch": 1.3915343915343916,
+      "grad_norm": 0.0,
+      "learning_rate": 1.6188189934720676e-05,
+      "loss": 0.0,
+      "step": 263
+    },
+    {
+      "epoch": 1.3968253968253967,
+      "grad_norm": 0.0,
+      "learning_rate": 1.6033518216485813e-05,
+      "loss": 0.0,
+      "step": 264
+    },
+    {
+      "epoch": 1.402116402116402,
+      "grad_norm": 0.0,
+      "learning_rate": 1.5879386641054356e-05,
+      "loss": 0.0,
+      "step": 265
+    },
+    {
+      "epoch": 1.402116402116402,
+      "eval_loss": 0.08925169706344604,
+      "eval_runtime": 115.9768,
+      "eval_samples_per_second": 26.316,
+      "eval_steps_per_second": 0.207,
+      "eval_sts-test_pearson_cosine": 0.8842448913937776,
+      "eval_sts-test_pearson_dot": 0.8618584457426092,
+      "eval_sts-test_pearson_euclidean": 0.9135135876297823,
+      "eval_sts-test_pearson_manhattan": 0.9125853937525384,
+      "eval_sts-test_pearson_max": 0.9135135876297823,
+      "eval_sts-test_spearman_cosine": 0.9093488983153196,
+      "eval_sts-test_spearman_dot": 0.8592465943455663,
+      "eval_sts-test_spearman_euclidean": 0.9094515129576901,
+      "eval_sts-test_spearman_manhattan": 0.9085058740532286,
+      "eval_sts-test_spearman_max": 0.9094515129576901,
+      "step": 265
+    },
+    {
+      "epoch": 1.4074074074074074,
+      "grad_norm": 0.0,
+      "learning_rate": 1.572581420232127e-05,
+      "loss": 0.0,
+      "step": 266
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 378,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 38,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 640,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-266/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2968b16dc322a2bed394ae83a32300ba89607cbee354667b1d1918a368bd3a9a
+size 5752