Training in progress, step 246, checkpoint

checkpoint-246/1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

checkpoint-246/README.md

@@ -0,0 +1,770 @@
+---
+base_model: bobox/DeBERTa-small-ST-v1-test-step3
+datasets: []
+language: []
+library_name: sentence-transformers
+metrics:
+- pearson_cosine
+- spearman_cosine
+- pearson_manhattan
+- spearman_manhattan
+- pearson_euclidean
+- spearman_euclidean
+- pearson_dot
+- spearman_dot
+- pearson_max
+- spearman_max
+pipeline_tag: sentence-similarity
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:260034
+- loss:CachedGISTEmbedLoss
+widget:
+- source_sentence: who used to present one man and his dog
+  sentences:
+  - One Man and His Dog One Man and His Dog is a BBC television series in the United
+    Kingdom featuring sheepdog trials, originally presented by Phil Drabble, with
+    commentary by Eric Halsall and, later, by Ray Ollerenshaw. It was first aired
+    on 17 February 1976 and continues today (since 2013) as a special annual edition
+    of Countryfile. In 1994, Robin Page replaced Drabble as the main presenter. Gus
+    Dermody took over as commentator until 2012.
+  - 'animal adjectives [was: ratto, Ratte, raton] - Google Groups animal adjectives
+    [was: ratto, Ratte, raton] Showing 1-9 of 9 messages While trying find the pronunciation
+    of the word "munger", I encountered the nearby word    murine [MYOO-ryn] = relating
+    to mice or rats    [from Latin _murinus_, which derives from _mus_,    mouse,
+    whose genetive form is _muris_] So if you need an adjective to refer to lab rodents
+    like _ratto_ or _mausu_, "murine" it is. (I would never have discovered this except
+    in an alphabetically arranged dictionary.) There are a lot of animal adjectives
+    of this type, such as ovine (sheep), equine (horse), bovine (bull, cow, calf),
+    aquiline (eagle), murine (rats and mice).   But what is needed is a way to lookup
+    an animal and find what the proper adjective is.  For example, is there an adjective
+    form for "goat"? for "seal"? for "elephant"? for "whale"? for "walrus"? By the
+    way, I never did find out how "munger" is pronounced; the answer is not found
+    in'
+  - A boat is docked and filled with bicycles next to a grassy area on a body of water.
+- source_sentence: There were 29 Muslims fatalities in the Cave of the Patriarchs
+    massacre .
+  sentences:
+  - 'Urban Dictionary: Dog and Bone Dog and Bone Cockney rhyming slang for phone -
+    the telephone. ''''Pick up the dog and bone now'''' by Brendan April 05, 2003
+    Create a mug The Urban Dictionary Mug One side has the word, one side has the
+    definition. Microwave and dishwasher safe. Lotsa space for your liquids. Buy the
+    t-shirt The Urban Dictionary T-Shirt Smooth, soft, slim fit American Apparel shirt.
+    Custom printed. 100% fine jersey cotton, except for heather grey (90% cotton).
+    ^Same as above except can be shortened further to ''Dogs'' or just ''dog'' Get
+    on the dogs and give us a bell when your ready. by Phaze October 14, 2004'
+  - RAF College Cranwell - Local Area Information RAF College Cranwell Local Area
+    Information Local Area Information RAF College Cranwell is situated in the North
+    Kesteven District Council area in the heart of rural Lincolnshire, 5 miles from
+    Sleaford and 14 miles from the City of Lincoln, surrounded by bustling market
+    towns, picturesque villages and landscapes steeped in aviation history. Lincolnshire
+    is currently home to several operational RAF airfields and was a key location
+    during WWII for bomber stations. Museums, memorials, former airfields, heritage
+    and visitor centres bear witness to the bravery of the men and women of this time.
+    The ancient City of Lincoln dates back at least to Roman times and boasts a spectacular
+    Cathedral and Castle area, whilst Sleaford is the home to the National Centre
+    for Craft & Design. Please click on the Logo to access website
+  - 29 Muslims were killed and more than 100 others wounded . [   Settlers remember
+    gunman Goldstein ; Hebron riots continue ] .
+- source_sentence: What requires energy for growth?
+  sentences:
+  - "an organism requires energy for growth. Fish Fish are the ultimate aquatic organism.\
+    \ \n a fish require energy for growth"
+  - In August , after the end of the war in June 1902 , Higgins Southampton left the
+    `` SSBavarian '' and returned to Cape Town the following month .
+  - Rhinestone Cowboy "Rhinestone Cowboy" is a song written by Larry Weiss and most
+    famously recorded by American country music singer Glen Campbell. The song enjoyed
+    huge popularity with both country and pop audiences when it was released in 1975.
+- source_sentence: Burning wood is used to produce what type of energy?
+  sentences:
+  - Shawnee Trails Council was formed from the merger of the Four Rivers Council and
+    the Audubon Council .
+  - A Mercedes parked next to a parking meter on a street.
+  - "burning wood is used to produce heat. Heat is kinetic energy. \n burning wood\
+    \ is used to produce kinetic energy."
+- source_sentence: As of March , more than 413,000 cases have been confirmed in more
+    than 190 countries with more than 107,000 recoveries .
+  sentences:
+  - As of 24 March , more than 414,000 cases of COVID-19 have been reported in more
+    than 190 countries and territories , resulting in more than 18,500 deaths and
+    more than 108,000 recoveries .
+  - 'Pope Francis makes first visit as head of state to Italy\''s president - YouTube
+    Pope Francis makes first visit as head of state to Italy\''s president Want to
+    watch this again later? Sign in to add this video to a playlist. Need to report
+    the video? Sign in to report inappropriate content. The interactive transcript
+    could not be loaded. Loading... Rating is available when the video has been rented.
+    This feature is not available right now. Please try again later. Published on
+    Nov 14, 2013 Pope Francis stepped out of the Vatican, several hundred feet into
+    the heart of Rome, to meet with Italian President Giorgio Napolitano, and the
+    country\''s Council of Ministers. . --------------------- Suscríbete al canal:
+    http://smarturl.it/RomeReports Visita nuestra web: http://www.romereports.com/
+    ROME REPORTS, www.romereports.com, is an independent international TV News Agency
+    based in Rome covering the activity of the Pope, the life of the Vatican and current
+    social, cultural and religious debates. Reporting on the Catholic Church requires
+    proximity to the source, in-depth knowledge of the Institution, and a high standard
+    of creativity and technical excellence. As few broadcasters have a permanent correspondent
+    in Rome, ROME REPORTS is geared to inform the public and meet the needs of television
+    broadcasting companies around the world through daily news packages, weekly newsprograms
+    and documentaries. ---------------------'
+  - German shepherds and retrievers are commonly used, but the Belgian Malinois has
+    proven to be one of the most outstanding working dogs used in military service.
+    Around 85 percent of military working dogs are purchased in Germany or the Netherlands,
+    where they have been breeding dogs for military purposes for hundreds of years.
+    In addition, the Air Force Security Forces Center, Army Veterinary Corps and the
+    341st Training Squadron combine efforts to raise their own dogs; nearly 15 percent
+    of all military working dogs are now bred here.
+model-index:
+- name: SentenceTransformer based on bobox/DeBERTa-small-ST-v1-test-step3
+  results:
+  - task:
+      type: semantic-similarity
+      name: Semantic Similarity
+    dataset:
+      name: sts test
+      type: sts-test
+    metrics:
+    - type: pearson_cosine
+      value: 0.8822954324429473
+      name: Pearson Cosine
+    - type: spearman_cosine
+      value: 0.9087387596175093
+      name: Spearman Cosine
+    - type: pearson_manhattan
+      value: 0.912099304650421
+      name: Pearson Manhattan
+    - type: spearman_manhattan
+      value: 0.9095372563745162
+      name: Spearman Manhattan
+    - type: pearson_euclidean
+      value: 0.9122889538029727
+      name: Pearson Euclidean
+    - type: spearman_euclidean
+      value: 0.9097842833373965
+      name: Spearman Euclidean
+    - type: pearson_dot
+      value: 0.8642863367586305
+      name: Pearson Dot
+    - type: spearman_dot
+      value: 0.8704923178256567
+      name: Spearman Dot
+    - type: pearson_max
+      value: 0.9122889538029727
+      name: Pearson Max
+    - type: spearman_max
+      value: 0.9097842833373965
+      name: Spearman Max
+---
+
+# SentenceTransformer based on bobox/DeBERTa-small-ST-v1-test-step3
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [bobox/DeBERTa-small-ST-v1-test-step3](https://huggingface.co/bobox/DeBERTa-small-ST-v1-test-step3) on the bobox/enhanced_nli-50_k dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [bobox/DeBERTa-small-ST-v1-test-step3](https://huggingface.co/bobox/DeBERTa-small-ST-v1-test-step3) <!-- at revision df9aaa75fe0c2791e5ed35ff33de1689d9a5f5ff -->
+- **Maximum Sequence Length:** 512 tokens
+- **Output Dimensionality:** 768 tokens
+- **Similarity Function:** Cosine Similarity
+- **Training Dataset:**
+    - bobox/enhanced_nli-50_k
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: DebertaV2Model 
+  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("bobox/DeBERTa-small-ST-v1-test-UnifiedDatasets-Ft2-checkpoints-tmp")
+# Run inference
+sentences = [
+    'As of March , more than 413,000 cases have been confirmed in more than 190 countries with more than 107,000 recoveries .',
+    'As of 24 March , more than 414,000 cases of COVID-19 have been reported in more than 190 countries and territories , resulting in more than 18,500 deaths and more than 108,000 recoveries .',
+    'German shepherds and retrievers are commonly used, but the Belgian Malinois has proven to be one of the most outstanding working dogs used in military service. Around 85 percent of military working dogs are purchased in Germany or the Netherlands, where they have been breeding dogs for military purposes for hundreds of years. In addition, the Air Force Security Forces Center, Army Veterinary Corps and the 341st Training Squadron combine efforts to raise their own dogs; nearly 15 percent of all military working dogs are now bred here.',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 768]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Semantic Similarity
+* Dataset: `sts-test`
+* Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.EmbeddingSimilarityEvaluator)
+
+| Metric              | Value      |
+|:--------------------|:-----------|
+| pearson_cosine      | 0.8823     |
+| **spearman_cosine** | **0.9087** |
+| pearson_manhattan   | 0.9121     |
+| spearman_manhattan  | 0.9095     |
+| pearson_euclidean   | 0.9123     |
+| spearman_euclidean  | 0.9098     |
+| pearson_dot         | 0.8643     |
+| spearman_dot        | 0.8705     |
+| pearson_max         | 0.9123     |
+| spearman_max        | 0.9098     |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### bobox/enhanced_nli-50_k
+
+* Dataset: bobox/enhanced_nli-50_k
+* Size: 260,034 training samples
+* Columns: <code>sentence1</code> and <code>sentence2</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | sentence1                                                                          | sentence2                                                                          |
+  |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
+  | type    | string                                                                             | string                                                                             |
+  | details | <ul><li>min: 4 tokens</li><li>mean: 39.12 tokens</li><li>max: 344 tokens</li></ul> | <ul><li>min: 2 tokens</li><li>mean: 60.17 tokens</li><li>max: 442 tokens</li></ul> |
+* Samples:
+  | sentence1                                                                                                                                                                                                                                                                                                                                                                                                                                                             | sentence2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+  |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>Temple Meads Railway Station is in which English city?</code>                                                                                                                                                                                                                                                                                                                                                                                                   | <code>Bristol Temple Meads station roof to be replaced - BBC News BBC News Bristol Temple Meads station roof to be replaced 17 October 2013 Image caption Bristol Temple Meads was designed by Isambard Kingdom Brunel Image caption It will cost Network Rail £15m to replace the station's roof Image caption A pact has been signed to redevelop the station over the next 25 years The entire roof on Bristol Temple Meads railway station is to be replaced. Network Rail says it has secured £15m to carry out maintenance of the roof and install new lighting and cables. The announcement was made as a pact was signed to "significantly transform" the station over the next 25 years. Network Rail, Bristol City Council, the West of England Local Enterprise Partnership, Homes and Communities Agency and English Heritage are supporting the plan. Each has signed the 25-year memorandum of understanding to redevelop the station. Patrick Hallgate, of Network Rail Western, said: "Our plans for Bristol will see the railway significantly transformed by the end of the decade, with more seats, better connections and more frequent services." The railway station was designed by Isambard Kingdom Brunel and opened in 1840.</code> |
+  | <code>Where do most of the digestion reactions occur?</code>                                                                                                                                                                                                                                                                                                                                                                                                          | <code>Most of the digestion reactions occur in the small intestine.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
+  | <code>Sacko, 22, joined Sporting from French top-flight side Bordeaux in 2014, but has so far been limited to playing for the Portuguese club's B team.<br>The former France Under-20 player joined Ligue 2 side Sochaux on loan in February and scored twice in 14 games.<br>He is Leeds' third signing of the transfer window, following the arrivals of Marcus Antonsson and Kyle Bartley.<br>Find all the latest football transfers on our dedicated page.</code> | <code>Leeds have signed Sporting Lisbon forward Hadi Sacko on a season-long loan with a view to a permanent deal.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
+* Loss: [<code>CachedGISTEmbedLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedgistembedloss) with these parameters:
+  ```json
+  {'guide': SentenceTransformer(
+    (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
+    (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+    (2): Normalize()
+  ), 'temperature': 0.025}
+  ```
+
+### Evaluation Dataset
+
+#### bobox/enhanced_nli-50_k
+
+* Dataset: bobox/enhanced_nli-50_k
+* Size: 1,506 evaluation samples
+* Columns: <code>sentence1</code> and <code>sentence2</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | sentence1                                                                          | sentence2                                                                         |
+  |:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
+  | type    | string                                                                             | string                                                                            |
+  | details | <ul><li>min: 3 tokens</li><li>mean: 31.16 tokens</li><li>max: 340 tokens</li></ul> | <ul><li>min: 2 tokens</li><li>mean: 62.3 tokens</li><li>max: 455 tokens</li></ul> |
+* Samples:
+  | sentence1                                                                                                                                     | sentence2                                                                                                                                                                                 |
+  |:----------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>Interestingly, snakes use their forked tongues to smell.</code>                                                                         | <code>Snakes use their tongue to smell things.</code>                                                                                                                                     |
+  | <code>A voltaic cell generates an electric current through a reaction known as a(n) spontaneous redox.</code>                                 | <code>A voltaic cell uses what type of reaction to generate an electric current</code>                                                                                                    |
+  | <code>As of March 22 , there were more than 321,000 cases with over 13,600 deaths and more than 96,000 recoveries reported worldwide .</code> | <code>As of 22 March , more than 321,000 cases of COVID-19 have been reported in over 180 countries and territories , resulting in more than 13,600 deaths and 96,000 recoveries .</code> |
+* Loss: [<code>CachedGISTEmbedLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedgistembedloss) with these parameters:
+  ```json
+  {'guide': SentenceTransformer(
+    (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
+    (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+    (2): Normalize()
+  ), 'temperature': 0.025}
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 320
+- `per_device_eval_batch_size`: 128
+- `learning_rate`: 2e-05
+- `weight_decay`: 0.0001
+- `num_train_epochs`: 1
+- `lr_scheduler_type`: cosine_with_restarts
+- `lr_scheduler_kwargs`: {'num_cycles': 3}
+- `warmup_ratio`: 0.25
+- `save_safetensors`: False
+- `fp16`: True
+- `push_to_hub`: True
+- `hub_model_id`: bobox/DeBERTa-small-ST-v1-test-UnifiedDatasets-Ft2-checkpoints-tmp
+- `hub_strategy`: all_checkpoints
+- `batch_sampler`: no_duplicates
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 320
+- `per_device_eval_batch_size`: 128
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 2e-05
+- `weight_decay`: 0.0001
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 1
+- `max_steps`: -1
+- `lr_scheduler_type`: cosine_with_restarts
+- `lr_scheduler_kwargs`: {'num_cycles': 3}
+- `warmup_ratio`: 0.25
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: False
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: True
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: True
+- `resume_from_checkpoint`: None
+- `hub_model_id`: bobox/DeBERTa-small-ST-v1-test-UnifiedDatasets-Ft2-checkpoints-tmp
+- `hub_strategy`: all_checkpoints
+- `hub_private_repo`: False
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `eval_use_gather_object`: False
+- `batch_sampler`: no_duplicates
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+<details><summary>Click to expand</summary>
+
+| Epoch  | Step | Training Loss | loss   | sts-test_spearman_cosine |
+|:------:|:----:|:-------------:|:------:|:------------------------:|
+| 0.0012 | 1    | 0.3208        | -      | -                        |
+| 0.0025 | 2    | 0.1703        | -      | -                        |
+| 0.0037 | 3    | 0.3362        | -      | -                        |
+| 0.0049 | 4    | 0.3346        | -      | -                        |
+| 0.0062 | 5    | 0.2484        | -      | -                        |
+| 0.0074 | 6    | 0.2249        | -      | -                        |
+| 0.0086 | 7    | 0.2724        | -      | -                        |
+| 0.0098 | 8    | 0.251         | -      | -                        |
+| 0.0111 | 9    | 0.2413        | -      | -                        |
+| 0.0123 | 10   | 0.382         | -      | -                        |
+| 0.0135 | 11   | 0.2695        | -      | -                        |
+| 0.0148 | 12   | 0.2392        | -      | -                        |
+| 0.0160 | 13   | 0.3603        | -      | -                        |
+| 0.0172 | 14   | 0.3282        | -      | -                        |
+| 0.0185 | 15   | 0.2878        | -      | -                        |
+| 0.0197 | 16   | 0.3046        | -      | -                        |
+| 0.0209 | 17   | 0.3946        | -      | -                        |
+| 0.0221 | 18   | 0.2038        | -      | -                        |
+| 0.0234 | 19   | 0.3542        | -      | -                        |
+| 0.0246 | 20   | 0.2369        | -      | -                        |
+| 0.0258 | 21   | 0.1967        | 0.1451 | 0.9081                   |
+| 0.0271 | 22   | 0.2368        | -      | -                        |
+| 0.0283 | 23   | 0.263         | -      | -                        |
+| 0.0295 | 24   | 0.3595        | -      | -                        |
+| 0.0308 | 25   | 0.3073        | -      | -                        |
+| 0.0320 | 26   | 0.2232        | -      | -                        |
+| 0.0332 | 27   | 0.1822        | -      | -                        |
+| 0.0344 | 28   | 0.251         | -      | -                        |
+| 0.0357 | 29   | 0.2677        | -      | -                        |
+| 0.0369 | 30   | 0.3252        | -      | -                        |
+| 0.0381 | 31   | 0.2058        | -      | -                        |
+| 0.0394 | 32   | 0.3083        | -      | -                        |
+| 0.0406 | 33   | 0.2109        | -      | -                        |
+| 0.0418 | 34   | 0.2751        | -      | -                        |
+| 0.0431 | 35   | 0.2269        | -      | -                        |
+| 0.0443 | 36   | 0.2333        | -      | -                        |
+| 0.0455 | 37   | 0.2747        | -      | -                        |
+| 0.0467 | 38   | 0.1285        | -      | -                        |
+| 0.0480 | 39   | 0.3659        | -      | -                        |
+| 0.0492 | 40   | 0.3991        | -      | -                        |
+| 0.0504 | 41   | 0.2647        | -      | -                        |
+| 0.0517 | 42   | 0.3627        | 0.1373 | 0.9084                   |
+| 0.0529 | 43   | 0.2026        | -      | -                        |
+| 0.0541 | 44   | 0.1923        | -      | -                        |
+| 0.0554 | 45   | 0.2369        | -      | -                        |
+| 0.0566 | 46   | 0.2268        | -      | -                        |
+| 0.0578 | 47   | 0.2975        | -      | -                        |
+| 0.0590 | 48   | 0.1922        | -      | -                        |
+| 0.0603 | 49   | 0.1906        | -      | -                        |
+| 0.0615 | 50   | 0.2379        | -      | -                        |
+| 0.0627 | 51   | 0.3796        | -      | -                        |
+| 0.0640 | 52   | 0.1821        | -      | -                        |
+| 0.0652 | 53   | 0.1257        | -      | -                        |
+| 0.0664 | 54   | 0.2368        | -      | -                        |
+| 0.0677 | 55   | 0.294         | -      | -                        |
+| 0.0689 | 56   | 0.2594        | -      | -                        |
+| 0.0701 | 57   | 0.2972        | -      | -                        |
+| 0.0713 | 58   | 0.2297        | -      | -                        |
+| 0.0726 | 59   | 0.1487        | -      | -                        |
+| 0.0738 | 60   | 0.182         | -      | -                        |
+| 0.0750 | 61   | 0.2516        | -      | -                        |
+| 0.0763 | 62   | 0.2809        | -      | -                        |
+| 0.0775 | 63   | 0.1371        | 0.1308 | 0.9068                   |
+| 0.0787 | 64   | 0.2149        | -      | -                        |
+| 0.0800 | 65   | 0.1806        | -      | -                        |
+| 0.0812 | 66   | 0.1458        | -      | -                        |
+| 0.0824 | 67   | 0.249         | -      | -                        |
+| 0.0836 | 68   | 0.2787        | -      | -                        |
+| 0.0849 | 69   | 0.288         | -      | -                        |
+| 0.0861 | 70   | 0.1461        | -      | -                        |
+| 0.0873 | 71   | 0.2304        | -      | -                        |
+| 0.0886 | 72   | 0.3505        | -      | -                        |
+| 0.0898 | 73   | 0.2227        | -      | -                        |
+| 0.0910 | 74   | 0.1746        | -      | -                        |
+| 0.0923 | 75   | 0.1484        | -      | -                        |
+| 0.0935 | 76   | 0.1346        | -      | -                        |
+| 0.0947 | 77   | 0.2112        | -      | -                        |
+| 0.0959 | 78   | 0.3138        | -      | -                        |
+| 0.0972 | 79   | 0.2675        | -      | -                        |
+| 0.0984 | 80   | 0.2849        | -      | -                        |
+| 0.0996 | 81   | 0.1719        | -      | -                        |
+| 0.1009 | 82   | 0.2749        | -      | -                        |
+| 0.1021 | 83   | 0.3097        | -      | -                        |
+| 0.1033 | 84   | 0.2068        | 0.1260 | 0.9045                   |
+| 0.1046 | 85   | 0.22          | -      | -                        |
+| 0.1058 | 86   | 0.2977        | -      | -                        |
+| 0.1070 | 87   | 0.209         | -      | -                        |
+| 0.1082 | 88   | 0.2215        | -      | -                        |
+| 0.1095 | 89   | 0.1948        | -      | -                        |
+| 0.1107 | 90   | 0.2084        | -      | -                        |
+| 0.1119 | 91   | 0.1823        | -      | -                        |
+| 0.1132 | 92   | 0.255         | -      | -                        |
+| 0.1144 | 93   | 0.2675        | -      | -                        |
+| 0.1156 | 94   | 0.18          | -      | -                        |
+| 0.1169 | 95   | 0.2891        | -      | -                        |
+| 0.1181 | 96   | 0.253         | -      | -                        |
+| 0.1193 | 97   | 0.3481        | -      | -                        |
+| 0.1205 | 98   | 0.1688        | -      | -                        |
+| 0.1218 | 99   | 0.1808        | -      | -                        |
+| 0.1230 | 100  | 0.2821        | -      | -                        |
+| 0.1242 | 101  | 0.1856        | -      | -                        |
+| 0.1255 | 102  | 0.1441        | -      | -                        |
+| 0.1267 | 103  | 0.226         | -      | -                        |
+| 0.1279 | 104  | 0.1662        | -      | -                        |
+| 0.1292 | 105  | 0.2043        | 0.1187 | 0.9051                   |
+| 0.1304 | 106  | 0.3907        | -      | -                        |
+| 0.1316 | 107  | 0.1332        | -      | -                        |
+| 0.1328 | 108  | 0.2243        | -      | -                        |
+| 0.1341 | 109  | 0.162         | -      | -                        |
+| 0.1353 | 110  | 0.1481        | -      | -                        |
+| 0.1365 | 111  | 0.2163        | -      | -                        |
+| 0.1378 | 112  | 0.24          | -      | -                        |
+| 0.1390 | 113  | 0.1406        | -      | -                        |
+| 0.1402 | 114  | 0.1522        | -      | -                        |
+| 0.1415 | 115  | 0.2593        | -      | -                        |
+| 0.1427 | 116  | 0.2426        | -      | -                        |
+| 0.1439 | 117  | 0.1781        | -      | -                        |
+| 0.1451 | 118  | 0.264         | -      | -                        |
+| 0.1464 | 119  | 0.1944        | -      | -                        |
+| 0.1476 | 120  | 0.1341        | -      | -                        |
+| 0.1488 | 121  | 0.155         | -      | -                        |
+| 0.1501 | 122  | 0.2052        | -      | -                        |
+| 0.1513 | 123  | 0.2023        | -      | -                        |
+| 0.1525 | 124  | 0.1519        | -      | -                        |
+| 0.1538 | 125  | 0.2118        | -      | -                        |
+| 0.1550 | 126  | 0.2489        | 0.1147 | 0.9058                   |
+| 0.1562 | 127  | 0.1988        | -      | -                        |
+| 0.1574 | 128  | 0.1541        | -      | -                        |
+| 0.1587 | 129  | 0.1819        | -      | -                        |
+| 0.1599 | 130  | 0.1582        | -      | -                        |
+| 0.1611 | 131  | 0.2866        | -      | -                        |
+| 0.1624 | 132  | 0.2766        | -      | -                        |
+| 0.1636 | 133  | 0.1299        | -      | -                        |
+| 0.1648 | 134  | 0.2558        | -      | -                        |
+| 0.1661 | 135  | 0.1687        | -      | -                        |
+| 0.1673 | 136  | 0.173         | -      | -                        |
+| 0.1685 | 137  | 0.2276        | -      | -                        |
+| 0.1697 | 138  | 0.2174        | -      | -                        |
+| 0.1710 | 139  | 0.2666        | -      | -                        |
+| 0.1722 | 140  | 0.1524        | -      | -                        |
+| 0.1734 | 141  | 0.1179        | -      | -                        |
+| 0.1747 | 142  | 0.2475        | -      | -                        |
+| 0.1759 | 143  | 0.2662        | -      | -                        |
+| 0.1771 | 144  | 0.1596        | -      | -                        |
+| 0.1784 | 145  | 0.2331        | -      | -                        |
+| 0.1796 | 146  | 0.2905        | -      | -                        |
+| 0.1808 | 147  | 0.1342        | 0.1088 | 0.9051                   |
+| 0.1820 | 148  | 0.0839        | -      | -                        |
+| 0.1833 | 149  | 0.2055        | -      | -                        |
+| 0.1845 | 150  | 0.2196        | -      | -                        |
+| 0.1857 | 151  | 0.2283        | -      | -                        |
+| 0.1870 | 152  | 0.2105        | -      | -                        |
+| 0.1882 | 153  | 0.1534        | -      | -                        |
+| 0.1894 | 154  | 0.1954        | -      | -                        |
+| 0.1907 | 155  | 0.1332        | -      | -                        |
+| 0.1919 | 156  | 0.19          | -      | -                        |
+| 0.1931 | 157  | 0.1878        | -      | -                        |
+| 0.1943 | 158  | 0.1518        | -      | -                        |
+| 0.1956 | 159  | 0.1906        | -      | -                        |
+| 0.1968 | 160  | 0.155         | -      | -                        |
+| 0.1980 | 161  | 0.1519        | -      | -                        |
+| 0.1993 | 162  | 0.1726        | -      | -                        |
+| 0.2005 | 163  | 0.1618        | -      | -                        |
+| 0.2017 | 164  | 0.2767        | -      | -                        |
+| 0.2030 | 165  | 0.1996        | -      | -                        |
+| 0.2042 | 166  | 0.1907        | -      | -                        |
+| 0.2054 | 167  | 0.1928        | -      | -                        |
+| 0.2066 | 168  | 0.1507        | 0.1082 | 0.9045                   |
+| 0.2079 | 169  | 0.1637        | -      | -                        |
+| 0.2091 | 170  | 0.1687        | -      | -                        |
+| 0.2103 | 171  | 0.2181        | -      | -                        |
+| 0.2116 | 172  | 0.1496        | -      | -                        |
+| 0.2128 | 173  | 0.1749        | -      | -                        |
+| 0.2140 | 174  | 0.2374        | -      | -                        |
+| 0.2153 | 175  | 0.2122        | -      | -                        |
+| 0.2165 | 176  | 0.1617        | -      | -                        |
+| 0.2177 | 177  | 0.168         | -      | -                        |
+| 0.2189 | 178  | 0.263         | -      | -                        |
+| 0.2202 | 179  | 0.1328        | -      | -                        |
+| 0.2214 | 180  | 0.3157        | -      | -                        |
+| 0.2226 | 181  | 0.2164        | -      | -                        |
+| 0.2239 | 182  | 0.1255        | -      | -                        |
+| 0.2251 | 183  | 0.2863        | -      | -                        |
+| 0.2263 | 184  | 0.155         | -      | -                        |
+| 0.2276 | 185  | 0.1271        | -      | -                        |
+| 0.2288 | 186  | 0.216         | -      | -                        |
+| 0.2300 | 187  | 0.205         | -      | -                        |
+| 0.2312 | 188  | 0.1575        | -      | -                        |
+| 0.2325 | 189  | 0.1939        | 0.1057 | 0.9046                   |
+| 0.2337 | 190  | 0.2209        | -      | -                        |
+| 0.2349 | 191  | 0.153         | -      | -                        |
+| 0.2362 | 192  | 0.2187        | -      | -                        |
+| 0.2374 | 193  | 0.1593        | -      | -                        |
+| 0.2386 | 194  | 0.173         | -      | -                        |
+| 0.2399 | 195  | 0.2377        | -      | -                        |
+| 0.2411 | 196  | 0.2281        | -      | -                        |
+| 0.2423 | 197  | 0.2651        | -      | -                        |
+| 0.2435 | 198  | 0.118         | -      | -                        |
+| 0.2448 | 199  | 0.1728        | -      | -                        |
+| 0.2460 | 200  | 0.2299        | -      | -                        |
+| 0.2472 | 201  | 0.2342        | -      | -                        |
+| 0.2485 | 202  | 0.2413        | -      | -                        |
+| 0.2497 | 203  | 0.168         | -      | -                        |
+| 0.2509 | 204  | 0.1474        | -      | -                        |
+| 0.2522 | 205  | 0.1102        | -      | -                        |
+| 0.2534 | 206  | 0.2326        | -      | -                        |
+| 0.2546 | 207  | 0.1787        | -      | -                        |
+| 0.2558 | 208  | 0.1423        | -      | -                        |
+| 0.2571 | 209  | 0.2069        | -      | -                        |
+| 0.2583 | 210  | 0.136         | 0.1040 | 0.9056                   |
+| 0.2595 | 211  | 0.2407        | -      | -                        |
+| 0.2608 | 212  | 0.212         | -      | -                        |
+| 0.2620 | 213  | 0.1361        | -      | -                        |
+| 0.2632 | 214  | 0.2356        | -      | -                        |
+| 0.2645 | 215  | 0.1059        | -      | -                        |
+| 0.2657 | 216  | 0.2501        | -      | -                        |
+| 0.2669 | 217  | 0.1817        | -      | -                        |
+| 0.2681 | 218  | 0.2022        | -      | -                        |
+| 0.2694 | 219  | 0.2235        | -      | -                        |
+| 0.2706 | 220  | 0.2437        | -      | -                        |
+| 0.2718 | 221  | 0.1859        | -      | -                        |
+| 0.2731 | 222  | 0.2167        | -      | -                        |
+| 0.2743 | 223  | 0.1495        | -      | -                        |
+| 0.2755 | 224  | 0.2876        | -      | -                        |
+| 0.2768 | 225  | 0.1842        | -      | -                        |
+| 0.2780 | 226  | 0.144         | -      | -                        |
+| 0.2792 | 227  | 0.1571        | -      | -                        |
+| 0.2804 | 228  | 0.209         | -      | -                        |
+| 0.2817 | 229  | 0.2075        | -      | -                        |
+| 0.2829 | 230  | 0.1722        | -      | -                        |
+| 0.2841 | 231  | 0.1464        | 0.1039 | 0.9087                   |
+| 0.2854 | 232  | 0.2675        | -      | -                        |
+| 0.2866 | 233  | 0.2585        | -      | -                        |
+| 0.2878 | 234  | 0.134         | -      | -                        |
+| 0.2891 | 235  | 0.1765        | -      | -                        |
+| 0.2903 | 236  | 0.1826        | -      | -                        |
+| 0.2915 | 237  | 0.222         | -      | -                        |
+| 0.2927 | 238  | 0.134         | -      | -                        |
+| 0.2940 | 239  | 0.1902        | -      | -                        |
+| 0.2952 | 240  | 0.2461        | -      | -                        |
+| 0.2964 | 241  | 0.3094        | -      | -                        |
+| 0.2977 | 242  | 0.2252        | -      | -                        |
+| 0.2989 | 243  | 0.2466        | -      | -                        |
+| 0.3001 | 244  | 0.139         | -      | -                        |
+| 0.3014 | 245  | 0.154         | -      | -                        |
+| 0.3026 | 246  | 0.1979        | -      | -                        |
+
+</details>
+
+### Framework Versions
+- Python: 3.10.14
+- Sentence Transformers: 3.0.1
+- Transformers: 4.44.0
+- PyTorch: 2.4.0
+- Accelerate: 0.33.0
+- Datasets: 2.21.0
+- Tokenizers: 0.19.1
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

checkpoint-246/added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "[MASK]": 128000
+}

checkpoint-246/config.json

@@ -0,0 +1,35 @@
+{
+  "_name_or_path": "bobox/DeBERTa-small-ST-v1-test-step3",
+  "architectures": [
+    "DebertaV2Model"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-07,
+  "max_position_embeddings": 512,
+  "max_relative_positions": -1,
+  "model_type": "deberta-v2",
+  "norm_rel_ebd": "layer_norm",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 6,
+  "pad_token_id": 0,
+  "pooler_dropout": 0,
+  "pooler_hidden_act": "gelu",
+  "pooler_hidden_size": 768,
+  "pos_att_type": [
+    "p2c",
+    "c2p"
+  ],
+  "position_biased_input": false,
+  "position_buckets": 256,
+  "relative_attention": true,
+  "share_att_key": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.44.0",
+  "type_vocab_size": 0,
+  "vocab_size": 128100
+}

checkpoint-246/config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.0.1",
+    "transformers": "4.44.0",
+    "pytorch": "2.4.0"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": null
+}

checkpoint-246/modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

checkpoint-246/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d14df1827acba68e2f1fdea4243ce74fcc3055bdc39a71a0902a8dcc00937841
+size 1130520122

checkpoint-246/pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8840185b70a5f9ca86d3e1e78df0f10119189c7527f4d6eb542871263a7d8cbd
+size 565251810

checkpoint-246/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be2a40c1ba16d13edd52a4d08ff6e9acf297af40f40ae8e75dbe4c1317df9009
+size 14244

checkpoint-246/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:def79ad31dd422be5855b32ca9c0efa55e56c79df1b050c0302c2d5bc44a7f08
+size 1064

checkpoint-246/sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": false
+}

checkpoint-246/special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-246/spm.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c679fbf93643d19aab7ee10c0b99e460bdbc02fedf34b92b05af343b4af586fd
+size 2464616

checkpoint-246/tokenizer_config.json

@@ -0,0 +1,65 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128000": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "[CLS]",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "eos_token": "[SEP]",
+  "mask_token": "[MASK]",
+  "max_length": 512,
+  "model_max_length": 512,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "sp_model_kwargs": {},
+  "split_by_punct": false,
+  "stride": 0,
+  "tokenizer_class": "DebertaV2Tokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]",
+  "vocab_type": "spm"
+}

checkpoint-246/trainer_state.json

@@ -0,0 +1,1953 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.3025830258302583,
+  "eval_steps": 21,
+  "global_step": 246,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0012300123001230013,
+      "grad_norm": 6.540346145629883,
+      "learning_rate": 9.803921568627452e-08,
+      "loss": 0.3208,
+      "step": 1
+    },
+    {
+      "epoch": 0.0024600246002460025,
+      "grad_norm": 5.055933475494385,
+      "learning_rate": 1.9607843137254904e-07,
+      "loss": 0.1703,
+      "step": 2
+    },
+    {
+      "epoch": 0.0036900369003690036,
+      "grad_norm": 6.361550331115723,
+      "learning_rate": 2.9411764705882356e-07,
+      "loss": 0.3362,
+      "step": 3
+    },
+    {
+      "epoch": 0.004920049200492005,
+      "grad_norm": 6.709433078765869,
+      "learning_rate": 3.921568627450981e-07,
+      "loss": 0.3346,
+      "step": 4
+    },
+    {
+      "epoch": 0.006150061500615006,
+      "grad_norm": 5.4415154457092285,
+      "learning_rate": 4.901960784313725e-07,
+      "loss": 0.2484,
+      "step": 5
+    },
+    {
+      "epoch": 0.007380073800738007,
+      "grad_norm": 5.709558010101318,
+      "learning_rate": 5.882352941176471e-07,
+      "loss": 0.2249,
+      "step": 6
+    },
+    {
+      "epoch": 0.008610086100861008,
+      "grad_norm": 6.553178787231445,
+      "learning_rate": 6.862745098039217e-07,
+      "loss": 0.2724,
+      "step": 7
+    },
+    {
+      "epoch": 0.00984009840098401,
+      "grad_norm": 5.640111446380615,
+      "learning_rate": 7.843137254901962e-07,
+      "loss": 0.251,
+      "step": 8
+    },
+    {
+      "epoch": 0.01107011070110701,
+      "grad_norm": 5.696380615234375,
+      "learning_rate": 8.823529411764707e-07,
+      "loss": 0.2413,
+      "step": 9
+    },
+    {
+      "epoch": 0.012300123001230012,
+      "grad_norm": 6.983877182006836,
+      "learning_rate": 9.80392156862745e-07,
+      "loss": 0.382,
+      "step": 10
+    },
+    {
+      "epoch": 0.013530135301353014,
+      "grad_norm": 6.066723346710205,
+      "learning_rate": 1.0784313725490197e-06,
+      "loss": 0.2695,
+      "step": 11
+    },
+    {
+      "epoch": 0.014760147601476014,
+      "grad_norm": 5.643115520477295,
+      "learning_rate": 1.1764705882352942e-06,
+      "loss": 0.2392,
+      "step": 12
+    },
+    {
+      "epoch": 0.015990159901599015,
+      "grad_norm": 6.062892436981201,
+      "learning_rate": 1.2745098039215686e-06,
+      "loss": 0.3603,
+      "step": 13
+    },
+    {
+      "epoch": 0.017220172201722016,
+      "grad_norm": 6.2491655349731445,
+      "learning_rate": 1.3725490196078434e-06,
+      "loss": 0.3282,
+      "step": 14
+    },
+    {
+      "epoch": 0.01845018450184502,
+      "grad_norm": 6.1164398193359375,
+      "learning_rate": 1.4705882352941177e-06,
+      "loss": 0.2878,
+      "step": 15
+    },
+    {
+      "epoch": 0.01968019680196802,
+      "grad_norm": 5.676611423492432,
+      "learning_rate": 1.5686274509803923e-06,
+      "loss": 0.3046,
+      "step": 16
+    },
+    {
+      "epoch": 0.020910209102091022,
+      "grad_norm": 7.181272983551025,
+      "learning_rate": 1.6666666666666667e-06,
+      "loss": 0.3946,
+      "step": 17
+    },
+    {
+      "epoch": 0.02214022140221402,
+      "grad_norm": 5.430984020233154,
+      "learning_rate": 1.7647058823529414e-06,
+      "loss": 0.2038,
+      "step": 18
+    },
+    {
+      "epoch": 0.023370233702337023,
+      "grad_norm": 7.2283220291137695,
+      "learning_rate": 1.8627450980392158e-06,
+      "loss": 0.3542,
+      "step": 19
+    },
+    {
+      "epoch": 0.024600246002460024,
+      "grad_norm": 5.587338924407959,
+      "learning_rate": 1.96078431372549e-06,
+      "loss": 0.2369,
+      "step": 20
+    },
+    {
+      "epoch": 0.025830258302583026,
+      "grad_norm": 4.456090927124023,
+      "learning_rate": 2.058823529411765e-06,
+      "loss": 0.1967,
+      "step": 21
+    },
+    {
+      "epoch": 0.025830258302583026,
+      "eval_loss": 0.14506277441978455,
+      "eval_runtime": 54.872,
+      "eval_samples_per_second": 27.446,
+      "eval_steps_per_second": 0.219,
+      "eval_sts-test_pearson_cosine": 0.8860152816653839,
+      "eval_sts-test_pearson_dot": 0.8766503125978379,
+      "eval_sts-test_pearson_euclidean": 0.9084101290541164,
+      "eval_sts-test_pearson_manhattan": 0.909121525028934,
+      "eval_sts-test_pearson_max": 0.909121525028934,
+      "eval_sts-test_spearman_cosine": 0.9080919696366193,
+      "eval_sts-test_spearman_dot": 0.8799434709726907,
+      "eval_sts-test_spearman_euclidean": 0.9044399981995129,
+      "eval_sts-test_spearman_manhattan": 0.9048055712538192,
+      "eval_sts-test_spearman_max": 0.9080919696366193,
+      "step": 21
+    },
+    {
+      "epoch": 0.02706027060270603,
+      "grad_norm": 6.088884353637695,
+      "learning_rate": 2.1568627450980393e-06,
+      "loss": 0.2368,
+      "step": 22
+    },
+    {
+      "epoch": 0.028290282902829027,
+      "grad_norm": 5.354013919830322,
+      "learning_rate": 2.254901960784314e-06,
+      "loss": 0.263,
+      "step": 23
+    },
+    {
+      "epoch": 0.02952029520295203,
+      "grad_norm": 7.822023391723633,
+      "learning_rate": 2.3529411764705885e-06,
+      "loss": 0.3595,
+      "step": 24
+    },
+    {
+      "epoch": 0.03075030750307503,
+      "grad_norm": 6.401333332061768,
+      "learning_rate": 2.450980392156863e-06,
+      "loss": 0.3073,
+      "step": 25
+    },
+    {
+      "epoch": 0.03198031980319803,
+      "grad_norm": 5.567343235015869,
+      "learning_rate": 2.549019607843137e-06,
+      "loss": 0.2232,
+      "step": 26
+    },
+    {
+      "epoch": 0.033210332103321034,
+      "grad_norm": 4.244979381561279,
+      "learning_rate": 2.647058823529412e-06,
+      "loss": 0.1822,
+      "step": 27
+    },
+    {
+      "epoch": 0.03444034440344403,
+      "grad_norm": 5.674376964569092,
+      "learning_rate": 2.7450980392156867e-06,
+      "loss": 0.251,
+      "step": 28
+    },
+    {
+      "epoch": 0.03567035670356704,
+      "grad_norm": 6.017494201660156,
+      "learning_rate": 2.843137254901961e-06,
+      "loss": 0.2677,
+      "step": 29
+    },
+    {
+      "epoch": 0.03690036900369004,
+      "grad_norm": 6.415028095245361,
+      "learning_rate": 2.9411764705882355e-06,
+      "loss": 0.3252,
+      "step": 30
+    },
+    {
+      "epoch": 0.038130381303813035,
+      "grad_norm": 5.484204292297363,
+      "learning_rate": 3.03921568627451e-06,
+      "loss": 0.2058,
+      "step": 31
+    },
+    {
+      "epoch": 0.03936039360393604,
+      "grad_norm": 5.997295379638672,
+      "learning_rate": 3.1372549019607846e-06,
+      "loss": 0.3083,
+      "step": 32
+    },
+    {
+      "epoch": 0.04059040590405904,
+      "grad_norm": 5.527047157287598,
+      "learning_rate": 3.2352941176470594e-06,
+      "loss": 0.2109,
+      "step": 33
+    },
+    {
+      "epoch": 0.041820418204182044,
+      "grad_norm": 5.817302227020264,
+      "learning_rate": 3.3333333333333333e-06,
+      "loss": 0.2751,
+      "step": 34
+    },
+    {
+      "epoch": 0.04305043050430504,
+      "grad_norm": 5.476433753967285,
+      "learning_rate": 3.431372549019608e-06,
+      "loss": 0.2269,
+      "step": 35
+    },
+    {
+      "epoch": 0.04428044280442804,
+      "grad_norm": 5.363610744476318,
+      "learning_rate": 3.529411764705883e-06,
+      "loss": 0.2333,
+      "step": 36
+    },
+    {
+      "epoch": 0.04551045510455105,
+      "grad_norm": 6.07395601272583,
+      "learning_rate": 3.6274509803921573e-06,
+      "loss": 0.2747,
+      "step": 37
+    },
+    {
+      "epoch": 0.046740467404674045,
+      "grad_norm": 4.726163864135742,
+      "learning_rate": 3.7254901960784316e-06,
+      "loss": 0.1285,
+      "step": 38
+    },
+    {
+      "epoch": 0.04797047970479705,
+      "grad_norm": 5.783392906188965,
+      "learning_rate": 3.8235294117647055e-06,
+      "loss": 0.3659,
+      "step": 39
+    },
+    {
+      "epoch": 0.04920049200492005,
+      "grad_norm": 6.566931247711182,
+      "learning_rate": 3.92156862745098e-06,
+      "loss": 0.3991,
+      "step": 40
+    },
+    {
+      "epoch": 0.05043050430504305,
+      "grad_norm": 5.311452388763428,
+      "learning_rate": 4.019607843137255e-06,
+      "loss": 0.2647,
+      "step": 41
+    },
+    {
+      "epoch": 0.05166051660516605,
+      "grad_norm": 6.0737152099609375,
+      "learning_rate": 4.11764705882353e-06,
+      "loss": 0.3627,
+      "step": 42
+    },
+    {
+      "epoch": 0.05166051660516605,
+      "eval_loss": 0.1373225301504135,
+      "eval_runtime": 54.8187,
+      "eval_samples_per_second": 27.472,
+      "eval_steps_per_second": 0.219,
+      "eval_sts-test_pearson_cosine": 0.8846111050777101,
+      "eval_sts-test_pearson_dot": 0.8747554197498655,
+      "eval_sts-test_pearson_euclidean": 0.9089352149126115,
+      "eval_sts-test_pearson_manhattan": 0.9098483550214526,
+      "eval_sts-test_pearson_max": 0.9098483550214526,
+      "eval_sts-test_spearman_cosine": 0.9084485029361248,
+      "eval_sts-test_spearman_dot": 0.8796038088987298,
+      "eval_sts-test_spearman_euclidean": 0.9055790073044468,
+      "eval_sts-test_spearman_manhattan": 0.9063848432683216,
+      "eval_sts-test_spearman_max": 0.9084485029361248,
+      "step": 42
+    },
+    {
+      "epoch": 0.05289052890528905,
+      "grad_norm": 4.857839584350586,
+      "learning_rate": 4.215686274509805e-06,
+      "loss": 0.2026,
+      "step": 43
+    },
+    {
+      "epoch": 0.05412054120541206,
+      "grad_norm": 5.248873233795166,
+      "learning_rate": 4.313725490196079e-06,
+      "loss": 0.1923,
+      "step": 44
+    },
+    {
+      "epoch": 0.055350553505535055,
+      "grad_norm": 5.329862117767334,
+      "learning_rate": 4.411764705882353e-06,
+      "loss": 0.2369,
+      "step": 45
+    },
+    {
+      "epoch": 0.056580565805658053,
+      "grad_norm": 5.581146240234375,
+      "learning_rate": 4.509803921568628e-06,
+      "loss": 0.2268,
+      "step": 46
+    },
+    {
+      "epoch": 0.05781057810578106,
+      "grad_norm": 5.818411350250244,
+      "learning_rate": 4.607843137254902e-06,
+      "loss": 0.2975,
+      "step": 47
+    },
+    {
+      "epoch": 0.05904059040590406,
+      "grad_norm": 5.096602916717529,
+      "learning_rate": 4.705882352941177e-06,
+      "loss": 0.1922,
+      "step": 48
+    },
+    {
+      "epoch": 0.06027060270602706,
+      "grad_norm": 5.256355285644531,
+      "learning_rate": 4.803921568627452e-06,
+      "loss": 0.1906,
+      "step": 49
+    },
+    {
+      "epoch": 0.06150061500615006,
+      "grad_norm": 5.3927388191223145,
+      "learning_rate": 4.901960784313726e-06,
+      "loss": 0.2379,
+      "step": 50
+    },
+    {
+      "epoch": 0.06273062730627306,
+      "grad_norm": 6.2723846435546875,
+      "learning_rate": 5e-06,
+      "loss": 0.3796,
+      "step": 51
+    },
+    {
+      "epoch": 0.06396063960639606,
+      "grad_norm": 4.595238208770752,
+      "learning_rate": 5.098039215686274e-06,
+      "loss": 0.1821,
+      "step": 52
+    },
+    {
+      "epoch": 0.06519065190651907,
+      "grad_norm": 4.342020511627197,
+      "learning_rate": 5.19607843137255e-06,
+      "loss": 0.1257,
+      "step": 53
+    },
+    {
+      "epoch": 0.06642066420664207,
+      "grad_norm": 4.998225212097168,
+      "learning_rate": 5.294117647058824e-06,
+      "loss": 0.2368,
+      "step": 54
+    },
+    {
+      "epoch": 0.06765067650676507,
+      "grad_norm": 5.510946273803711,
+      "learning_rate": 5.392156862745098e-06,
+      "loss": 0.294,
+      "step": 55
+    },
+    {
+      "epoch": 0.06888068880688807,
+      "grad_norm": 4.788788318634033,
+      "learning_rate": 5.4901960784313735e-06,
+      "loss": 0.2594,
+      "step": 56
+    },
+    {
+      "epoch": 0.07011070110701106,
+      "grad_norm": 5.827020645141602,
+      "learning_rate": 5.588235294117647e-06,
+      "loss": 0.2972,
+      "step": 57
+    },
+    {
+      "epoch": 0.07134071340713408,
+      "grad_norm": 4.821737289428711,
+      "learning_rate": 5.686274509803922e-06,
+      "loss": 0.2297,
+      "step": 58
+    },
+    {
+      "epoch": 0.07257072570725707,
+      "grad_norm": 4.880247592926025,
+      "learning_rate": 5.784313725490197e-06,
+      "loss": 0.1487,
+      "step": 59
+    },
+    {
+      "epoch": 0.07380073800738007,
+      "grad_norm": 4.447835445404053,
+      "learning_rate": 5.882352941176471e-06,
+      "loss": 0.182,
+      "step": 60
+    },
+    {
+      "epoch": 0.07503075030750307,
+      "grad_norm": 5.5556640625,
+      "learning_rate": 5.980392156862746e-06,
+      "loss": 0.2516,
+      "step": 61
+    },
+    {
+      "epoch": 0.07626076260762607,
+      "grad_norm": 5.217922687530518,
+      "learning_rate": 6.07843137254902e-06,
+      "loss": 0.2809,
+      "step": 62
+    },
+    {
+      "epoch": 0.07749077490774908,
+      "grad_norm": 4.436608791351318,
+      "learning_rate": 6.176470588235295e-06,
+      "loss": 0.1371,
+      "step": 63
+    },
+    {
+      "epoch": 0.07749077490774908,
+      "eval_loss": 0.13080179691314697,
+      "eval_runtime": 54.9188,
+      "eval_samples_per_second": 27.422,
+      "eval_steps_per_second": 0.219,
+      "eval_sts-test_pearson_cosine": 0.882074513745531,
+      "eval_sts-test_pearson_dot": 0.8709046425878566,
+      "eval_sts-test_pearson_euclidean": 0.9081794284297221,
+      "eval_sts-test_pearson_manhattan": 0.9093974331692458,
+      "eval_sts-test_pearson_max": 0.9093974331692458,
+      "eval_sts-test_spearman_cosine": 0.9067824582257844,
+      "eval_sts-test_spearman_dot": 0.8757477717096785,
+      "eval_sts-test_spearman_euclidean": 0.9051085820447002,
+      "eval_sts-test_spearman_manhattan": 0.9064308923162935,
+      "eval_sts-test_spearman_max": 0.9067824582257844,
+      "step": 63
+    },
+    {
+      "epoch": 0.07872078720787208,
+      "grad_norm": 5.6947021484375,
+      "learning_rate": 6.274509803921569e-06,
+      "loss": 0.2149,
+      "step": 64
+    },
+    {
+      "epoch": 0.07995079950799508,
+      "grad_norm": 4.272282600402832,
+      "learning_rate": 6.372549019607843e-06,
+      "loss": 0.1806,
+      "step": 65
+    },
+    {
+      "epoch": 0.08118081180811808,
+      "grad_norm": 4.575979232788086,
+      "learning_rate": 6.470588235294119e-06,
+      "loss": 0.1458,
+      "step": 66
+    },
+    {
+      "epoch": 0.08241082410824108,
+      "grad_norm": 4.315216541290283,
+      "learning_rate": 6.568627450980393e-06,
+      "loss": 0.249,
+      "step": 67
+    },
+    {
+      "epoch": 0.08364083640836409,
+      "grad_norm": 5.67277193069458,
+      "learning_rate": 6.666666666666667e-06,
+      "loss": 0.2787,
+      "step": 68
+    },
+    {
+      "epoch": 0.08487084870848709,
+      "grad_norm": 5.964886665344238,
+      "learning_rate": 6.764705882352942e-06,
+      "loss": 0.288,
+      "step": 69
+    },
+    {
+      "epoch": 0.08610086100861009,
+      "grad_norm": 4.218502521514893,
+      "learning_rate": 6.862745098039216e-06,
+      "loss": 0.1461,
+      "step": 70
+    },
+    {
+      "epoch": 0.08733087330873308,
+      "grad_norm": 5.179543972015381,
+      "learning_rate": 6.96078431372549e-06,
+      "loss": 0.2304,
+      "step": 71
+    },
+    {
+      "epoch": 0.08856088560885608,
+      "grad_norm": 5.720668792724609,
+      "learning_rate": 7.058823529411766e-06,
+      "loss": 0.3505,
+      "step": 72
+    },
+    {
+      "epoch": 0.0897908979089791,
+      "grad_norm": 5.2965497970581055,
+      "learning_rate": 7.15686274509804e-06,
+      "loss": 0.2227,
+      "step": 73
+    },
+    {
+      "epoch": 0.0910209102091021,
+      "grad_norm": 4.685606956481934,
+      "learning_rate": 7.2549019607843145e-06,
+      "loss": 0.1746,
+      "step": 74
+    },
+    {
+      "epoch": 0.09225092250922509,
+      "grad_norm": 4.2930145263671875,
+      "learning_rate": 7.352941176470589e-06,
+      "loss": 0.1484,
+      "step": 75
+    },
+    {
+      "epoch": 0.09348093480934809,
+      "grad_norm": 3.764916181564331,
+      "learning_rate": 7.450980392156863e-06,
+      "loss": 0.1346,
+      "step": 76
+    },
+    {
+      "epoch": 0.09471094710947109,
+      "grad_norm": 5.033151626586914,
+      "learning_rate": 7.549019607843138e-06,
+      "loss": 0.2112,
+      "step": 77
+    },
+    {
+      "epoch": 0.0959409594095941,
+      "grad_norm": 5.817330837249756,
+      "learning_rate": 7.647058823529411e-06,
+      "loss": 0.3138,
+      "step": 78
+    },
+    {
+      "epoch": 0.0971709717097171,
+      "grad_norm": 6.147035121917725,
+      "learning_rate": 7.745098039215687e-06,
+      "loss": 0.2675,
+      "step": 79
+    },
+    {
+      "epoch": 0.0984009840098401,
+      "grad_norm": 5.131881237030029,
+      "learning_rate": 7.84313725490196e-06,
+      "loss": 0.2849,
+      "step": 80
+    },
+    {
+      "epoch": 0.0996309963099631,
+      "grad_norm": 4.2269368171691895,
+      "learning_rate": 7.941176470588236e-06,
+      "loss": 0.1719,
+      "step": 81
+    },
+    {
+      "epoch": 0.1008610086100861,
+      "grad_norm": 5.200590133666992,
+      "learning_rate": 8.03921568627451e-06,
+      "loss": 0.2749,
+      "step": 82
+    },
+    {
+      "epoch": 0.10209102091020911,
+      "grad_norm": 5.44044303894043,
+      "learning_rate": 8.137254901960784e-06,
+      "loss": 0.3097,
+      "step": 83
+    },
+    {
+      "epoch": 0.1033210332103321,
+      "grad_norm": 4.603049278259277,
+      "learning_rate": 8.23529411764706e-06,
+      "loss": 0.2068,
+      "step": 84
+    },
+    {
+      "epoch": 0.1033210332103321,
+      "eval_loss": 0.1260141134262085,
+      "eval_runtime": 54.8932,
+      "eval_samples_per_second": 27.435,
+      "eval_steps_per_second": 0.219,
+      "eval_sts-test_pearson_cosine": 0.8775839612260851,
+      "eval_sts-test_pearson_dot": 0.8664914414909934,
+      "eval_sts-test_pearson_euclidean": 0.9054210798291935,
+      "eval_sts-test_pearson_manhattan": 0.9069843565115414,
+      "eval_sts-test_pearson_max": 0.9069843565115414,
+      "eval_sts-test_spearman_cosine": 0.9044597335057865,
+      "eval_sts-test_spearman_dot": 0.872940077569982,
+      "eval_sts-test_spearman_euclidean": 0.9027100934391671,
+      "eval_sts-test_spearman_manhattan": 0.904476380975024,
+      "eval_sts-test_spearman_max": 0.904476380975024,
+      "step": 84
+    },
+    {
+      "epoch": 0.1045510455104551,
+      "grad_norm": 4.813210964202881,
+      "learning_rate": 8.333333333333334e-06,
+      "loss": 0.22,
+      "step": 85
+    },
+    {
+      "epoch": 0.1057810578105781,
+      "grad_norm": 4.659386157989502,
+      "learning_rate": 8.43137254901961e-06,
+      "loss": 0.2977,
+      "step": 86
+    },
+    {
+      "epoch": 0.1070110701107011,
+      "grad_norm": 4.895315647125244,
+      "learning_rate": 8.529411764705883e-06,
+      "loss": 0.209,
+      "step": 87
+    },
+    {
+      "epoch": 0.10824108241082411,
+      "grad_norm": 5.339110851287842,
+      "learning_rate": 8.627450980392157e-06,
+      "loss": 0.2215,
+      "step": 88
+    },
+    {
+      "epoch": 0.10947109471094711,
+      "grad_norm": 4.615406036376953,
+      "learning_rate": 8.725490196078433e-06,
+      "loss": 0.1948,
+      "step": 89
+    },
+    {
+      "epoch": 0.11070110701107011,
+      "grad_norm": 5.0383734703063965,
+      "learning_rate": 8.823529411764707e-06,
+      "loss": 0.2084,
+      "step": 90
+    },
+    {
+      "epoch": 0.11193111931119311,
+      "grad_norm": 3.9511592388153076,
+      "learning_rate": 8.921568627450982e-06,
+      "loss": 0.1823,
+      "step": 91
+    },
+    {
+      "epoch": 0.11316113161131611,
+      "grad_norm": 5.13690710067749,
+      "learning_rate": 9.019607843137256e-06,
+      "loss": 0.255,
+      "step": 92
+    },
+    {
+      "epoch": 0.11439114391143912,
+      "grad_norm": 5.1460747718811035,
+      "learning_rate": 9.11764705882353e-06,
+      "loss": 0.2675,
+      "step": 93
+    },
+    {
+      "epoch": 0.11562115621156212,
+      "grad_norm": 4.207213878631592,
+      "learning_rate": 9.215686274509804e-06,
+      "loss": 0.18,
+      "step": 94
+    },
+    {
+      "epoch": 0.11685116851168512,
+      "grad_norm": 4.802348613739014,
+      "learning_rate": 9.31372549019608e-06,
+      "loss": 0.2891,
+      "step": 95
+    },
+    {
+      "epoch": 0.11808118081180811,
+      "grad_norm": 4.9332966804504395,
+      "learning_rate": 9.411764705882354e-06,
+      "loss": 0.253,
+      "step": 96
+    },
+    {
+      "epoch": 0.11931119311193111,
+      "grad_norm": 5.841371536254883,
+      "learning_rate": 9.509803921568628e-06,
+      "loss": 0.3481,
+      "step": 97
+    },
+    {
+      "epoch": 0.12054120541205413,
+      "grad_norm": 3.70485782623291,
+      "learning_rate": 9.607843137254903e-06,
+      "loss": 0.1688,
+      "step": 98
+    },
+    {
+      "epoch": 0.12177121771217712,
+      "grad_norm": 4.415471076965332,
+      "learning_rate": 9.705882352941177e-06,
+      "loss": 0.1808,
+      "step": 99
+    },
+    {
+      "epoch": 0.12300123001230012,
+      "grad_norm": 5.058602809906006,
+      "learning_rate": 9.803921568627451e-06,
+      "loss": 0.2821,
+      "step": 100
+    },
+    {
+      "epoch": 0.12423124231242312,
+      "grad_norm": 4.303729057312012,
+      "learning_rate": 9.901960784313727e-06,
+      "loss": 0.1856,
+      "step": 101
+    },
+    {
+      "epoch": 0.12546125461254612,
+      "grad_norm": 4.048065185546875,
+      "learning_rate": 1e-05,
+      "loss": 0.1441,
+      "step": 102
+    },
+    {
+      "epoch": 0.12669126691266913,
+      "grad_norm": 4.463968753814697,
+      "learning_rate": 1.0098039215686275e-05,
+      "loss": 0.226,
+      "step": 103
+    },
+    {
+      "epoch": 0.12792127921279212,
+      "grad_norm": 3.401120901107788,
+      "learning_rate": 1.0196078431372549e-05,
+      "loss": 0.1662,
+      "step": 104
+    },
+    {
+      "epoch": 0.12915129151291513,
+      "grad_norm": 4.119345188140869,
+      "learning_rate": 1.0294117647058823e-05,
+      "loss": 0.2043,
+      "step": 105
+    },
+    {
+      "epoch": 0.12915129151291513,
+      "eval_loss": 0.11874283850193024,
+      "eval_runtime": 54.7282,
+      "eval_samples_per_second": 27.518,
+      "eval_steps_per_second": 0.219,
+      "eval_sts-test_pearson_cosine": 0.8767520821963045,
+      "eval_sts-test_pearson_dot": 0.8648481444888331,
+      "eval_sts-test_pearson_euclidean": 0.9053937497921556,
+      "eval_sts-test_pearson_manhattan": 0.9071737646452815,
+      "eval_sts-test_pearson_max": 0.9071737646452815,
+      "eval_sts-test_spearman_cosine": 0.9050983787571032,
+      "eval_sts-test_spearman_dot": 0.8730474805973213,
+      "eval_sts-test_spearman_euclidean": 0.9035385735413058,
+      "eval_sts-test_spearman_manhattan": 0.9054231834122819,
+      "eval_sts-test_spearman_max": 0.9054231834122819,
+      "step": 105
+    },
+    {
+      "epoch": 0.13038130381303814,
+      "grad_norm": 5.826413154602051,
+      "learning_rate": 1.03921568627451e-05,
+      "loss": 0.3907,
+      "step": 106
+    },
+    {
+      "epoch": 0.13161131611316113,
+      "grad_norm": 3.2629737854003906,
+      "learning_rate": 1.0490196078431374e-05,
+      "loss": 0.1332,
+      "step": 107
+    },
+    {
+      "epoch": 0.13284132841328414,
+      "grad_norm": 4.044755458831787,
+      "learning_rate": 1.0588235294117648e-05,
+      "loss": 0.2243,
+      "step": 108
+    },
+    {
+      "epoch": 0.13407134071340712,
+      "grad_norm": 3.9784040451049805,
+      "learning_rate": 1.0686274509803922e-05,
+      "loss": 0.162,
+      "step": 109
+    },
+    {
+      "epoch": 0.13530135301353013,
+      "grad_norm": 3.1851444244384766,
+      "learning_rate": 1.0784313725490196e-05,
+      "loss": 0.1481,
+      "step": 110
+    },
+    {
+      "epoch": 0.13653136531365315,
+      "grad_norm": 4.281413555145264,
+      "learning_rate": 1.0882352941176471e-05,
+      "loss": 0.2163,
+      "step": 111
+    },
+    {
+      "epoch": 0.13776137761377613,
+      "grad_norm": 4.62849235534668,
+      "learning_rate": 1.0980392156862747e-05,
+      "loss": 0.24,
+      "step": 112
+    },
+    {
+      "epoch": 0.13899138991389914,
+      "grad_norm": 3.92616868019104,
+      "learning_rate": 1.1078431372549021e-05,
+      "loss": 0.1406,
+      "step": 113
+    },
+    {
+      "epoch": 0.14022140221402213,
+      "grad_norm": 3.8505780696868896,
+      "learning_rate": 1.1176470588235295e-05,
+      "loss": 0.1522,
+      "step": 114
+    },
+    {
+      "epoch": 0.14145141451414514,
+      "grad_norm": 5.220509052276611,
+      "learning_rate": 1.1274509803921569e-05,
+      "loss": 0.2593,
+      "step": 115
+    },
+    {
+      "epoch": 0.14268142681426815,
+      "grad_norm": 4.459743499755859,
+      "learning_rate": 1.1372549019607844e-05,
+      "loss": 0.2426,
+      "step": 116
+    },
+    {
+      "epoch": 0.14391143911439114,
+      "grad_norm": 4.434360504150391,
+      "learning_rate": 1.1470588235294118e-05,
+      "loss": 0.1781,
+      "step": 117
+    },
+    {
+      "epoch": 0.14514145141451415,
+      "grad_norm": 4.638584613800049,
+      "learning_rate": 1.1568627450980394e-05,
+      "loss": 0.264,
+      "step": 118
+    },
+    {
+      "epoch": 0.14637146371463713,
+      "grad_norm": 4.5364484786987305,
+      "learning_rate": 1.1666666666666668e-05,
+      "loss": 0.1944,
+      "step": 119
+    },
+    {
+      "epoch": 0.14760147601476015,
+      "grad_norm": 3.597980499267578,
+      "learning_rate": 1.1764705882352942e-05,
+      "loss": 0.1341,
+      "step": 120
+    },
+    {
+      "epoch": 0.14883148831488316,
+      "grad_norm": 3.5174648761749268,
+      "learning_rate": 1.1862745098039217e-05,
+      "loss": 0.155,
+      "step": 121
+    },
+    {
+      "epoch": 0.15006150061500614,
+      "grad_norm": 4.771029949188232,
+      "learning_rate": 1.1960784313725491e-05,
+      "loss": 0.2052,
+      "step": 122
+    },
+    {
+      "epoch": 0.15129151291512916,
+      "grad_norm": 4.15376615524292,
+      "learning_rate": 1.2058823529411765e-05,
+      "loss": 0.2023,
+      "step": 123
+    },
+    {
+      "epoch": 0.15252152521525214,
+      "grad_norm": 3.5796732902526855,
+      "learning_rate": 1.215686274509804e-05,
+      "loss": 0.1519,
+      "step": 124
+    },
+    {
+      "epoch": 0.15375153751537515,
+      "grad_norm": 3.759777545928955,
+      "learning_rate": 1.2254901960784315e-05,
+      "loss": 0.2118,
+      "step": 125
+    },
+    {
+      "epoch": 0.15498154981549817,
+      "grad_norm": 4.691242218017578,
+      "learning_rate": 1.235294117647059e-05,
+      "loss": 0.2489,
+      "step": 126
+    },
+    {
+      "epoch": 0.15498154981549817,
+      "eval_loss": 0.11467884480953217,
+      "eval_runtime": 54.6969,
+      "eval_samples_per_second": 27.534,
+      "eval_steps_per_second": 0.219,
+      "eval_sts-test_pearson_cosine": 0.8783763084873629,
+      "eval_sts-test_pearson_dot": 0.8674040012483692,
+      "eval_sts-test_pearson_euclidean": 0.9069725735634968,
+      "eval_sts-test_pearson_manhattan": 0.908783443457056,
+      "eval_sts-test_pearson_max": 0.908783443457056,
+      "eval_sts-test_spearman_cosine": 0.9058364613112314,
+      "eval_sts-test_spearman_dot": 0.8738751104254939,
+      "eval_sts-test_spearman_euclidean": 0.904588080123457,
+      "eval_sts-test_spearman_manhattan": 0.9067583820471556,
+      "eval_sts-test_spearman_max": 0.9067583820471556,
+      "step": 126
+    },
+    {
+      "epoch": 0.15621156211562115,
+      "grad_norm": 4.087611198425293,
+      "learning_rate": 1.2450980392156864e-05,
+      "loss": 0.1988,
+      "step": 127
+    },
+    {
+      "epoch": 0.15744157441574416,
+      "grad_norm": 3.754612684249878,
+      "learning_rate": 1.2549019607843138e-05,
+      "loss": 0.1541,
+      "step": 128
+    },
+    {
+      "epoch": 0.15867158671586715,
+      "grad_norm": 3.9258835315704346,
+      "learning_rate": 1.2647058823529412e-05,
+      "loss": 0.1819,
+      "step": 129
+    },
+    {
+      "epoch": 0.15990159901599016,
+      "grad_norm": 3.88478422164917,
+      "learning_rate": 1.2745098039215686e-05,
+      "loss": 0.1582,
+      "step": 130
+    },
+    {
+      "epoch": 0.16113161131611317,
+      "grad_norm": 4.9845428466796875,
+      "learning_rate": 1.2843137254901964e-05,
+      "loss": 0.2866,
+      "step": 131
+    },
+    {
+      "epoch": 0.16236162361623616,
+      "grad_norm": 4.692960262298584,
+      "learning_rate": 1.2941176470588238e-05,
+      "loss": 0.2766,
+      "step": 132
+    },
+    {
+      "epoch": 0.16359163591635917,
+      "grad_norm": 3.9432125091552734,
+      "learning_rate": 1.3039215686274511e-05,
+      "loss": 0.1299,
+      "step": 133
+    },
+    {
+      "epoch": 0.16482164821648215,
+      "grad_norm": 4.439709663391113,
+      "learning_rate": 1.3137254901960785e-05,
+      "loss": 0.2558,
+      "step": 134
+    },
+    {
+      "epoch": 0.16605166051660517,
+      "grad_norm": 3.631169319152832,
+      "learning_rate": 1.323529411764706e-05,
+      "loss": 0.1687,
+      "step": 135
+    },
+    {
+      "epoch": 0.16728167281672818,
+      "grad_norm": 4.130221843719482,
+      "learning_rate": 1.3333333333333333e-05,
+      "loss": 0.173,
+      "step": 136
+    },
+    {
+      "epoch": 0.16851168511685116,
+      "grad_norm": 4.169937610626221,
+      "learning_rate": 1.3431372549019607e-05,
+      "loss": 0.2276,
+      "step": 137
+    },
+    {
+      "epoch": 0.16974169741697417,
+      "grad_norm": 4.4349751472473145,
+      "learning_rate": 1.3529411764705885e-05,
+      "loss": 0.2174,
+      "step": 138
+    },
+    {
+      "epoch": 0.17097170971709716,
+      "grad_norm": 4.688521862030029,
+      "learning_rate": 1.3627450980392158e-05,
+      "loss": 0.2666,
+      "step": 139
+    },
+    {
+      "epoch": 0.17220172201722017,
+      "grad_norm": 3.7199971675872803,
+      "learning_rate": 1.3725490196078432e-05,
+      "loss": 0.1524,
+      "step": 140
+    },
+    {
+      "epoch": 0.17343173431734318,
+      "grad_norm": 2.8609495162963867,
+      "learning_rate": 1.3823529411764706e-05,
+      "loss": 0.1179,
+      "step": 141
+    },
+    {
+      "epoch": 0.17466174661746617,
+      "grad_norm": 4.374091625213623,
+      "learning_rate": 1.392156862745098e-05,
+      "loss": 0.2475,
+      "step": 142
+    },
+    {
+      "epoch": 0.17589175891758918,
+      "grad_norm": 5.200084209442139,
+      "learning_rate": 1.4019607843137256e-05,
+      "loss": 0.2662,
+      "step": 143
+    },
+    {
+      "epoch": 0.17712177121771217,
+      "grad_norm": 3.720994710922241,
+      "learning_rate": 1.4117647058823532e-05,
+      "loss": 0.1596,
+      "step": 144
+    },
+    {
+      "epoch": 0.17835178351783518,
+      "grad_norm": 3.991046905517578,
+      "learning_rate": 1.4215686274509805e-05,
+      "loss": 0.2331,
+      "step": 145
+    },
+    {
+      "epoch": 0.1795817958179582,
+      "grad_norm": 4.76691198348999,
+      "learning_rate": 1.431372549019608e-05,
+      "loss": 0.2905,
+      "step": 146
+    },
+    {
+      "epoch": 0.18081180811808117,
+      "grad_norm": 3.6453163623809814,
+      "learning_rate": 1.4411764705882353e-05,
+      "loss": 0.1342,
+      "step": 147
+    },
+    {
+      "epoch": 0.18081180811808117,
+      "eval_loss": 0.10875426232814789,
+      "eval_runtime": 54.7153,
+      "eval_samples_per_second": 27.524,
+      "eval_steps_per_second": 0.219,
+      "eval_sts-test_pearson_cosine": 0.8768801472189502,
+      "eval_sts-test_pearson_dot": 0.8620776961156391,
+      "eval_sts-test_pearson_euclidean": 0.9073367408863471,
+      "eval_sts-test_pearson_manhattan": 0.9086519830687241,
+      "eval_sts-test_pearson_max": 0.9086519830687241,
+      "eval_sts-test_spearman_cosine": 0.905147068129497,
+      "eval_sts-test_spearman_dot": 0.869011811845045,
+      "eval_sts-test_spearman_euclidean": 0.9050077574527855,
+      "eval_sts-test_spearman_manhattan": 0.9068115017944273,
+      "eval_sts-test_spearman_max": 0.9068115017944273,
+      "step": 147
+    },
+    {
+      "epoch": 0.1820418204182042,
+      "grad_norm": 2.6028358936309814,
+      "learning_rate": 1.4509803921568629e-05,
+      "loss": 0.0839,
+      "step": 148
+    },
+    {
+      "epoch": 0.18327183271832717,
+      "grad_norm": 4.445943832397461,
+      "learning_rate": 1.4607843137254903e-05,
+      "loss": 0.2055,
+      "step": 149
+    },
+    {
+      "epoch": 0.18450184501845018,
+      "grad_norm": 4.500098705291748,
+      "learning_rate": 1.4705882352941179e-05,
+      "loss": 0.2196,
+      "step": 150
+    },
+    {
+      "epoch": 0.1857318573185732,
+      "grad_norm": 4.317416667938232,
+      "learning_rate": 1.4803921568627453e-05,
+      "loss": 0.2283,
+      "step": 151
+    },
+    {
+      "epoch": 0.18696186961869618,
+      "grad_norm": 4.395689010620117,
+      "learning_rate": 1.4901960784313726e-05,
+      "loss": 0.2105,
+      "step": 152
+    },
+    {
+      "epoch": 0.1881918819188192,
+      "grad_norm": 3.5757391452789307,
+      "learning_rate": 1.5000000000000002e-05,
+      "loss": 0.1534,
+      "step": 153
+    },
+    {
+      "epoch": 0.18942189421894218,
+      "grad_norm": 3.860861301422119,
+      "learning_rate": 1.5098039215686276e-05,
+      "loss": 0.1954,
+      "step": 154
+    },
+    {
+      "epoch": 0.1906519065190652,
+      "grad_norm": 3.4191622734069824,
+      "learning_rate": 1.519607843137255e-05,
+      "loss": 0.1332,
+      "step": 155
+    },
+    {
+      "epoch": 0.1918819188191882,
+      "grad_norm": 3.8505654335021973,
+      "learning_rate": 1.5294117647058822e-05,
+      "loss": 0.19,
+      "step": 156
+    },
+    {
+      "epoch": 0.1931119311193112,
+      "grad_norm": 4.127209663391113,
+      "learning_rate": 1.53921568627451e-05,
+      "loss": 0.1878,
+      "step": 157
+    },
+    {
+      "epoch": 0.1943419434194342,
+      "grad_norm": 3.7976646423339844,
+      "learning_rate": 1.5490196078431373e-05,
+      "loss": 0.1518,
+      "step": 158
+    },
+    {
+      "epoch": 0.19557195571955718,
+      "grad_norm": 4.613111972808838,
+      "learning_rate": 1.558823529411765e-05,
+      "loss": 0.1906,
+      "step": 159
+    },
+    {
+      "epoch": 0.1968019680196802,
+      "grad_norm": 3.911393880844116,
+      "learning_rate": 1.568627450980392e-05,
+      "loss": 0.155,
+      "step": 160
+    },
+    {
+      "epoch": 0.1980319803198032,
+      "grad_norm": 3.694939374923706,
+      "learning_rate": 1.5784313725490197e-05,
+      "loss": 0.1519,
+      "step": 161
+    },
+    {
+      "epoch": 0.1992619926199262,
+      "grad_norm": 4.334694862365723,
+      "learning_rate": 1.5882352941176473e-05,
+      "loss": 0.1726,
+      "step": 162
+    },
+    {
+      "epoch": 0.2004920049200492,
+      "grad_norm": 3.6630055904388428,
+      "learning_rate": 1.5980392156862748e-05,
+      "loss": 0.1618,
+      "step": 163
+    },
+    {
+      "epoch": 0.2017220172201722,
+      "grad_norm": 4.7789130210876465,
+      "learning_rate": 1.607843137254902e-05,
+      "loss": 0.2767,
+      "step": 164
+    },
+    {
+      "epoch": 0.2029520295202952,
+      "grad_norm": 4.171343803405762,
+      "learning_rate": 1.6176470588235296e-05,
+      "loss": 0.1996,
+      "step": 165
+    },
+    {
+      "epoch": 0.20418204182041821,
+      "grad_norm": 4.386513710021973,
+      "learning_rate": 1.627450980392157e-05,
+      "loss": 0.1907,
+      "step": 166
+    },
+    {
+      "epoch": 0.2054120541205412,
+      "grad_norm": 4.183532238006592,
+      "learning_rate": 1.6372549019607844e-05,
+      "loss": 0.1928,
+      "step": 167
+    },
+    {
+      "epoch": 0.2066420664206642,
+      "grad_norm": 3.8950257301330566,
+      "learning_rate": 1.647058823529412e-05,
+      "loss": 0.1507,
+      "step": 168
+    },
+    {
+      "epoch": 0.2066420664206642,
+      "eval_loss": 0.10821738839149475,
+      "eval_runtime": 54.7389,
+      "eval_samples_per_second": 27.512,
+      "eval_steps_per_second": 0.219,
+      "eval_sts-test_pearson_cosine": 0.8772991124680096,
+      "eval_sts-test_pearson_dot": 0.861322579093208,
+      "eval_sts-test_pearson_euclidean": 0.9072621517681675,
+      "eval_sts-test_pearson_manhattan": 0.9086600802981594,
+      "eval_sts-test_pearson_max": 0.9086600802981594,
+      "eval_sts-test_spearman_cosine": 0.9044609865411055,
+      "eval_sts-test_spearman_dot": 0.8661539962925903,
+      "eval_sts-test_spearman_euclidean": 0.904084091417667,
+      "eval_sts-test_spearman_manhattan": 0.9054917423447336,
+      "eval_sts-test_spearman_max": 0.9054917423447336,
+      "step": 168
+    },
+    {
+      "epoch": 0.2078720787207872,
+      "grad_norm": 4.002283096313477,
+      "learning_rate": 1.6568627450980395e-05,
+      "loss": 0.1637,
+      "step": 169
+    },
+    {
+      "epoch": 0.2091020910209102,
+      "grad_norm": 4.142872333526611,
+      "learning_rate": 1.6666666666666667e-05,
+      "loss": 0.1687,
+      "step": 170
+    },
+    {
+      "epoch": 0.21033210332103322,
+      "grad_norm": 4.345719337463379,
+      "learning_rate": 1.6764705882352943e-05,
+      "loss": 0.2181,
+      "step": 171
+    },
+    {
+      "epoch": 0.2115621156211562,
+      "grad_norm": 3.7364888191223145,
+      "learning_rate": 1.686274509803922e-05,
+      "loss": 0.1496,
+      "step": 172
+    },
+    {
+      "epoch": 0.21279212792127922,
+      "grad_norm": 4.202157974243164,
+      "learning_rate": 1.696078431372549e-05,
+      "loss": 0.1749,
+      "step": 173
+    },
+    {
+      "epoch": 0.2140221402214022,
+      "grad_norm": 4.639451503753662,
+      "learning_rate": 1.7058823529411767e-05,
+      "loss": 0.2374,
+      "step": 174
+    },
+    {
+      "epoch": 0.21525215252152521,
+      "grad_norm": 4.011781215667725,
+      "learning_rate": 1.715686274509804e-05,
+      "loss": 0.2122,
+      "step": 175
+    },
+    {
+      "epoch": 0.21648216482164823,
+      "grad_norm": 4.113095760345459,
+      "learning_rate": 1.7254901960784314e-05,
+      "loss": 0.1617,
+      "step": 176
+    },
+    {
+      "epoch": 0.2177121771217712,
+      "grad_norm": 4.0442681312561035,
+      "learning_rate": 1.735294117647059e-05,
+      "loss": 0.168,
+      "step": 177
+    },
+    {
+      "epoch": 0.21894218942189422,
+      "grad_norm": 4.375425338745117,
+      "learning_rate": 1.7450980392156866e-05,
+      "loss": 0.263,
+      "step": 178
+    },
+    {
+      "epoch": 0.2201722017220172,
+      "grad_norm": 3.2303390502929688,
+      "learning_rate": 1.7549019607843138e-05,
+      "loss": 0.1328,
+      "step": 179
+    },
+    {
+      "epoch": 0.22140221402214022,
+      "grad_norm": 4.832092761993408,
+      "learning_rate": 1.7647058823529414e-05,
+      "loss": 0.3157,
+      "step": 180
+    },
+    {
+      "epoch": 0.22263222632226323,
+      "grad_norm": 3.57254695892334,
+      "learning_rate": 1.7745098039215686e-05,
+      "loss": 0.2164,
+      "step": 181
+    },
+    {
+      "epoch": 0.22386223862238622,
+      "grad_norm": 3.135535717010498,
+      "learning_rate": 1.7843137254901965e-05,
+      "loss": 0.1255,
+      "step": 182
+    },
+    {
+      "epoch": 0.22509225092250923,
+      "grad_norm": 4.719324588775635,
+      "learning_rate": 1.7941176470588237e-05,
+      "loss": 0.2863,
+      "step": 183
+    },
+    {
+      "epoch": 0.22632226322263221,
+      "grad_norm": 3.8961801528930664,
+      "learning_rate": 1.8039215686274513e-05,
+      "loss": 0.155,
+      "step": 184
+    },
+    {
+      "epoch": 0.22755227552275523,
+      "grad_norm": 2.8389103412628174,
+      "learning_rate": 1.8137254901960785e-05,
+      "loss": 0.1271,
+      "step": 185
+    },
+    {
+      "epoch": 0.22878228782287824,
+      "grad_norm": 4.103536128997803,
+      "learning_rate": 1.823529411764706e-05,
+      "loss": 0.216,
+      "step": 186
+    },
+    {
+      "epoch": 0.23001230012300122,
+      "grad_norm": 4.006705284118652,
+      "learning_rate": 1.8333333333333333e-05,
+      "loss": 0.205,
+      "step": 187
+    },
+    {
+      "epoch": 0.23124231242312424,
+      "grad_norm": 3.424255847930908,
+      "learning_rate": 1.843137254901961e-05,
+      "loss": 0.1575,
+      "step": 188
+    },
+    {
+      "epoch": 0.23247232472324722,
+      "grad_norm": 4.568851947784424,
+      "learning_rate": 1.8529411764705884e-05,
+      "loss": 0.1939,
+      "step": 189
+    },
+    {
+      "epoch": 0.23247232472324722,
+      "eval_loss": 0.1056687980890274,
+      "eval_runtime": 54.7136,
+      "eval_samples_per_second": 27.525,
+      "eval_steps_per_second": 0.219,
+      "eval_sts-test_pearson_cosine": 0.8789160692756717,
+      "eval_sts-test_pearson_dot": 0.8639029174125306,
+      "eval_sts-test_pearson_euclidean": 0.9084173029414142,
+      "eval_sts-test_pearson_manhattan": 0.9093131544369648,
+      "eval_sts-test_pearson_max": 0.9093131544369648,
+      "eval_sts-test_spearman_cosine": 0.904571298400435,
+      "eval_sts-test_spearman_dot": 0.8658778810098052,
+      "eval_sts-test_spearman_euclidean": 0.9046812074984125,
+      "eval_sts-test_spearman_manhattan": 0.9056302027474785,
+      "eval_sts-test_spearman_max": 0.9056302027474785,
+      "step": 189
+    },
+    {
+      "epoch": 0.23370233702337023,
+      "grad_norm": 3.5909903049468994,
+      "learning_rate": 1.862745098039216e-05,
+      "loss": 0.2209,
+      "step": 190
+    },
+    {
+      "epoch": 0.23493234932349324,
+      "grad_norm": 3.443946361541748,
+      "learning_rate": 1.8725490196078432e-05,
+      "loss": 0.153,
+      "step": 191
+    },
+    {
+      "epoch": 0.23616236162361623,
+      "grad_norm": 3.8604445457458496,
+      "learning_rate": 1.8823529411764708e-05,
+      "loss": 0.2187,
+      "step": 192
+    },
+    {
+      "epoch": 0.23739237392373924,
+      "grad_norm": 3.5916690826416016,
+      "learning_rate": 1.892156862745098e-05,
+      "loss": 0.1593,
+      "step": 193
+    },
+    {
+      "epoch": 0.23862238622386223,
+      "grad_norm": 3.8676974773406982,
+      "learning_rate": 1.9019607843137255e-05,
+      "loss": 0.173,
+      "step": 194
+    },
+    {
+      "epoch": 0.23985239852398524,
+      "grad_norm": 4.338643550872803,
+      "learning_rate": 1.911764705882353e-05,
+      "loss": 0.2377,
+      "step": 195
+    },
+    {
+      "epoch": 0.24108241082410825,
+      "grad_norm": 4.509932994842529,
+      "learning_rate": 1.9215686274509807e-05,
+      "loss": 0.2281,
+      "step": 196
+    },
+    {
+      "epoch": 0.24231242312423124,
+      "grad_norm": 4.282917022705078,
+      "learning_rate": 1.931372549019608e-05,
+      "loss": 0.2651,
+      "step": 197
+    },
+    {
+      "epoch": 0.24354243542435425,
+      "grad_norm": 3.1566977500915527,
+      "learning_rate": 1.9411764705882355e-05,
+      "loss": 0.118,
+      "step": 198
+    },
+    {
+      "epoch": 0.24477244772447723,
+      "grad_norm": 4.118341445922852,
+      "learning_rate": 1.950980392156863e-05,
+      "loss": 0.1728,
+      "step": 199
+    },
+    {
+      "epoch": 0.24600246002460024,
+      "grad_norm": 4.250949859619141,
+      "learning_rate": 1.9607843137254903e-05,
+      "loss": 0.2299,
+      "step": 200
+    },
+    {
+      "epoch": 0.24723247232472326,
+      "grad_norm": 4.084754943847656,
+      "learning_rate": 1.9705882352941178e-05,
+      "loss": 0.2342,
+      "step": 201
+    },
+    {
+      "epoch": 0.24846248462484624,
+      "grad_norm": 3.939434051513672,
+      "learning_rate": 1.9803921568627454e-05,
+      "loss": 0.2413,
+      "step": 202
+    },
+    {
+      "epoch": 0.24969249692496925,
+      "grad_norm": 3.9612276554107666,
+      "learning_rate": 1.9901960784313726e-05,
+      "loss": 0.168,
+      "step": 203
+    },
+    {
+      "epoch": 0.25092250922509224,
+      "grad_norm": 3.401622772216797,
+      "learning_rate": 2e-05,
+      "loss": 0.1474,
+      "step": 204
+    },
+    {
+      "epoch": 0.2521525215252153,
+      "grad_norm": 3.2245850563049316,
+      "learning_rate": 1.9998802517966852e-05,
+      "loss": 0.1102,
+      "step": 205
+    },
+    {
+      "epoch": 0.25338253382533826,
+      "grad_norm": 4.254729270935059,
+      "learning_rate": 1.9995210358660037e-05,
+      "loss": 0.2326,
+      "step": 206
+    },
+    {
+      "epoch": 0.25461254612546125,
+      "grad_norm": 3.603159189224243,
+      "learning_rate": 1.9989224382388813e-05,
+      "loss": 0.1787,
+      "step": 207
+    },
+    {
+      "epoch": 0.25584255842558423,
+      "grad_norm": 3.434582471847534,
+      "learning_rate": 1.9980846022772978e-05,
+      "loss": 0.1423,
+      "step": 208
+    },
+    {
+      "epoch": 0.2570725707257073,
+      "grad_norm": 3.8560950756073,
+      "learning_rate": 1.997007728639956e-05,
+      "loss": 0.2069,
+      "step": 209
+    },
+    {
+      "epoch": 0.25830258302583026,
+      "grad_norm": 3.4417314529418945,
+      "learning_rate": 1.9956920752342226e-05,
+      "loss": 0.136,
+      "step": 210
+    },
+    {
+      "epoch": 0.25830258302583026,
+      "eval_loss": 0.10401736944913864,
+      "eval_runtime": 54.8034,
+      "eval_samples_per_second": 27.48,
+      "eval_steps_per_second": 0.219,
+      "eval_sts-test_pearson_cosine": 0.8792920980078169,
+      "eval_sts-test_pearson_dot": 0.8612401830255582,
+      "eval_sts-test_pearson_euclidean": 0.9094380100842928,
+      "eval_sts-test_pearson_manhattan": 0.9095661408662257,
+      "eval_sts-test_pearson_max": 0.9095661408662257,
+      "eval_sts-test_spearman_cosine": 0.905583034917972,
+      "eval_sts-test_spearman_dot": 0.8658094563311378,
+      "eval_sts-test_spearman_euclidean": 0.906560626223067,
+      "eval_sts-test_spearman_manhattan": 0.906644400584392,
+      "eval_sts-test_spearman_max": 0.906644400584392,
+      "step": 210
+    },
+    {
+      "epoch": 0.25953259532595324,
+      "grad_norm": 4.170388221740723,
+      "learning_rate": 1.9941379571543597e-05,
+      "loss": 0.2407,
+      "step": 211
+    },
+    {
+      "epoch": 0.2607626076260763,
+      "grad_norm": 4.218827247619629,
+      "learning_rate": 1.9923457466060637e-05,
+      "loss": 0.212,
+      "step": 212
+    },
+    {
+      "epoch": 0.26199261992619927,
+      "grad_norm": 3.6592209339141846,
+      "learning_rate": 1.9903158728173206e-05,
+      "loss": 0.1361,
+      "step": 213
+    },
+    {
+      "epoch": 0.26322263222632225,
+      "grad_norm": 4.208631992340088,
+      "learning_rate": 1.9880488219356086e-05,
+      "loss": 0.2356,
+      "step": 214
+    },
+    {
+      "epoch": 0.2644526445264453,
+      "grad_norm": 2.9232637882232666,
+      "learning_rate": 1.9855451369114677e-05,
+      "loss": 0.1059,
+      "step": 215
+    },
+    {
+      "epoch": 0.2656826568265683,
+      "grad_norm": 4.299160480499268,
+      "learning_rate": 1.9828054173684646e-05,
+      "loss": 0.2501,
+      "step": 216
+    },
+    {
+      "epoch": 0.26691266912669126,
+      "grad_norm": 4.013469219207764,
+      "learning_rate": 1.9798303194595846e-05,
+      "loss": 0.1817,
+      "step": 217
+    },
+    {
+      "epoch": 0.26814268142681424,
+      "grad_norm": 3.691553831100464,
+      "learning_rate": 1.976620555710087e-05,
+      "loss": 0.2022,
+      "step": 218
+    },
+    {
+      "epoch": 0.2693726937269373,
+      "grad_norm": 4.433103561401367,
+      "learning_rate": 1.973176894846855e-05,
+      "loss": 0.2235,
+      "step": 219
+    },
+    {
+      "epoch": 0.27060270602706027,
+      "grad_norm": 4.862768173217773,
+      "learning_rate": 1.9695001616142916e-05,
+      "loss": 0.2437,
+      "step": 220
+    },
+    {
+      "epoch": 0.27183271832718325,
+      "grad_norm": 3.9157614707946777,
+      "learning_rate": 1.965591236576794e-05,
+      "loss": 0.1859,
+      "step": 221
+    },
+    {
+      "epoch": 0.2730627306273063,
+      "grad_norm": 4.705247402191162,
+      "learning_rate": 1.9614510559078626e-05,
+      "loss": 0.2167,
+      "step": 222
+    },
+    {
+      "epoch": 0.2742927429274293,
+      "grad_norm": 3.890500068664551,
+      "learning_rate": 1.95708061116589e-05,
+      "loss": 0.1495,
+      "step": 223
+    },
+    {
+      "epoch": 0.27552275522755226,
+      "grad_norm": 4.393867492675781,
+      "learning_rate": 1.9524809490566878e-05,
+      "loss": 0.2876,
+      "step": 224
+    },
+    {
+      "epoch": 0.2767527675276753,
+      "grad_norm": 3.782416582107544,
+      "learning_rate": 1.9476531711828027e-05,
+      "loss": 0.1842,
+      "step": 225
+    },
+    {
+      "epoch": 0.2779827798277983,
+      "grad_norm": 3.32236647605896,
+      "learning_rate": 1.942598433779687e-05,
+      "loss": 0.144,
+      "step": 226
+    },
+    {
+      "epoch": 0.27921279212792127,
+      "grad_norm": 3.9284870624542236,
+      "learning_rate": 1.9373179474387858e-05,
+      "loss": 0.1571,
+      "step": 227
+    },
+    {
+      "epoch": 0.28044280442804426,
+      "grad_norm": 3.847404956817627,
+      "learning_rate": 1.9318129768176033e-05,
+      "loss": 0.209,
+      "step": 228
+    },
+    {
+      "epoch": 0.2816728167281673,
+      "grad_norm": 4.21238899230957,
+      "learning_rate": 1.926084840336821e-05,
+      "loss": 0.2075,
+      "step": 229
+    },
+    {
+      "epoch": 0.2829028290282903,
+      "grad_norm": 4.167908191680908,
+      "learning_rate": 1.9201349098645433e-05,
+      "loss": 0.1722,
+      "step": 230
+    },
+    {
+      "epoch": 0.28413284132841327,
+      "grad_norm": 3.7701351642608643,
+      "learning_rate": 1.9139646103877378e-05,
+      "loss": 0.1464,
+      "step": 231
+    },
+    {
+      "epoch": 0.28413284132841327,
+      "eval_loss": 0.10392418503761292,
+      "eval_runtime": 54.7341,
+      "eval_samples_per_second": 27.515,
+      "eval_steps_per_second": 0.219,
+      "eval_sts-test_pearson_cosine": 0.8822954324429473,
+      "eval_sts-test_pearson_dot": 0.8642863367586305,
+      "eval_sts-test_pearson_euclidean": 0.9122889538029727,
+      "eval_sts-test_pearson_manhattan": 0.912099304650421,
+      "eval_sts-test_pearson_max": 0.9122889538029727,
+      "eval_sts-test_spearman_cosine": 0.9087387596175093,
+      "eval_sts-test_spearman_dot": 0.8704923178256567,
+      "eval_sts-test_spearman_euclidean": 0.9097842833373965,
+      "eval_sts-test_spearman_manhattan": 0.9095372563745162,
+      "eval_sts-test_spearman_max": 0.9097842833373965,
+      "step": 231
+    },
+    {
+      "epoch": 0.2853628536285363,
+      "grad_norm": 4.548703193664551,
+      "learning_rate": 1.9075754196709574e-05,
+      "loss": 0.2675,
+      "step": 232
+    },
+    {
+      "epoch": 0.2865928659286593,
+      "grad_norm": 5.041469573974609,
+      "learning_rate": 1.900968867902419e-05,
+      "loss": 0.2585,
+      "step": 233
+    },
+    {
+      "epoch": 0.2878228782287823,
+      "grad_norm": 3.0036237239837646,
+      "learning_rate": 1.894146537327533e-05,
+      "loss": 0.134,
+      "step": 234
+    },
+    {
+      "epoch": 0.2890528905289053,
+      "grad_norm": 3.6082603931427,
+      "learning_rate": 1.8871100618699553e-05,
+      "loss": 0.1765,
+      "step": 235
+    },
+    {
+      "epoch": 0.2902829028290283,
+      "grad_norm": 3.8336241245269775,
+      "learning_rate": 1.8798611267402745e-05,
+      "loss": 0.1826,
+      "step": 236
+    },
+    {
+      "epoch": 0.2915129151291513,
+      "grad_norm": 4.307932376861572,
+      "learning_rate": 1.872401468032406e-05,
+      "loss": 0.222,
+      "step": 237
+    },
+    {
+      "epoch": 0.29274292742927427,
+      "grad_norm": 3.153963088989258,
+      "learning_rate": 1.864732872307804e-05,
+      "loss": 0.134,
+      "step": 238
+    },
+    {
+      "epoch": 0.2939729397293973,
+      "grad_norm": 4.044833660125732,
+      "learning_rate": 1.8568571761675893e-05,
+      "loss": 0.1902,
+      "step": 239
+    },
+    {
+      "epoch": 0.2952029520295203,
+      "grad_norm": 4.640310287475586,
+      "learning_rate": 1.8487762658126872e-05,
+      "loss": 0.2461,
+      "step": 240
+    },
+    {
+      "epoch": 0.2964329643296433,
+      "grad_norm": 4.932340145111084,
+      "learning_rate": 1.8404920765920898e-05,
+      "loss": 0.3094,
+      "step": 241
+    },
+    {
+      "epoch": 0.2976629766297663,
+      "grad_norm": 4.0233917236328125,
+      "learning_rate": 1.8320065925393468e-05,
+      "loss": 0.2252,
+      "step": 242
+    },
+    {
+      "epoch": 0.2988929889298893,
+      "grad_norm": 4.369536399841309,
+      "learning_rate": 1.8233218458973984e-05,
+      "loss": 0.2466,
+      "step": 243
+    },
+    {
+      "epoch": 0.3001230012300123,
+      "grad_norm": 3.6295106410980225,
+      "learning_rate": 1.814439916631857e-05,
+      "loss": 0.139,
+      "step": 244
+    },
+    {
+      "epoch": 0.3013530135301353,
+      "grad_norm": 3.705105781555176,
+      "learning_rate": 1.8053629319328662e-05,
+      "loss": 0.154,
+      "step": 245
+    },
+    {
+      "epoch": 0.3025830258302583,
+      "grad_norm": 3.7480130195617676,
+      "learning_rate": 1.796093065705644e-05,
+      "loss": 0.1979,
+      "step": 246
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 813,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 82,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 320,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-246/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cd608c44b8e7ee1bfc13e8632dd3ba8e3075b88a8d04aa48d7120c1f23659d10
+size 5752