Add new SentenceTransformer model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+    "word_embedding_dimension": 768,
+    "pooling_mode_cls_token": false,
+    "pooling_mode_mean_tokens": true,
+    "pooling_mode_max_tokens": false,
+    "pooling_mode_mean_sqrt_len_tokens": false,
+    "pooling_mode_weightedmean_tokens": false,
+    "pooling_mode_lasttoken": false,
+    "include_prompt": true
+}

README.md

@@ -0,0 +1,601 @@
+---
+language:
+- en
+license: apache-2.0
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- dense
+- generated_from_trainer
+- dataset_size:90000
+- loss:CachedMultipleNegativesRankingLoss
+base_model: microsoft/mpnet-base
+widget:
+- source_sentence: what is the difference between trojan virus and worm?
+  sentences:
+  - Worms spread from computer to computer, but unlike a virus, it has the capability
+    to travel without any help from a person. ... A Trojan horse is not a virus. It
+    is a destructive program that looks as a genuine application. Unlike viruses,
+    Trojan horses do not replicate themselves but they can be just as destructive.
+  - You're usually no longer infectious 24 hours after starting a course of antibiotics,
+    but this time period can sometimes vary. For example, the antibiotics may take
+    longer to work if your body takes longer to absorb them, or if you're taking other
+    medicine that interacts with the antibiotics.
+  - Eating salt raises the amount of sodium in your bloodstream and wrecks the delicate
+    balance, reducing the ability of your kidneys to remove the water. The result
+    is a higher blood pressure due to the extra fluid and extra strain on the delicate
+    blood vessels leading to the kidneys.
+- source_sentence: which are the neighbouring countries of pakistan?
+  sentences:
+  - Pakistan is bordered by India on the east, the Arabian Sea on the south, Iran
+    on the southwest, and Afghanistan on the west and north; in the northeast is the
+    disputed territory (with India) of Kashmir, of which the part occupied by Pakistan
+    borders on China.
+  - Age is a big factor when it comes to how much sleep a dog needs. Just as human
+    babies need a lot of sleep, the AKC notes your puppy needs 15-20 hours of sleep
+    a day to help his central nervous system, immune system and muscles develop properly.
+  - 'Step 1: Connect your iPhone to your computer using n USB cable through any of
+    the USB ports available on your computer. Step 2: Open iTunes, click the "Files"
+    tab and check the boxes to sync or transfer your files. Step 3: Select your desired
+    destination folder for the files and click "Sync" to complete the transfer.'
+- source_sentence: what can you do with 1gb of data?
+  sentences:
+  - You could even contact your email provider, complain that somebody else is using
+    your email address, and say that you are worried about your account being compromised.
+    They're very unlikely to do anything, but if something goes wrong, at least you
+    can prove you forewarned them.
+  - 1) Under Section 80CCD(1), investment in Atal Pension Yojana or NPS up to ₹ 1.5
+    lakh qualifies for income tax deduction. But remember that the total amount of
+    deduction under sections 80C, 80CCC and 80CCD cannot exceed ₹ 1.5 lakh.
+  - 1GB (or 1024MB) of data lets you send or receive about 1,000 emails and browse
+    the Internet for about 20 hours every month. (This limit relates only to your
+    1GB mobile data allocation; if you are an 'inclusive mobile broadband customer'
+    you also get 2000 BT Wi-fi wi-fi minutes every month.)
+- source_sentence: how many carbon atoms are in carbon dioxide?
+  sentences:
+  - For CO2 there is one atom of carbon and two atoms of oxygen. For H2O, there is
+    one atom of oxygen and two atoms of hydrogen. A molecule can be made of only one
+    type of atom.
+  - Avian influenza refers to the disease caused by infection with avian (bird) influenza
+    (flu) Type A viruses. These viruses occur naturally among wild aquatic birds worldwide
+    and can infect domestic poultry and other bird and animal species. Avian flu viruses
+    do not normally infect humans.
+  - At the end of "Inception," Dom Cobb (Leonardo DiCaprio) finally returns home to
+    his kids after spending a long time in the dream world. Cobb carries a little
+    top with him. If the top keeps spinning, that means he is in a dream. ... The
+    final shot shows the top spinning, but it never reveals whether it falls over.
+- source_sentence: is duchenne muscular dystrophy a dominant or recessive trait?
+  sentences:
+  - Duchenne muscular dystrophy is inherited in an X-linked recessive pattern. Males
+    have only one copy of the X chromosome from their mother and one copy of the Y
+    chromosome from their father. If their X chromosome has a DMD gene mutation, they
+    will have Duchenne muscular dystrophy.
+  - An automatic transmission will downshift for you when you drive uphill. However,
+    for moderately steep slopes, it's wise to shift to the gear range marked D2, 2,
+    or L to ascend and descend the hill. For steep slopes that you can't ascend at
+    a speed faster than 10 mph (about 15 kph), shift to D1 or 1.
+  - The dream suggests captivity and it refers to your fear of punishment. Another
+    interpretation of this dream refers to a need to do what you feel is right in
+    waking life. Being in jail suggests that your feelings may be trapped by a limited
+    mind and body. ... Jail also suggests repressed feelings.
+datasets:
+- sentence-transformers/gooaq
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+metrics:
+- cosine_accuracy@1
+- cosine_accuracy@3
+- cosine_accuracy@5
+- cosine_accuracy@10
+- cosine_precision@1
+- cosine_precision@3
+- cosine_precision@5
+- cosine_precision@10
+- cosine_recall@1
+- cosine_recall@3
+- cosine_recall@5
+- cosine_recall@10
+- cosine_ndcg@10
+- cosine_mrr@10
+- cosine_map@100
+co2_eq_emissions:
+  emissions: 25.654063970832134
+  energy_consumed: 0.09585932386288172
+  source: codecarbon
+  training_type: fine-tuning
+  on_cloud: false
+  cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
+  ram_total_size: 31.777088165283203
+  hours_used: 0.27
+  hardware_used: 1 x NVIDIA GeForce RTX 3090
+model-index:
+- name: MPNet base trained on GooAQ triplets using CachedMultipleNegativesRankingLoss
+    with GradCache
+  results:
+  - task:
+      type: information-retrieval
+      name: Information Retrieval
+    dataset:
+      name: gooaq dev
+      type: gooaq-dev
+    metrics:
+    - type: cosine_accuracy@1
+      value: 0.7568
+      name: Cosine Accuracy@1
+    - type: cosine_accuracy@3
+      value: 0.8954
+      name: Cosine Accuracy@3
+    - type: cosine_accuracy@5
+      value: 0.9344
+      name: Cosine Accuracy@5
+    - type: cosine_accuracy@10
+      value: 0.9645
+      name: Cosine Accuracy@10
+    - type: cosine_precision@1
+      value: 0.7568
+      name: Cosine Precision@1
+    - type: cosine_precision@3
+      value: 0.2984666666666666
+      name: Cosine Precision@3
+    - type: cosine_precision@5
+      value: 0.18688000000000002
+      name: Cosine Precision@5
+    - type: cosine_precision@10
+      value: 0.09645
+      name: Cosine Precision@10
+    - type: cosine_recall@1
+      value: 0.7568
+      name: Cosine Recall@1
+    - type: cosine_recall@3
+      value: 0.8954
+      name: Cosine Recall@3
+    - type: cosine_recall@5
+      value: 0.9344
+      name: Cosine Recall@5
+    - type: cosine_recall@10
+      value: 0.9645
+      name: Cosine Recall@10
+    - type: cosine_ndcg@10
+      value: 0.8651990399153616
+      name: Cosine Ndcg@10
+    - type: cosine_mrr@10
+      value: 0.8328272619047579
+      name: Cosine Mrr@10
+    - type: cosine_map@100
+      value: 0.8345033162131785
+      name: Cosine Map@100
+---
+
+# MPNet base trained on GooAQ triplets using CachedMultipleNegativesRankingLoss with GradCache
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [microsoft/mpnet-base](https://huggingface.co/microsoft/mpnet-base) on the [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [microsoft/mpnet-base](https://huggingface.co/microsoft/mpnet-base) <!-- at revision 6996ce1e91bd2a9c7d7f61daec37463394f73f09 -->
+- **Maximum Sequence Length:** 512 tokens
+- **Output Dimensionality:** 768 dimensions
+- **Similarity Function:** Cosine Similarity
+- **Training Dataset:**
+    - [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq)
+- **Language:** en
+- **License:** apache-2.0
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'MPNetModel'})
+  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("tomaarsen/mpnet-base-gooaq-cmnrl-1024bs-GradCache")
+# Run inference
+queries = [
+    "is duchenne muscular dystrophy a dominant or recessive trait?",
+]
+documents = [
+    'Duchenne muscular dystrophy is inherited in an X-linked recessive pattern. Males have only one copy of the X chromosome from their mother and one copy of the Y chromosome from their father. If their X chromosome has a DMD gene mutation, they will have Duchenne muscular dystrophy.',
+    'The dream suggests captivity and it refers to your fear of punishment. Another interpretation of this dream refers to a need to do what you feel is right in waking life. Being in jail suggests that your feelings may be trapped by a limited mind and body. ... Jail also suggests repressed feelings.',
+    "An automatic transmission will downshift for you when you drive uphill. However, for moderately steep slopes, it's wise to shift to the gear range marked D2, 2, or L to ascend and descend the hill. For steep slopes that you can't ascend at a speed faster than 10 mph (about 15 kph), shift to D1 or 1.",
+]
+query_embeddings = model.encode_query(queries)
+document_embeddings = model.encode_document(documents)
+print(query_embeddings.shape, document_embeddings.shape)
+# [1, 768] [3, 768]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(query_embeddings, document_embeddings)
+print(similarities)
+# tensor([[0.8208, 0.1993, 0.1582]])
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Information Retrieval
+
+* Dataset: `gooaq-dev`
+* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
+
+| Metric              | Value      |
+|:--------------------|:-----------|
+| cosine_accuracy@1   | 0.7568     |
+| cosine_accuracy@3   | 0.8954     |
+| cosine_accuracy@5   | 0.9344     |
+| cosine_accuracy@10  | 0.9645     |
+| cosine_precision@1  | 0.7568     |
+| cosine_precision@3  | 0.2985     |
+| cosine_precision@5  | 0.1869     |
+| cosine_precision@10 | 0.0964     |
+| cosine_recall@1     | 0.7568     |
+| cosine_recall@3     | 0.8954     |
+| cosine_recall@5     | 0.9344     |
+| cosine_recall@10    | 0.9645     |
+| **cosine_ndcg@10**  | **0.8652** |
+| cosine_mrr@10       | 0.8328     |
+| cosine_map@100      | 0.8345     |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### gooaq
+
+* Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
+* Size: 90,000 training samples
+* Columns: <code>question</code> and <code>answer</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | question                                                                          | answer                                                                              |
+  |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                              |
+  | details | <ul><li>min: 8 tokens</li><li>mean: 11.83 tokens</li><li>max: 20 tokens</li></ul> | <ul><li>min: 15 tokens</li><li>mean: 60.45 tokens</li><li>max: 180 tokens</li></ul> |
+* Samples:
+  | question                                                            | answer                                                                                                                                                                                                                                                                                                                                       |
+  |:--------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>how long does halifax take to transfer mortgage funds?</code> | <code>Bear in mind that the speed of application will vary depending on your own personal circumstances and the lender's present day-to-day performance. In some cases, applications can be approved by the lender within 24 hours, while some can take weeks or even months.</code>                                                         |
+  | <code>can you get a false pregnancy test?</code>                    | <code>In very rare cases, you can have a false-positive result. This means you're not pregnant but the test says you are. You could have a false-positive result if you have blood or protein in your pee. Certain drugs, such as tranquilizers, anticonvulsants, hypnotics, and fertility drugs, could cause false-positive results.</code> |
+  | <code>are ahead of its time?</code>                                 | <code>Definition of ahead of one's/its time : too advanced or modern to be understood or appreciated during the time when one lives or works As a director, he was ahead of his time.</code>                                                                                                                                                 |
+* Loss: [<code>CachedMultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedmultiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "cos_sim",
+      "mini_batch_size": 64,
+      "gather_across_devices": false,
+      "use_cont_accum": false,
+      "cache_size": 0,
+      "prev_cache": false
+  }
+  ```
+
+### Evaluation Dataset
+
+#### gooaq
+
+* Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
+* Size: 10,000 evaluation samples
+* Columns: <code>question</code> and <code>answer</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | question                                                                          | answer                                                                              |
+  |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                              |
+  | details | <ul><li>min: 8 tokens</li><li>mean: 11.93 tokens</li><li>max: 25 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 60.84 tokens</li><li>max: 127 tokens</li></ul> |
+* Samples:
+  | question                                                         | answer                                                                                                                                                                                                                                                                                                                                                        |
+  |:-----------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>should you take ibuprofen with high blood pressure?</code> | <code>In general, people with high blood pressure should use acetaminophen or possibly aspirin for over-the-counter pain relief. Unless your health care provider has said it's OK, you should not use ibuprofen, ketoprofen, or naproxen sodium. If aspirin or acetaminophen doesn't help with your pain, call your doctor.</code>                           |
+  | <code>how old do you have to be to work in sc?</code>            | <code>The general minimum age of employment for South Carolina youth is 14, although the state allows younger children who are performers to work in show business. If their families are agricultural workers, children younger than age 14 may also participate in farm labor.</code>                                                                       |
+  | <code>how to write a topic proposal for a research paper?</code> | <code>['Write down the main topic of your paper. ... ', 'Write two or three short sentences under the main topic that explain why you chose that topic. ... ', 'Write a thesis sentence that states the angle and purpose of your research paper. ... ', 'List the items you will cover in the body of the paper that support your thesis statement.']</code> |
+* Loss: [<code>CachedMultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedmultiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "cos_sim",
+      "mini_batch_size": 64,
+      "gather_across_devices": false,
+      "use_cont_accum": false,
+      "cache_size": 0,
+      "prev_cache": false
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 1024
+- `per_device_eval_batch_size`: 1024
+- `learning_rate`: 8e-05
+- `num_train_epochs`: 1
+- `warmup_ratio`: 0.1
+- `bf16`: True
+- `batch_sampler`: no_duplicates
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 1024
+- `per_device_eval_batch_size`: 1024
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 8e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 1
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: None
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `bf16`: True
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `parallelism_config`: None
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch_fused
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `project`: huggingface
+- `trackio_space_id`: trackio
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `hub_revision`: None
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: no
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `liger_kernel_config`: None
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: True
+- `prompts`: None
+- `batch_sampler`: no_duplicates
+- `multi_dataset_batch_sampler`: proportional
+- `router_mapping`: {}
+- `learning_rate_mapping`: {}
+
+</details>
+
+### Training Logs
+| Epoch  | Step | Training Loss | Validation Loss | gooaq-dev_cosine_ndcg@10 |
+|:------:|:----:|:-------------:|:---------------:|:------------------------:|
+| -1     | -1   | -             | -               | 0.2154                   |
+| 0.0114 | 1    | 6.4064        | -               | -                        |
+| 0.0568 | 5    | 5.8799        | -               | -                        |
+| 0.1023 | 9    | -             | 1.1303          | 0.6342                   |
+| 0.1136 | 10   | 2.4927        | -               | -                        |
+| 0.1705 | 15   | 1.0794        | -               | -                        |
+| 0.2045 | 18   | -             | 0.5062          | 0.7918                   |
+| 0.2273 | 20   | 0.7681        | -               | -                        |
+| 0.2841 | 25   | 0.622         | -               | -                        |
+| 0.3068 | 27   | -             | 0.3818          | 0.8250                   |
+| 0.3409 | 30   | 0.5329        | -               | -                        |
+| 0.3977 | 35   | 0.4671        | -               | -                        |
+| 0.4091 | 36   | -             | 0.3244          | 0.8404                   |
+| 0.4545 | 40   | 0.4472        | -               | -                        |
+| 0.5114 | 45   | 0.4308        | 0.2978          | 0.8496                   |
+| 0.5682 | 50   | 0.4292        | -               | -                        |
+| 0.6136 | 54   | -             | 0.2803          | 0.8554                   |
+| 0.625  | 55   | 0.3973        | -               | -                        |
+| 0.6818 | 60   | 0.3789        | -               | -                        |
+| 0.7159 | 63   | -             | 0.2673          | 0.8601                   |
+| 0.7386 | 65   | 0.3477        | -               | -                        |
+| 0.7955 | 70   | 0.3725        | -               | -                        |
+| 0.8182 | 72   | -             | 0.2548          | 0.8632                   |
+| 0.8523 | 75   | 0.3541        | -               | -                        |
+| 0.9091 | 80   | 0.352         | -               | -                        |
+| 0.9205 | 81   | -             | 0.2498          | 0.8644                   |
+| 0.9659 | 85   | 0.339         | -               | -                        |
+| -1     | -1   | -             | -               | 0.8652                   |
+
+
+### Environmental Impact
+Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
+- **Energy Consumed**: 0.096 kWh
+- **Carbon Emitted**: 0.026 kg of CO2
+- **Hours Used**: 0.27 hours
+
+### Training Hardware
+- **On Cloud**: No
+- **GPU Model**: 1 x NVIDIA GeForce RTX 3090
+- **CPU Model**: 13th Gen Intel(R) Core(TM) i7-13700K
+- **RAM Size**: 31.78 GB
+
+### Framework Versions
+- Python: 3.11.6
+- Sentence Transformers: 5.3.0.dev0
+- Transformers: 4.57.6
+- PyTorch: 2.10.0+cu128
+- Accelerate: 1.6.0
+- Datasets: 4.5.0
+- Tokenizers: 0.22.2
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### CachedMultipleNegativesRankingLoss
+```bibtex
+@misc{gao2021scaling,
+    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
+    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
+    year={2021},
+    eprint={2101.06983},
+    archivePrefix={arXiv},
+    primaryClass={cs.LG}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,23 @@
+{
+  "architectures": [
+    "MPNetModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "dtype": "float32",
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "mpnet",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "relative_attention_num_buckets": 32,
+  "transformers_version": "4.57.6",
+  "vocab_size": 30527
+}

config_sentence_transformers.json

@@ -0,0 +1,14 @@
+{
+  "model_type": "SentenceTransformer",
+  "__version__": {
+    "sentence_transformers": "5.3.0.dev0",
+    "transformers": "4.57.6",
+    "pytorch": "2.10.0+cu128"
+  },
+  "prompts": {
+    "query": "",
+    "document": ""
+  },
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b07336e3fb4977ab89a0a0a50b6617e815f5206c5a473efdc4d0e0ef7aae7330
+size 437967672

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+    "max_seq_length": 512,
+    "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,51 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,66 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "104": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "30526": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<s>",
+  "do_lower_case": true,
+  "eos_token": "</s>",
+  "extra_special_tokens": {},
+  "mask_token": "<mask>",
+  "model_max_length": 512,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "MPNetTokenizer",
+  "unk_token": "[UNK]"
+}