mpnet-base-all-mqp-binary

README.md

@@ -5,7 +5,7 @@ tags:
 - feature-extraction
 - generated_from_trainer
 - dataset_size:2437
-- loss:CoSENTLoss
+- loss:ContrastiveLoss
 base_model: sentence-transformers/all-mpnet-base-v2
 widget:
 - source_sentence: I am having troubles and confusing moments with my body and I am
@@ -49,6 +49,45 @@ widget:
   - What causes stringy and shredded stools?
 pipeline_tag: sentence-similarity
 library_name: sentence-transformers
+metrics:
+- cosine_accuracy
+- cosine_accuracy_threshold
+- cosine_f1
+- cosine_f1_threshold
+- cosine_precision
+- cosine_recall
+- cosine_ap
+model-index:
+- name: SentenceTransformer based on sentence-transformers/all-mpnet-base-v2
+  results:
+  - task:
+      type: binary-classification
+      name: Binary Classification
+    dataset:
+      name: all mqp test
+      type: all-mqp-test
+    metrics:
+    - type: cosine_accuracy
+      value: 0.8786885245901639
+      name: Cosine Accuracy
+    - type: cosine_accuracy_threshold
+      value: 0.7678120136260986
+      name: Cosine Accuracy Threshold
+    - type: cosine_f1
+      value: 0.8796147672552167
+      name: Cosine F1
+    - type: cosine_f1_threshold
+      value: 0.7446306943893433
+      name: Cosine F1 Threshold
+    - type: cosine_precision
+      value: 0.8810289389067524
+      name: Cosine Precision
+    - type: cosine_recall
+      value: 0.8782051282051282
+      name: Cosine Recall
+    - type: cosine_ap
+      value: 0.9474266832530879
+      name: Cosine Ap
 ---
 
 # SentenceTransformer based on sentence-transformers/all-mpnet-base-v2
@@ -139,6 +178,25 @@ You can finetune this model on your own dataset.
 *List how the model may foreseeably be misused and address what users ought not to do with the model.*
 -->
 
+## Evaluation
+
+### Metrics
+
+#### Binary Classification
+
+* Dataset: `all-mqp-test`
+* Evaluated with [<code>BinaryClassificationEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.BinaryClassificationEvaluator)
+
+| Metric                    | Value      |
+|:--------------------------|:-----------|
+| cosine_accuracy           | 0.8787     |
+| cosine_accuracy_threshold | 0.7678     |
+| cosine_f1                 | 0.8796     |
+| cosine_f1_threshold       | 0.7446     |
+| cosine_precision          | 0.881      |
+| cosine_recall             | 0.8782     |
+| **cosine_ap**             | **0.9474** |
+
 <!--
 ## Bias, Risks and Limitations
 
@@ -171,11 +229,12 @@ You can finetune this model on your own dataset.
   | <code>I discovered I get this weakness in my hand whenever I try to snap my fingers, slight pain runs across elbow and wrist?</code> | <code>When I try to snap my fingers there is weakness and pain across elbow and wrist? May I know what are the causes?</code> | <code>1</code> |
   | <code>If a mother has celiac should the daughter be tested?</code>                                                                   | <code>What is Celiac disease?</code>                                                                                          | <code>0</code> |
   | <code>Hi im 18 and I would like to know what I would use or take to get taller?</code>                                               | <code>Can growth hormone taken in minimal quantities increase height after 21 years in a male?</code>                         | <code>0</code> |
-* Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
+* Loss: [<code>ContrastiveLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#contrastiveloss) with these parameters:
   ```json
   {
-      "scale": 20.0,
-      "similarity_fct": "pairwise_cos_sim"
+      "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
+      "margin": 0.5,
+      "size_average": true
   }
   ```
 
@@ -197,11 +256,12 @@ You can finetune this model on your own dataset.
   | <code>Okay so i'm on bc and I have had sex (it hurts) i'm bleeding brown and my vagina hurts almost itchy but it hurts?</code>                             | <code>I noticed a brown discharge and itching in my vaginal area to the point that it hurts. I am also on birth control and have sexual intercourse. What do you think is causing this?</code>                                          | <code>1</code> |
   | <code>I've had body aches, blocked stuffy nose, headaches, pressure in my face and throat tightness and it feels dry for 6 months is it a bad cold?</code> | <code>For the last 6 months, I've noticed symptoms like body aches, stuffy nose, headaches, pressure sensation in the face, throat tightness and feels dry. Can a cold last this long or should I be looking for something else?</code> | <code>1</code> |
   | <code>Is there any way to stop my period for a little while without a prescription?</code>                                                                 | <code>Are there any natural ways to stop my period without having to visit a local doctor?</code>                                                                                                                                       | <code>1</code> |
-* Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
+* Loss: [<code>ContrastiveLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#contrastiveloss) with these parameters:
   ```json
   {
-      "scale": 20.0,
-      "similarity_fct": "pairwise_cos_sim"
+      "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
+      "margin": 0.5,
+      "size_average": true
   }
   ```
 
@@ -339,9 +399,10 @@ You can finetune this model on your own dataset.
 </details>
 
 ### Training Logs
-| Epoch  | Step | Training Loss | Validation Loss |
-|:------:|:----:|:-------------:|:---------------:|
-| 0.6536 | 100  | 2.7722        | 2.8215          |
+| Epoch  | Step | Training Loss | Validation Loss | all-mqp-test_cosine_ap |
+|:------:|:----:|:-------------:|:---------------:|:----------------------:|
+| 0.6536 | 100  | 0.0137        | 0.0135          | -                      |
+| 1.0    | 153  | -             | -               | 0.9474                 |
 
 
 ### Framework Versions
@@ -370,14 +431,17 @@ You can finetune this model on your own dataset.
 }
 ```
 
-#### CoSENTLoss
+#### ContrastiveLoss
 ```bibtex
-@online{kexuefm-8847,
-    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
-    author={Su Jianlin},
-    year={2022},
-    month={Jan},
-    url={https://kexue.fm/archives/8847},
+@inproceedings{hadsell2006dimensionality,
+    author={Hadsell, R. and Chopra, S. and LeCun, Y.},
+    booktitle={2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06)},
+    title={Dimensionality Reduction by Learning an Invariant Mapping},
+    year={2006},
+    volume={2},
+    number={},
+    pages={1735-1742},
+    doi={10.1109/CVPR.2006.100}
 }
 ```
 

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cd2e9b97c7846d73e8557d1b464f9b664fc209ba757c70625369243f1968de3d
+oid sha256:e7293fe1746e1badcf524f725bf62ee5a361bbe7301987361cd0450c9a4c0784
 size 437967672

runs/Feb06_09-20-14_97946eed5a20/events.out.tfevents.1738833655.97946eed5a20.601.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:74f4f420c61699f175ff358dfb46426c07196af861eb9e864c446de9bb012b76
-size 4900
+oid sha256:6aad1eba4732e71463a576290c36dde2d3ed41da80351580cbb3f5d60685cbf6
+size 5254