Pascalymb
/

result_model

@@ -4,42 +4,56 @@ tags:
 - sentence-similarity
 - feature-extraction
 - generated_from_trainer
-- dataset_size:80
 - loss:CoSENTLoss
 base_model: abdeljalilELmajjodi/model
 widget:
-- source_sentence: A person on a horse jumps over a broken down airplane.
   sentences:
-  - A person is at a diner, ordering an omelette.
-  - The man with the sign is caucasian.
-  - A man in a restaurant is waiting for his meal to arrive.
-- source_sentence: Woman in white in foreground and a man slightly behind walking
-    with a sign for John's Pizza and Gyro in the background.
   sentences:
-  - The women enjoy having a good fashion sense.
-  - The man is sitting down while he has a sign for John's Pizza and Gyro in his arms.
-  - Olympic swimming.
-- source_sentence: A man and a woman cross the street in front of a pizza and gyro
-    restaurant.
   sentences:
-  - The people are standing still on the curb.
-  - two coworkers cross pathes on a street
-  - Two adults swimming in water
-- source_sentence: A few people in a restaurant setting, one of them is drinking orange
-    juice.
   sentences:
-  - The diners are at a restaurant.
-  - A family of three is at the mall shopping.
-  - The mans briefcase is for work.
-- source_sentence: Two adults, one female in white, with shades and one male, gray
-    clothes, walking across a street, away from a eatery with a blurred image of a
-    dark colored red shirted person in the foreground.
   sentences:
-  - Two women are at a restaurant drinking wine.
-  - A man and a woman walk down a crowded city street.
-  - Two adults walk across the street.
 pipeline_tag: sentence-similarity
 library_name: sentence-transformers
 ---
 # SentenceTransformer based on abdeljalilELmajjodi/model
@@ -92,9 +106,9 @@ from sentence_transformers import SentenceTransformer
 model = SentenceTransformer("sentence_transformers_model_id")
 # Run inference
 sentences = [
-    'Two adults, one female in white, with shades and one male, gray clothes, walking across a street, away from a eatery with a blurred image of a dark colored red shirted person in the foreground.',
-    'Two adults walk across the street.',
-    'Two women are at a restaurant drinking wine.',
 ]
 embeddings = model.encode(sentences)
 print(embeddings.shape)
@@ -103,9 +117,9 @@ print(embeddings.shape)
 # Get the similarity scores for the embeddings
 similarities = model.similarity(embeddings, embeddings)
 print(similarities)
-# tensor([[1.0000, 0.9900, 0.9921],
-#         [0.9900, 1.0000, 0.9920],
-#         [0.9921, 0.9920, 1.0000]])
 ```
 <!--
 ### Direct Usage (Transformers)
@@ -131,6 +145,20 @@ You can finetune this model on your own dataset.
 *List how the model may foreseeably be misused and address what users ought not to do with the model.*
 -->
 <!--
 ## Bias, Risks and Limitations
@@ -150,19 +178,19 @@ You can finetune this model on your own dataset.
 #### all-nli
 * Dataset: all-nli
-* Size: 80 training samples
 * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
-* Approximate statistics based on the first 80 samples:
-  |         | sentence1                                                                          | sentence2                                                                         | score                                                          |
-  |:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------|
-  | type    | string                                                                             | string                                                                            | float                                                          |
-  | details | <ul><li>min: 10 tokens</li><li>mean: 24.73 tokens</li><li>max: 52 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 11.85 tokens</li><li>max: 29 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.53</li><li>max: 1.0</li></ul> |
 * Samples:
-  | sentence1                                                                                                                            | sentence2                                                               | score            |
-  |:-------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------|:-----------------|
-  | <code>A man and a woman cross the street in front of a pizza and gyro restaurant.</code>                                             | <code>The couple are walking across the street together.</code>         | <code>0.5</code> |
-  | <code>A couple playing with a little boy on the beach.</code>                                                                        | <code>A couple watch a little girl play by herself on the beach.</code> | <code>0.0</code> |
-  | <code>Woman in white in foreground and a man slightly behind walking with a sign for John's Pizza and Gyro in the background.</code> | <code>The woman is wearing white.</code>                                | <code>1.0</code> |
 * Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
   ```json
   {
@@ -176,19 +204,19 @@ You can finetune this model on your own dataset.
 #### all-nli
 * Dataset: all-nli
-* Size: 20 evaluation samples
 * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
-* Approximate statistics based on the first 20 samples:
-  |         | sentence1                                                                          | sentence2                                                                        | score                                                          |
-  |:--------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------|
-  | type    | string                                                                             | string                                                                           | float                                                          |
-  | details | <ul><li>min: 15 tokens</li><li>mean: 29.75 tokens</li><li>max: 52 tokens</li></ul> | <ul><li>min: 5 tokens</li><li>mean: 12.5 tokens</li><li>max: 24 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.42</li><li>max: 1.0</li></ul> |
 * Samples:
-  | sentence1                                                                                                                                                                                                      | sentence2                                        | score            |
-  |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------|:-----------------|
-  | <code>Two adults, one female in white, with shades and one male, gray clothes, walking across a street, away from a eatery with a blurred image of a dark colored red shirted person in the foreground.</code> | <code>Two adults walking across a road</code>    | <code>1.0</code> |
-  | <code>A woman is walking across the street eating a banana, while a man is following with his briefcase.</code>                                                                                                | <code>The mans briefcase is for work.</code>     | <code>0.5</code> |
-  | <code>Woman in white in foreground and a man slightly behind walking with a sign for John's Pizza and Gyro in the background.</code>                                                                           | <code>The man with the sign is caucasian.</code> | <code>0.5</code> |
 * Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
   ```json
   {
@@ -197,6 +225,145 @@ You can finetune this model on your own dataset.
   }
   ```
 ### Framework Versions
 - Python: 3.12.13
 - Sentence Transformers: 5.4.1

 - sentence-similarity
 - feature-extraction
 - generated_from_trainer
+- dataset_size:16000
 - loss:CoSENTLoss
 base_model: abdeljalilELmajjodi/model
 widget:
+- source_sentence: A man and woman are walking in a restaurant that has signs in Chinese.
   sentences:
+  - A newlywed couple is walking through a Chinese restaurant.
+  - The woman is sitting on the ground.
+  - they are playing basketball
+- source_sentence: Several Asian cooks in a kitchen wearing white Dress Shirts.
   sentences:
+  - A group of women carry woven baskets and large red rugs as they walk down a street.
+  - A group of people is observing aquatic life.
+  - Severl cooks are in a kitchen.
+- source_sentence: An Asian fish market with fish being cut up for sale.
   sentences:
+  - A man in a blue and white shirt buys fish.
+  - Fish are being sold.
+  - The only amish person to own a cellphone.
+- source_sentence: An Asian man pushing a wheelchair.
   sentences:
+  - A surfer is in the water.
+  - There are at least four people.
+  - A girl sitting in a wheelchair waiting for a friend.
+- source_sentence: Seven people are wading in a natural pool.
   sentences:
+  - there were five people
+  - A man is standing alone.
+  - The man is riding a unicycle.
 pipeline_tag: sentence-similarity
 library_name: sentence-transformers
+metrics:
+- pearson_cosine
+- spearman_cosine
+model-index:
+- name: SentenceTransformer based on abdeljalilELmajjodi/model
+  results:
+  - task:
+      type: semantic-similarity
+      name: Semantic Similarity
+    dataset:
+      name: pair score evaluator dev
+      type: pair-score-evaluator-dev
+    metrics:
+    - type: pearson_cosine
+      value: 0.6451319585193018
+      name: Pearson Cosine
+    - type: spearman_cosine
+      value: 0.6628169159235177
+      name: Spearman Cosine
 ---
 # SentenceTransformer based on abdeljalilELmajjodi/model
 model = SentenceTransformer("sentence_transformers_model_id")
 # Run inference
 sentences = [
+    'Seven people are wading in a natural pool.',
+    'there were five people',
+    'The man is riding a unicycle.',
 ]
 embeddings = model.encode(sentences)
 print(embeddings.shape)
 # Get the similarity scores for the embeddings
 similarities = model.similarity(embeddings, embeddings)
 print(similarities)
+# tensor([[1.0000, 0.7814, 0.6510],
+#         [0.7814, 1.0000, 0.7047],
+#         [0.6510, 0.7047, 1.0000]])
 ```
 <!--
 ### Direct Usage (Transformers)
 *List how the model may foreseeably be misused and address what users ought not to do with the model.*
 -->
+## Evaluation
+### Metrics
+#### Semantic Similarity
+* Dataset: `pair-score-evaluator-dev`
+* Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.sentence_transformer.evaluation.EmbeddingSimilarityEvaluator)
+| Metric              | Value      |
+|:--------------------|:-----------|
+| pearson_cosine      | 0.6451     |
+| **spearman_cosine** | **0.6628** |
 <!--
 ## Bias, Risks and Limitations
 #### all-nli
 * Dataset: all-nli
+* Size: 16,000 training samples
 * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | sentence1                                                                         | sentence2                                                                         | score                                                          |
+  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                            | float                                                          |
+  | details | <ul><li>min: 7 tokens</li><li>mean: 18.91 tokens</li><li>max: 66 tokens</li></ul> | <ul><li>min: 4 tokens</li><li>mean: 11.42 tokens</li><li>max: 36 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.53</li><li>max: 1.0</li></ul> |
 * Samples:
+  | sentence1                                                                                    | sentence2                                             | score            |
+  |:---------------------------------------------------------------------------------------------|:------------------------------------------------------|:-----------------|
+  | <code>A man in a white shirt is on a rooftop lifting a board.</code>                         | <code>A man has a white shirt.</code>                 | <code>1.0</code> |
+  | <code>Two men in shorts and sandals are carrying food and drinks at a farmers market.</code> | <code>Men shopping their local market.</code>         | <code>1.0</code> |
+  | <code>A woman with her head down is in a very run down area.</code>                          | <code>the man in a suit talks on his cellphone</code> | <code>0.0</code> |
 * Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
   ```json
   {
 #### all-nli
 * Dataset: all-nli
+* Size: 4,000 evaluation samples
 * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | sentence1                                                                         | sentence2                                                                         | score                                                          |
+  |:--------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                            | float                                                          |
+  | details | <ul><li>min: 7 tokens</li><li>mean: 19.28 tokens</li><li>max: 70 tokens</li></ul> | <ul><li>min: 5 tokens</li><li>mean: 11.46 tokens</li><li>max: 46 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.49</li><li>max: 1.0</li></ul> |
 * Samples:
+  | sentence1                                                                                                            | sentence2                                                             | score            |
+  |:---------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------|:-----------------|
+  | <code>A person dressed in natural clothing taking their picture in a mirror.</code>                                  | <code>the person is nude at the bay</code>                            | <code>0.0</code> |
+  | <code>A basketball team of 8 girls is doing a hand huddle.</code>                                                    | <code>An all girls basketball team gets ready to start a game.</code> | <code>1.0</code> |
+  | <code>A person in a tan and blue sweater hanging clothes on a clothesline outside the window of her building.</code> | <code>There are no clothes.</code>                                    | <code>0.0</code> |
 * Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
   ```json
   {
   }
   ```
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+- `per_device_eval_batch_size`: 16
+- `gradient_accumulation_steps`: 4
+- `learning_rate`: 2e-05
+- `num_train_epochs`: 4
+- `warmup_steps`: 0.05
+- `bf16`: True
+- `bf16_full_eval`: True
+- `dataloader_num_workers`: 4
+- `load_best_model_at_end`: True
+- `push_to_hub`: True
+- `gradient_checkpointing`: True
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+- `do_predict`: False
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 8
+- `per_device_eval_batch_size`: 16
+- `gradient_accumulation_steps`: 4
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 2e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 4
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: None
+- `warmup_ratio`: None
+- `warmup_steps`: 0.05
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `enable_jit_checkpoint`: False
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `use_cpu`: False
+- `seed`: 42
+- `data_seed`: None
+- `bf16`: True
+- `fp16`: False
+- `bf16_full_eval`: True
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: -1
+- `ddp_backend`: None
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 4
+- `dataloader_prefetch_factor`: None
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: True
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `parallelism_config`: None
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch_fused
+- `optim_args`: None
+- `group_by_length`: False
+- `length_column_name`: length
+- `project`: huggingface
+- `trackio_space_id`: trackio
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `push_to_hub`: True
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `hub_revision`: None
+- `gradient_checkpointing`: True
+- `gradient_checkpointing_kwargs`: None
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `include_num_input_tokens_seen`: no
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `liger_kernel_config`: None
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: True
+- `use_cache`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: proportional
+- `router_mapping`: {}
+- `learning_rate_mapping`: {}
+</details>
+### Training Logs
+| Epoch   | Step     | Training Loss | Validation Loss | pair-score-evaluator-dev_spearman_cosine |
+|:-------:|:--------:|:-------------:|:---------------:|:----------------------------------------:|
+| -1      | -1       | -             | -               | 0.1511                                   |
+| 0.002   | 1        | 3.0124        | -               | -                                        |
+| 0.4     | 200      | 2.8949        | -               | -                                        |
+| 0.8     | 400      | 2.7600        | -               | -                                        |
+| 1.2     | 600      | 2.6245        | -               | -                                        |
+| 1.6     | 800      | 2.5053        | -               | -                                        |
+| 2.0     | 1000     | 2.4926        | -               | -                                        |
+| 2.4     | 1200     | 2.2290        | -               | -                                        |
+| 2.8     | 1400     | 2.1972        | -               | -                                        |
+| 3.2     | 1600     | 2.0796        | -               | -                                        |
+| 3.6     | 1800     | 1.9814        | -               | -                                        |
+| **4.0** | **2000** | **1.9559**    | **4.461**       | **0.6628**                               |
+| -1      | -1       | -             | -               | 0.6628                                   |
+* The bold row denotes the saved checkpoint.
+### Training Time
+- **Training**: 1.6 hours
 ### Framework Versions
 - Python: 3.12.13
 - Sentence Transformers: 5.4.1