Hugging Face's logo

mkiram
/
result_model

Sentence Similarity
sentence-transformers
Safetensors
xlm-roberta
feature-extraction
Generated from Trainer
dataset_size:80
loss:CoSENTLoss
Eval Results (legacy)
text-embeddings-inference
Model card Files
xet
 Community
mkiram commited on
Commit
eb4b0fc
·
verified ·
1 Parent(s): 8d11da0

Training in progress, step 10

Browse files
Files changed (4) hide show
  1. README.md +68 -73
  2. eval/similarity_evaluation_pair-score-evaluator-dev_results.csv +1 -0
  3. model.safetensors +1 -1
  4. training_args.bin +2 -2
README.md CHANGED
@@ -8,37 +8,35 @@ tags:
8
  - loss:CoSENTLoss
9
  base_model: abdeljalilELmajjodi/model
10
  widget:
11
- - source_sentence: A woman is walking across the street eating a banana, while a man
12
- is following with his briefcase.
13
  sentences:
14
- - A woman eats ice cream walking down the sidewalk, and there is another woman in
15
- front of her with a purse.
16
- - A man in a restaurant is waiting for his meal to arrive.
17
- - Nobody has food.
18
- - source_sentence: Two adults, one female in white, with shades and one male, gray
19
- clothes, walking across a street, away from a eatery with a blurred image of a
20
- dark colored red shirted person in the foreground.
21
  sentences:
22
- - An elderly man sits in a small shop.
23
- - They are protesting outside the capital.
24
- - Two adults walking across a road
25
- - source_sentence: A few people in a restaurant setting, one of them is drinking orange
26
- juice.
27
  sentences:
28
- - The diners are at a restaurant.
29
- - The boy skates down the sidewalk.
30
- - The people are sitting at desks in school.
31
- - source_sentence: A man with blond-hair, and a brown shirt drinking out of a public
32
- water fountain.
33
  sentences:
34
- - A blond man getting a drink of water from a fountain in the park.
35
- - A female is next to a man.
36
- - The woman and man are outdoors.
37
- - source_sentence: Two women who just had lunch hugging and saying goodbye.
 
 
38
  sentences:
 
39
  - A man and a woman walk down a crowded city street.
40
- - The couple are walking across the street together.
41
- - There are two woman in this picture.
42
  pipeline_tag: sentence-similarity
43
  library_name: sentence-transformers
44
  metrics:
@@ -55,10 +53,10 @@ model-index:
55
  type: pair-score-evaluator-dev
56
  metrics:
57
  - type: pearson_cosine
58
- value: -0.19455064165452754
59
  name: Pearson Cosine
60
  - type: spearman_cosine
61
- value: -0.23640181564913884
62
  name: Spearman Cosine
63
  ---
64
 
@@ -109,12 +107,12 @@ Then you can load this model and run inference.
109
  from sentence_transformers import SentenceTransformer
110
 
111
  # Download from the 🤗 Hub
112
- model = SentenceTransformer("sentence_transformers_model_id")
113
  # Run inference
114
  sentences = [
115
- 'Two women who just had lunch hugging and saying goodbye.',
116
- 'There are two woman in this picture.',
117
- 'A man and a woman walk down a crowded city street.',
118
  ]
119
  embeddings = model.encode(sentences)
120
  print(embeddings.shape)
@@ -123,9 +121,9 @@ print(embeddings.shape)
123
  # Get the similarity scores for the embeddings
124
  similarities = model.similarity(embeddings, embeddings)
125
  print(similarities)
126
- # tensor([[1.0000, 0.9875, 0.9876],
127
- # [0.9875, 1.0000, 0.9877],
128
- # [0.9876, 0.9877, 1.0000]])
129
  ```
130
  <!--
131
  ### Direct Usage (Transformers)
@@ -160,10 +158,10 @@ You can finetune this model on your own dataset.
160
  * Dataset: `pair-score-evaluator-dev`
161
  * Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.sentence_transformer.evaluation.EmbeddingSimilarityEvaluator)
162
 
163
- | Metric | Value |
164
- |:--------------------|:------------|
165
- | pearson_cosine | -0.1946 |
166
- | **spearman_cosine** | **-0.2364** |
167
 
168
  <!--
169
  ## Bias, Risks and Limitations
@@ -187,17 +185,17 @@ You can finetune this model on your own dataset.
187
  * Size: 80 training samples
188
  * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
189
  * Approximate statistics based on the first 80 samples:
190
- | | sentence1 | sentence2 | score |
191
- |:---------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------|
192
- | type | string | string | float |
193
- | modality | text | text | |
194
- | details | <ul><li>min: 10 tokens</li><li>mean: 25.19 tokens</li><li>max: 52 tokens</li></ul> | <ul><li>min: 5 tokens</li><li>mean: 12.0 tokens</li><li>max: 29 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.49</li><li>max: 1.0</li></ul> |
195
  * Samples:
196
- | sentence1 | sentence2 | score |
197
- |:----------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------|:-----------------|
198
- | <code>Two women, holding food carryout containers, hug.</code> | <code>The two sisters saw each other across the crowded diner and shared a hug, both clutching their doggie bags.</code> | <code>0.5</code> |
199
- | <code>People waiting to get on a train or just getting off.</code> | <code>There are people just getting on a train</code> | <code>1.0</code> |
200
- | <code>Two women who just had lunch hugging and saying goodbye.</code> | <code>The friends have just met for the first time in 20 years, and have had a great time catching up.</code> | <code>0.5</code> |
201
  * Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
202
  ```json
203
  {
@@ -214,17 +212,17 @@ You can finetune this model on your own dataset.
214
  * Size: 20 evaluation samples
215
  * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
216
  * Approximate statistics based on the first 20 samples:
217
- | | sentence1 | sentence2 | score |
218
- |:---------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------|
219
- | type | string | string | float |
220
- | modality | text | text | |
221
- | details | <ul><li>min: 10 tokens</li><li>mean: 27.9 tokens</li><li>max: 52 tokens</li></ul> | <ul><li>min: 6 tokens</li><li>mean: 11.9 tokens</li><li>max: 29 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.57</li><li>max: 1.0</li></ul> |
222
  * Samples:
223
- | sentence1 | sentence2 | score |
224
- |:----------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------|:-----------------|
225
- | <code>A woman is walking across the street eating a banana, while a man is following with his briefcase.</code> | <code>A woman eats ice cream walking down the sidewalk, and there is another woman in front of her with a purse.</code> | <code>0.0</code> |
226
- | <code>Two women who just had lunch hugging and saying goodbye.</code> | <code>There are two woman in this picture.</code> | <code>1.0</code> |
227
- | <code>A woman wearing all white and eating, walks next to a man holding a briefcase.</code> | <code>A female is next to a man.</code> | <code>1.0</code> |
228
  * Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
229
  ```json
230
  {
@@ -238,11 +236,10 @@ You can finetune this model on your own dataset.
238
 
239
  - `num_train_epochs`: 1
240
  - `warmup_steps`: 0.05
241
- - `bf16`: True
242
- - `fp16_full_eval`: True
243
  - `load_best_model_at_end`: True
244
  - `push_to_hub`: True
245
- - `gradient_checkpointing`: True
246
 
247
  #### All Hyperparameters
248
  <details><summary>Click to expand</summary>
@@ -277,10 +274,10 @@ You can finetune this model on your own dataset.
277
  - `use_cpu`: False
278
  - `seed`: 42
279
  - `data_seed`: None
280
- - `bf16`: True
281
- - `fp16`: False
282
  - `bf16_full_eval`: False
283
- - `fp16_full_eval`: True
284
  - `tf32`: None
285
  - `local_rank`: -1
286
  - `ddp_backend`: None
@@ -313,12 +310,12 @@ You can finetune this model on your own dataset.
313
  - `skip_memory_metrics`: True
314
  - `push_to_hub`: True
315
  - `resume_from_checkpoint`: None
316
- - `hub_model_id`: None
317
  - `hub_strategy`: every_save
318
  - `hub_private_repo`: None
319
  - `hub_always_push`: False
320
  - `hub_revision`: None
321
- - `gradient_checkpointing`: True
322
  - `gradient_checkpointing_kwargs`: None
323
  - `include_for_metrics`: []
324
  - `eval_do_concat_batches`: True
@@ -347,16 +344,14 @@ You can finetune this model on your own dataset.
347
  </details>
348
 
349
  ### Training Logs
350
- | Epoch | Step | Training Loss | Validation Loss | pair-score-evaluator-dev_spearman_cosine |
351
- |:-------:|:------:|:-------------:|:---------------:|:----------------------------------------:|
352
- | 0.1 | 1 | 3.0474 | - | - |
353
- | 0.5 | 5 | 2.8526 | - | - |
354
- | **1.0** | **10** | **2.7798** | **2.6701** | **-0.2364** |
355
 
356
- * The bold row denotes the saved checkpoint.
357
 
358
  ### Training Time
359
- - **Training**: 3.1 minutes
360
 
361
  ### Framework Versions
362
  - Python: 3.12.13
 
8
  - loss:CoSENTLoss
9
  base_model: abdeljalilELmajjodi/model
10
  widget:
11
+ - source_sentence: Woman in white in foreground and a man slightly behind walking
12
+ with a sign for John's Pizza and Gyro in the background.
13
  sentences:
14
+ - A woman ordering pizza.
15
+ - The people are eating omelettes.
16
+ - Some people board a train.
17
+ - source_sentence: Children smiling and waving at camera
 
 
 
18
  sentences:
19
+ - Two groups of rival gang members flipped each other off.
20
+ - A blond man drinking water from a fountain.
21
+ - They are smiling at their parents
22
+ - source_sentence: Two women, holding food carryout containers, hug.
 
23
  sentences:
24
+ - Two women hug each other.
25
+ - The women do not care what clothes they wear.
26
+ - Two people walk home after a tasty steak dinner.
27
+ - source_sentence: The school is having a special event in order to show the american
28
+ culture on how other cultures are dealt with in parties.
29
  sentences:
30
+ - A high school is hosting an event.
31
+ - The people are sitting at desks in school.
32
+ - An older man drinks his juice as he waits for his daughter to get off work.
33
+ - source_sentence: Two adults, one female in white, with shades and one male, gray
34
+ clothes, walking across a street, away from a eatery with a blurred image of a
35
+ dark colored red shirted person in the foreground.
36
  sentences:
37
+ - The women are sleeping.
38
  - A man and a woman walk down a crowded city street.
39
+ - Two adults walking across a road near the convicted prisoner dressed in red
 
40
  pipeline_tag: sentence-similarity
41
  library_name: sentence-transformers
42
  metrics:
 
53
  type: pair-score-evaluator-dev
54
  metrics:
55
  - type: pearson_cosine
56
+ value: 0.2699926667525232
57
  name: Pearson Cosine
58
  - type: spearman_cosine
59
+ value: 0.26478786196946036
60
  name: Spearman Cosine
61
  ---
62
 
 
107
  from sentence_transformers import SentenceTransformer
108
 
109
  # Download from the 🤗 Hub
110
+ model = SentenceTransformer("mkiram/result_model")
111
  # Run inference
112
  sentences = [
113
+ 'Two adults, one female in white, with shades and one male, gray clothes, walking across a street, away from a eatery with a blurred image of a dark colored red shirted person in the foreground.',
114
+ 'Two adults walking across a road near the convicted prisoner dressed in red',
115
+ 'The women are sleeping.',
116
  ]
117
  embeddings = model.encode(sentences)
118
  print(embeddings.shape)
 
121
  # Get the similarity scores for the embeddings
122
  similarities = model.similarity(embeddings, embeddings)
123
  print(similarities)
124
+ # tensor([[1.0000, 0.9940, 0.9884],
125
+ # [0.9940, 1.0000, 0.9837],
126
+ # [0.9884, 0.9837, 1.0000]])
127
  ```
128
  <!--
129
  ### Direct Usage (Transformers)
 
158
  * Dataset: `pair-score-evaluator-dev`
159
  * Evaluated with [<code>EmbeddingSimilarityEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.sentence_transformer.evaluation.EmbeddingSimilarityEvaluator)
160
 
161
+ | Metric | Value |
162
+ |:--------------------|:-----------|
163
+ | pearson_cosine | 0.27 |
164
+ | **spearman_cosine** | **0.2648** |
165
 
166
  <!--
167
  ## Bias, Risks and Limitations
 
185
  * Size: 80 training samples
186
  * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
187
  * Approximate statistics based on the first 80 samples:
188
+ | | sentence1 | sentence2 | score |
189
+ |:---------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:---------------------------------------------------------------|
190
+ | type | string | string | float |
191
+ | modality | text | text | |
192
+ | details | <ul><li>min: 10 tokens</li><li>mean: 24.8 tokens</li><li>max: 52 tokens</li></ul> | <ul><li>min: 5 tokens</li><li>mean: 12.1 tokens</li><li>max: 29 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.53</li><li>max: 1.0</li></ul> |
193
  * Samples:
194
+ | sentence1 | sentence2 | score |
195
+ |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------|:-----------------|
196
+ | <code>A man and a woman cross the street in front of a pizza and gyro restaurant.</code> | <code>The people are standing still on the curb.</code> | <code>0.0</code> |
197
+ | <code>Two adults, one female in white, with shades and one male, gray clothes, walking across a street, away from a eatery with a blurred image of a dark colored red shirted person in the foreground.</code> | <code>Two adults walk across the street to get away from a red shirted person who is chasing them.</code> | <code>0.5</code> |
198
+ | <code>A couple playing with a little boy on the beach.</code> | <code>A couple are playing frisbee with a young child at the beach.</code> | <code>0.5</code> |
199
  * Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
200
  ```json
201
  {
 
212
  * Size: 20 evaluation samples
213
  * Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>score</code>
214
  * Approximate statistics based on the first 20 samples:
215
+ | | sentence1 | sentence2 | score |
216
+ |:---------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------|:--------------------------------------------------------------|
217
+ | type | string | string | float |
218
+ | modality | text | text | |
219
+ | details | <ul><li>min: 10 tokens</li><li>mean: 29.45 tokens</li><li>max: 52 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 11.5 tokens</li><li>max: 21 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.4</li><li>max: 1.0</li></ul> |
220
  * Samples:
221
+ | sentence1 | sentence2 | score |
222
+ |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------|:-----------------|
223
+ | <code>Two adults, one female in white, with shades and one male, gray clothes, walking across a street, away from a eatery with a blurred image of a dark colored red shirted person in the foreground.</code> | <code>The people are related.</code> | <code>0.5</code> |
224
+ | <code>A woman wearing all white and eating, walks next to a man holding a briefcase.</code> | <code>A female is next to a man.</code> | <code>1.0</code> |
225
+ | <code>Two adults, one female in white, with shades and one male, gray clothes, walking across a street, away from a eatery with a blurred image of a dark colored red shirted person in the foreground.</code> | <code>Two people walk home after a tasty steak dinner.</code> | <code>0.5</code> |
226
  * Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
227
  ```json
228
  {
 
236
 
237
  - `num_train_epochs`: 1
238
  - `warmup_steps`: 0.05
239
+ - `fp16`: True
 
240
  - `load_best_model_at_end`: True
241
  - `push_to_hub`: True
242
+ - `hub_model_id`: mkiram/result_model
243
 
244
  #### All Hyperparameters
245
  <details><summary>Click to expand</summary>
 
274
  - `use_cpu`: False
275
  - `seed`: 42
276
  - `data_seed`: None
277
+ - `bf16`: False
278
+ - `fp16`: True
279
  - `bf16_full_eval`: False
280
+ - `fp16_full_eval`: False
281
  - `tf32`: None
282
  - `local_rank`: -1
283
  - `ddp_backend`: None
 
310
  - `skip_memory_metrics`: True
311
  - `push_to_hub`: True
312
  - `resume_from_checkpoint`: None
313
+ - `hub_model_id`: mkiram/result_model
314
  - `hub_strategy`: every_save
315
  - `hub_private_repo`: None
316
  - `hub_always_push`: False
317
  - `hub_revision`: None
318
+ - `gradient_checkpointing`: False
319
  - `gradient_checkpointing_kwargs`: None
320
  - `include_for_metrics`: []
321
  - `eval_do_concat_batches`: True
 
344
  </details>
345
 
346
  ### Training Logs
347
+ | Epoch | Step | Training Loss | Validation Loss | pair-score-evaluator-dev_spearman_cosine |
348
+ |:-----:|:----:|:-------------:|:---------------:|:----------------------------------------:|
349
+ | 0.5 | 5 | 3.0564 | - | - |
350
+ | 1.0 | 10 | 3.0411 | 2.7269 | 0.2648 |
 
351
 
 
352
 
353
  ### Training Time
354
+ - **Training**: 27.5 seconds
355
 
356
  ### Framework Versions
357
  - Python: 3.12.13
eval/similarity_evaluation_pair-score-evaluator-dev_results.csv CHANGED
@@ -1,2 +1,3 @@
1
  epoch,steps,cosine_pearson,cosine_spearman
2
  1.0,10,-0.19455064165452754,-0.23640181564913884
 
 
1
  epoch,steps,cosine_pearson,cosine_spearman
2
  1.0,10,-0.19455064165452754,-0.23640181564913884
3
+ 1.0,10,0.2699926667525232,0.26478786196946036
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:cd543476f0f9a01f618a31f4d22cdeb7059819c2e9923664bd2c8775967be487
3
  size 2239607120
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f1d8f89cad89f8c94259a961ebea5bb41fa0d41ddade3b72bfe02c222ea1e968
3
  size 2239607120
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ed3793454249ac09153c91935ccdccdf6002b921720ebd53025cf7c59189c6b8
3
- size 5457
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0c9819dc4a6da81718bff90c604e03ddb46b261047ab4bc6fa308559d2e0a564
3
+ size 5521