File size: 40,503 Bytes
d20edb4
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
---
language:
- en
license: mit
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- dense
- generated_from_trainer
- dataset_size:80184
- loss:CachedMultipleNegativesRankingLoss
base_model: BAAI/bge-large-en-v1.5
widget:
- source_sentence: who does lennie choose in the sky is everywhere
  sentences:
  - Bank of England £1 note The Bank of England £1 note was a banknote of the pound
    sterling. After the ten shilling note was withdrawn in 1970 it became the smallest
    denomination note issued by the Bank of England. The one pound note was issued
    by the Bank of England for the first time in 1797 and continued to be printed
    until 1984. The note was withdrawn in 1988 in favour of the one pound coin.
  - The Sky Is Everywhere Lennie tries to make up with Joe by taking him some of Gram's
    roses, but doesn't succeed. Gram becomes furious with Lennie for cutting her roses
    and criticizes her for being selfish. Lennie realizes that she needs to change,
    apologizes to her grandmother, and tells her about the situation with Joe. Gram
    reassures Lennie that Joe is in love with her. Lennie writes Joe a letter expressing
    her feelings, and Joe ultimately forgives her and they reconcile. Toby and Lennie
    become good friends and visit Bailey's grave together to apologize to her. Lennie
    walks away from the grave with a smile, knowing that her sister would have forgiven
    her and that the only way to deal with grief is to accept that it is a part of
    you and to look ahead to the future.
  - 'Senate of the Philippines The Senate of the Philippines (Filipino: Senado ng
    Pilipinas, also Mataas na Kapulungan ng Pilipinas or "upper chamber") is the upper
    house of the bicameral legislature of the Philippines, the Congress; the House
    of Representatives is the lower house. The Senate is composed of 24 senators who
    are elected at-large with the country as one district under plurality-at-large
    voting.'
- source_sentence: who played charlie in charlie and the chocolate factory 2005
  sentences:
  - Charlie and the Chocolate Factory (film) Charlie and the Chocolate Factory is
    a 2005 musical fantasy comedy film directed by Tim Burton and written by John
    August, based on the 1964 British novel of the same name by Roald Dahl. The film
    stars Johnny Depp as Willy Wonka and Freddie Highmore as Charlie Bucket. The storyline
    follows Charlie, who wins a contest and is along with four other contest winners,
    subsequently led by Wonka on a tour of his chocolate factory, the most magnificent
    in the world.
  - The Punisher (TV series) The Punisher is scheduled to be released on November
    17, 2017.
  - The Vampire Diaries (season 2) The Vampire Diaries, an American supernatural drama,
    was officially renewed by The CW for a full 22-episode season on February 16,
    2010.[1] The first episode premiered on September 9, 2010, at 8 p.m. ET.[2] The
    season picks up immediately after the events of the season one finale. All the
    series regulars returned.[3] Season two focuses on the return of Elena Gilbert's
    (Nina Dobrev) doppelgänger, Katherine Pierce, the introduction of werewolves,
    the sun and moon curse, and the arrival of the original vampires. Tyler Lockwood's
    (Michael Trevino) uncle, Mason Lockwood (Taylor Kinney), arrives in town searching
    for the moonstone, a family heirloom. Tyler later learns of his family's werewolf
    curse. Meanwhile, Caroline Forbes (Candice Accola) is killed by Katherine while
    having Damon Salvatore's (Ian Somerhalder) blood in her system, turning her into
    a vampire. The arrival of the original vampires, Elijah (Daniel Gillies) and Klaus
    Mikaelson (Joseph Morgan), also bring about complications. Klaus is a vampire-werewolf
    hybrid, but his werewolf side had been forced into dormancy by witches, as nature
    would not stand for such an imbalance in power. Therefore, Klaus arrives in town
    with plans to break the curse and unleash his werewolf side by channelling the
    power of the full moon into the moonstone, sacrificing a vampire and a werewolf,
    and drinking the blood of the doppelgänger. The season is currently on air in
    Urdu on filmax channel in Pakistan. It became available on DVD and Blu-ray on
    August 30, 2011.[4]
- source_sentence: most of the really good agricultural land in mexico is owned by
  sentences:
  - 'State of the art The origin of the concept of "state of the art" took place in
    the beginning of the twentieth century.[3] The earliest use of the term "state
    of the art" documented by the Oxford English Dictionary dates back to 1910, from
    an engineering manual by Henry Harrison Suplee (1856-post 1943), an engineering
    graduate (University of Pennsylvania, 1876), titled Gas Turbine: progress in the
    design and construction of turbines operated by gases of combustion. The relevant
    passage reads: "In the present state of the art this is all that can be done".
    The term "art" refers to technics, rather than performing or fine arts.[4]'
  - London sewerage system Joseph Bazalgette, a civil engineer and Chief Engineer
    of the Metropolitan Board of Works, was given responsibility for the work. He
    designed an extensive underground sewerage system that diverted waste to the Thames
    Estuary, downstream of the main centre of population. Six main interceptor sewers,
    totalling almost 160 km (100 miles) in length, were constructed, some incorporating
    stretches of London's "lost" rivers. Three of these sewers were north of the river,
    the southernmost, low-level one being incorporated in the Thames Embankment. The
    Embankment also allowed new roads, new public gardens, and the Circle line of
    the London Underground. Victoria Embankment was finally officially opened on 13
    July 1870.[3][4]
  - Agriculture in Mexico During the early colonial period, the Spanish introduced
    more plants and the concept of domesticated animals, principally cattle, horses,
    donkeys, mules, goats and sheep, and barn yard animals such as chickens and pigs.
    Farming from the colonial period until the Mexican Revolution was focused on large
    private properties. After the Revolution these were broken up and the land redistributed.
    Since the latter 20th century NAFTA and economic policies have again favored large
    scale commercial agricultural holdings.
- source_sentence: who is the person who plays black panther
  sentences:
  - United States Capitol The United States Capitol, often called the Capitol Building,
    is the home of the United States Congress, and the seat of the legislative branch
    of the U.S. federal government. It is located on Capitol Hill at the eastern end
    of the National Mall in Washington, D.C. Though not at the geographic center of
    the Federal District, the Capitol forms the origin point for the District's street-numbering
    system and the District's four quadrants.
  - Supreme Court of the United States The Supreme Court of the United States is the
    highest federal court of the United States. Established pursuant to Article Three
    of the United States Constitution in 1789, it has ultimate (and largely discretionary)
    appellate jurisdiction over all federal courts and state court cases involving
    issues of federal law plus original jurisdiction over a small range of cases.
    In the legal system of the United States, the Supreme Court is generally the final
    interpreter of federal law including the United States Constitution, but it may
    act only within the context of a case in which it has jurisdiction. The Court
    may decide cases having political overtones but does not have power to decide
    nonjusticiable political questions, and its enforcement arm is in the executive
    rather than judicial branch of government.
  - Chadwick Boseman Chadwick Aaron Boseman[1] (born November 29, 1977)[2][3] is an
    American actor. He is known for portraying Jackie Robinson in 42 (2013), James
    Brown in Get on Up (2014), Black Panther in the Marvel Cinematic Universe (since
    2016), and Thurgood Marshall in Marshall (2017). He also had roles in the television
    series Lincoln Heights (2008) and Persons Unknown (2010), and the films The Express
    (2008), Draft Day (2014), and Message from the King (2016).
- source_sentence: can you find a pearl in a mussel
  sentences:
  - Freshwater pearl mussel Although the name "freshwater pearl mussel" is often used
    for this species, other freshwater mussel species can also create pearls and some
    can also be used as a source of mother of pearl. In fact, most cultured pearls
    today come from Hyriopsis species in Asia, or Amblema species in North America,
    both members of the related family Unionidae; pearls are also found within species
    in the genus Unio.
  - Ellis Island Generally, those immigrants who were approved spent from two to five
    hours at Ellis Island. Arrivals were asked 29 questions including name, occupation,
    and the amount of money carried. It was important to the American government that
    the new arrivals could support themselves and have money to get started. The average
    the government wanted the immigrants to have was between 18 and 25 dollars ($600
    in 2015 adjusted for inflation). Those with visible health problems or diseases
    were sent home or held in the island's hospital facilities for long periods of
    time. More than 3,000 would-be immigrants died on Ellis Island while being held
    in the hospital facilities. Some unskilled workers were rejected because they
    were considered "likely to become a public charge." About 2% were denied admission
    to the U.S. and sent back to their countries of origin for reasons such as having
    a chronic contagious disease, criminal background, or insanity.[43] Ellis Island
    was sometimes known as "The Island of Tears" or "Heartbreak Island"[44] because
    of those 2% who were not admitted after the long transatlantic voyage. The Kissing
    Post is a wooden column outside the Registry Room, where new arrivals were greeted
    by their relatives and friends, typically with tears, hugs, and kisses.[45][46]
  - Glee (season 1) The first season of the musical comedy-drama television series
    Glee originally aired on Fox in the United States. The pilot episode was broadcast
    as an advanced preview of the series on May 19, 2009, with the remainder of the
    season airing between September 9, 2009 and June 8, 2010. The season consisted
    of 22 episodes; the first 13 aired on Wednesdays at 9 pm (ET) and the final 9
    aired on Tuesdays at 9 pm (ET). The season was executive produced by Ryan Murphy,
    Brad Falchuk, and Dante Di Loreto; Murphy's production company helped co-produce
    the series alongside 20th Century Fox.
datasets:
- sentence-transformers/natural-questions
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
model-index:
- name: bge-large-en-v1.5
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: NanoQuoraRetrieval
      type: NanoQuoraRetrieval
    metrics:
    - type: cosine_accuracy@1
      value: 0.88
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.98
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.98
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 1.0
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.88
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.4133333333333333
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.25199999999999995
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.13999999999999999
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.7673333333333332
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.9520000000000001
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.9553333333333334
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 1.0
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.9435612217207588
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.9295238095238095
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.919404761904762
      name: Cosine Map@100
    - type: cosine_accuracy@1
      value: 0.88
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.98
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.98
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 1.0
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.88
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.4133333333333333
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.25199999999999995
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.13999999999999999
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.7673333333333332
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.9520000000000001
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.9553333333333334
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 1.0
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.9435612217207588
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.9295238095238095
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.919404761904762
      name: Cosine Map@100
---

# bge-large-en-v1.5

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5) on the [natural-questions](https://huggingface.co/datasets/sentence-transformers/natural-questions) dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [BAAI/bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5) <!-- at revision d4aa6901d3a41ba39fb536a557fa166f842b0e09 -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 1024 dimensions
- **Similarity Function:** Cosine Similarity
- **Training Dataset:**
    - [natural-questions](https://huggingface.co/datasets/sentence-transformers/natural-questions)
- **Language:** en
- **License:** mit

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("DannyAI/embedding_fine_tuning_with_peft_bge_large_en_v1.5")
# Run inference
queries = [
    "can you find a pearl in a mussel",
]
documents = [
    'Freshwater pearl mussel Although the name "freshwater pearl mussel" is often used for this species, other freshwater mussel species can also create pearls and some can also be used as a source of mother of pearl. In fact, most cultured pearls today come from Hyriopsis species in Asia, or Amblema species in North America, both members of the related family Unionidae; pearls are also found within species in the genus Unio.',
    'Ellis Island Generally, those immigrants who were approved spent from two to five hours at Ellis Island. Arrivals were asked 29 questions including name, occupation, and the amount of money carried. It was important to the American government that the new arrivals could support themselves and have money to get started. The average the government wanted the immigrants to have was between 18 and 25 dollars ($600 in 2015 adjusted for inflation). Those with visible health problems or diseases were sent home or held in the island\'s hospital facilities for long periods of time. More than 3,000 would-be immigrants died on Ellis Island while being held in the hospital facilities. Some unskilled workers were rejected because they were considered "likely to become a public charge." About 2% were denied admission to the U.S. and sent back to their countries of origin for reasons such as having a chronic contagious disease, criminal background, or insanity.[43] Ellis Island was sometimes known as "The Island of Tears" or "Heartbreak Island"[44] because of those 2% who were not admitted after the long transatlantic voyage. The Kissing Post is a wooden column outside the Registry Room, where new arrivals were greeted by their relatives and friends, typically with tears, hugs, and kisses.[45][46]',
    "Glee (season 1) The first season of the musical comedy-drama television series Glee originally aired on Fox in the United States. The pilot episode was broadcast as an advanced preview of the series on May 19, 2009, with the remainder of the season airing between September 9, 2009 and June 8, 2010. The season consisted of 22 episodes; the first 13 aired on Wednesdays at 9\xa0pm (ET) and the final 9 aired on Tuesdays at 9\xa0pm (ET). The season was executive produced by Ryan Murphy, Brad Falchuk, and Dante Di Loreto; Murphy's production company helped co-produce the series alongside 20th Century Fox.",
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.7103, 0.3918, 0.2758]])
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval

* Dataset: `NanoQuoraRetrieval`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) with these parameters:
  ```json
  {
      "query_prompt": "query: ",
      "corpus_prompt": "document: "
  }
  ```

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.88       |
| cosine_accuracy@3   | 0.98       |
| cosine_accuracy@5   | 0.98       |
| cosine_accuracy@10  | 1.0        |
| cosine_precision@1  | 0.88       |
| cosine_precision@3  | 0.4133     |
| cosine_precision@5  | 0.252      |
| cosine_precision@10 | 0.14       |
| cosine_recall@1     | 0.7673     |
| cosine_recall@3     | 0.952      |
| cosine_recall@5     | 0.9553     |
| cosine_recall@10    | 1.0        |
| **cosine_ndcg@10**  | **0.9436** |
| cosine_mrr@10       | 0.9295     |
| cosine_map@100      | 0.9194     |

#### Information Retrieval

* Dataset: `NanoQuoraRetrieval`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) with these parameters:
  ```json
  {
      "query_prompt": "query: ",
      "corpus_prompt": "document: "
  }
  ```

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.88       |
| cosine_accuracy@3   | 0.98       |
| cosine_accuracy@5   | 0.98       |
| cosine_accuracy@10  | 1.0        |
| cosine_precision@1  | 0.88       |
| cosine_precision@3  | 0.4133     |
| cosine_precision@5  | 0.252      |
| cosine_precision@10 | 0.14       |
| cosine_recall@1     | 0.7673     |
| cosine_recall@3     | 0.952      |
| cosine_recall@5     | 0.9553     |
| cosine_recall@10    | 1.0        |
| **cosine_ndcg@10**  | **0.9436** |
| cosine_mrr@10       | 0.9295     |
| cosine_map@100      | 0.9194     |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### natural-questions

* Dataset: [natural-questions](https://huggingface.co/datasets/sentence-transformers/natural-questions) at [f9e894e](https://huggingface.co/datasets/sentence-transformers/natural-questions/tree/f9e894e1081e206e577b4eaa9ee6de2b06ae6f17)
* Size: 80,184 training samples
* Columns: <code>query</code> and <code>answer</code>
* Approximate statistics based on the first 1000 samples:
  |         | query                                                                              | answer                                                                               |
  |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
  | type    | string                                                                             | string                                                                               |
  | details | <ul><li>min: 10 tokens</li><li>mean: 11.72 tokens</li><li>max: 24 tokens</li></ul> | <ul><li>min: 11 tokens</li><li>mean: 132.91 tokens</li><li>max: 512 tokens</li></ul> |
* Samples:
  | query                                                         | answer                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
  |:--------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>who wrote i came in like a wrecking ball</code>         | <code>Wrecking Ball (Miley Cyrus song) "Wrecking Ball" is a song recorded by American singer Miley Cyrus for her fourth studio album Bangerz (2013). It was released on August 25, 2013, by RCA Records as the album's second single. The song was written by MoZella, Stephan Moccio, Sacha Skarbek, Kiyanu Kim,[2] Lukasz Gottwald, and Henry Russell Walter;[3] production was helmed by the last two. "Wrecking Ball" is a pop ballad which lyrically discusses the deterioration of a relationship.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
  | <code>what was the purpose of the three-field system</code>   | <code>Three-field system The three-field system is a regime of crop rotation that was used in medieval and early-modern Europe. Crop rotation is the practice of growing a series of different types of crops in the same area in sequential seasons. Under this system, the arable land of an estate or village was divided into three large fields: one was planted in the autumn with winter wheat or rye; the second field was planted with other crops such as peas, lentils, or beans; and the third was left fallow, in order to allow the soil of that field to regain its nutrients. With each rotation, the field would be used differently, so that a field would be planted for two out of the three years used, whilst one year it "rested". Previously a "two field system" had been in place, with half the land being left fallow. The three field system allowed farmers to plant more crops and therefore to increase production and legumes have the ability to fix nitrogen and so fertilize the soil. With more crops ava...</code> |
  | <code>who is the main person in the legislative branch</code> | <code>Article One of the United States Constitution Section 1 is a vesting clause that bestows federal legislative power exclusively to Congress. Similar clauses are found in Articles II and III. The former confers executive power upon the President alone, and the latter grants judicial power solely to the federal judiciary. These three articles create a separation of powers among the three branches of the federal government. This separation of powers, by which each department may exercise only its own constitutional powers and no others,[1][2] is fundamental to the idea of a limited government accountable to the people.</code>                                                                                                                                                                                                                                                                                                                                                                                              |
* Loss: [<code>CachedMultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedmultiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "mini_batch_size": 16,
      "gather_across_devices": false
  }
  ```

### Evaluation Dataset

#### natural-questions

* Dataset: [natural-questions](https://huggingface.co/datasets/sentence-transformers/natural-questions) at [f9e894e](https://huggingface.co/datasets/sentence-transformers/natural-questions/tree/f9e894e1081e206e577b4eaa9ee6de2b06ae6f17)
* Size: 20,047 evaluation samples
* Columns: <code>query</code> and <code>answer</code>
* Approximate statistics based on the first 1000 samples:
  |         | query                                                                              | answer                                                                              |
  |:--------|:-----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
  | type    | string                                                                             | string                                                                              |
  | details | <ul><li>min: 10 tokens</li><li>mean: 11.79 tokens</li><li>max: 25 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 135.48 tokens</li><li>max: 512 tokens</li></ul> |
* Samples:
  | query                                                   | answer                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
  |:--------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>when did call of duty ww2 come out</code>         | <code>Call of Duty: WWII Call of Duty: WWII is a first-person shooter video game developed by Sledgehammer Games and published by Activision. It is the fourteenth main installment in the Call of Duty series and was released worldwide on November 3, 2017 for Microsoft Windows, PlayStation 4 and Xbox One. It is the first title in the series to be set primarily during World War II since Call of Duty: World at War in 2008.[2] The game is set in the European theatre, and is centered around a squad in the 1st Infantry Division, following their battles on the Western Front, and set mainly in the historical events of Operation Overlord; the multiplayer expands to different fronts not seen in the campaign.</code>                                               |
  | <code>who is doing the half time super bowl</code>      | <code>Super Bowl LII halftime show The Super Bowl LII Halftime Show (officially known as the Pepsi Super Bowl LII Halftime Show) took place on February 4, 2018 at U.S. Bank Stadium in Minneapolis, Minnesota, as part of Super Bowl LII. Justin Timberlake was the featured performer, as confirmed by the National Football League (NFL) on October 22, 2017.[1] It was televised nationally by NBC.</code>                                                                                                                                                                                                                                                                                                                                                                          |
  | <code>when was the sewage system built in london</code> | <code>London sewerage system Joseph Bazalgette, a civil engineer and Chief Engineer of the Metropolitan Board of Works, was given responsibility for the work. He designed an extensive underground sewerage system that diverted waste to the Thames Estuary, downstream of the main centre of population. Six main interceptor sewers, totalling almost 160 km (100 miles) in length, were constructed, some incorporating stretches of London's "lost" rivers. Three of these sewers were north of the river, the southernmost, low-level one being incorporated in the Thames Embankment. The Embankment also allowed new roads, new public gardens, and the Circle line of the London Underground. Victoria Embankment was finally officially opened on 13 July 1870.[3][4]</code> |
* Loss: [<code>CachedMultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedmultiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "mini_batch_size": 16,
      "gather_across_devices": false
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: steps
- `per_device_train_batch_size`: 5
- `per_device_eval_batch_size`: 5
- `learning_rate`: 2e-05
- `max_steps`: 100
- `warmup_ratio`: 0.1
- `seed`: 30
- `bf16`: True
- `load_best_model_at_end`: True
- `prompts`: {'query': 'query: ', 'answer': 'document: '}
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 5
- `per_device_eval_batch_size`: 5
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 3.0
- `max_steps`: 100
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 30
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `parallelism_config`: None
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `hub_revision`: None
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `liger_kernel_config`: None
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: {'query': 'query: ', 'answer': 'document: '}
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional
- `router_mapping`: {}
- `learning_rate_mapping`: {}

</details>

### Training Logs
| Epoch      | Step    | Training Loss | Validation Loss | NanoQuoraRetrieval_cosine_ndcg@10 |
|:----------:|:-------:|:-------------:|:---------------:|:---------------------------------:|
| -1         | -1      | -             | -               | 0.9583                            |
| **0.0062** | **100** | **0.0156**    | **0.0067**      | **0.9436**                        |
| -1         | -1      | -             | -               | 0.9436                            |

* The bold row denotes the saved checkpoint.

### Framework Versions
- Python: 3.12.11
- Sentence Transformers: 5.1.0
- Transformers: 4.56.1
- PyTorch: 2.8.0+cu126
- Accelerate: 1.10.1
- Datasets: 4.0.0
- Tokenizers: 0.22.0

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### CachedMultipleNegativesRankingLoss
```bibtex
@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->