File size: 41,588 Bytes
8e44d8f 3eeba7f 8e44d8f 3b7c221 8e44d8f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 | ---
tags:
- sentence-transformers
- multi-vector
- colbert
- late-interaction
- generated_from_trainer
- dataset_size:1000000
- loss:CachedMultiVectorMultipleNegativesRankingLoss
base_model: lightonai/mLateOn-unsupervised
widget:
- text: Cytomegalovirus³ (CMV)³ is³ a³ DNA³ virus³ that³ causes³ various³ dermatologic³
and³ systemic³ disorders,³especially³in³immunocompromised³ subjects. 1 ³Cutaneous³manifestation³of³CMV³is³
a³rare³condition³that³its³diagnosis³is³oten³made³ with³delay. 3 ³Surprisingly,³patients³with³in³autosomal³recessive³HIES³sufer³
from³eczema³and³recurrent³cutaneous³infections 3 ³and³se-vere³viral³infections³have³been³frequently³seen³in³DOCK8³
deiciency. 4 It³ should³ be³ emphasized³ that³ early³ diagnosis³ of³ cu-taneous³CMV³in³patients³with³underlying³immunodefi-ciencies³and³appropriate³treatment³are³essential³to³avoid³
dissemination³ of³ the³ disease,³ further³ complications³ and³ even³ death³ in³
such³ patients.³ However,³ further³ molecu-lar³studies³are³needed³to³make³definite³diagnosis³in³such³
cases.
- text: "Согласно результатам крупного когортного исследо-вания здоровых лиц без сахарного\
\ диабета, пришли к выводу, что повышение уровня ИАП-1 в начале иссле-дования\
\ было предиктором развития сахарного диабета в течение 5 лет наблюдений [51]\
\ . Следовательно, повы-шение уровня ИАП-1 при сахарном диабете может повышать\
\ риск развития атеросклеротических пора-жений и окклюзионных тромботических осложнений.\
\ Например, около 30% пациентов с инсультом страдают от сахарного диабета [7,\
\ 52] . Новые данные свиде-тельствуют о том, что снижение фибринолитической активности\
\ или повышение уровня ИАП-1 играют важную роль в развитии сосудистых заболеваний\
\ у пациентов с сахарным диабетом, поскольку усиленная потеря массы тела и снижение\
\ концентрации ИАП-1 в плазме крови позволяют снизить риск развития тромбоза у\
\ пациентов с инсулинорезистентностью и большим числом факторов риска развития\
\ сосудистых заболеваний [50] . Снижение уровня глюкозы или заболеваемости сахарным\
\ диабетом путем повышения чувствительности к инсулину также приводит к умень-шению\
\ уровня циркулирующего ИАП-1 и снижает скорость прогрессирования сахарного диабета.\
\ Это позволяет предположить, что ингибирование ИАП-1 может снизить прогрессирование\
\ сахарного диабета и связанных с ним сосудистых осложнений в группе пациентов\
\ высокого риска [53] .\n\n Артериальная гипертензия является основным фактором\
\ риска развития тромботических сосудис-тых заболеваний, включая ишемический инсульт\
\ [54] . Ускорение развития атеросклероза и повышение час-тоты разрыва бляшек\
\ могут повысить риск развития ишемических событий при артериальной гипертензии,\
\ однако становится все более очевидным, что артери-альная гипертензия (АГ) может\
\ смещать баланс гемос-таза в сторону протромботического состояния или состояния\
\ гиперкоагуляции [5] . У пациентов с АГ обна-ружили повышение активности ИАП-1\
\ и снижение активности ТАП в плазме и эндотелии, но основные механизмы этих явлений\
\ до сих пор не выяснены."
- text: How does leptin impact the expression and function of LDLR and PCSK9, and
what potential implications does this have for obesity and cardiovascular disease?
- text: "In such a situation if the distal perforators from the descending branch\
\ of the lateral circumflex femoral artery are intact, a flap can be designed\
\ on it and rotated 180 degree like a propeller flap to cover the defect over\
\ the knee. Extending the dissection into the muscle will help to avoid the kinking\
\ of the pedicle and increasing the reach. \n\n One such flap is lateral supragenicular\
\ artery perforator flap which has a constant anatomy and is located around 5.3\
\ cm from the superolateral corner of patella. This is marked by Doppler preoperatively\
\ and then isolated by surgical dissection. The flap can go up to the mid-thigh\
\ level. 42 \n\n This is based on cutaneous perforator from the large muscular\
\ branch of vastus medialis muscle from superficial femoral artery which traverses\
\ through the muscle and called medial vastus medialis artery. The perforator\
\ site is relatively constant and located ~9 cm proximal to the adductor tubercle.\
\ A line is drawn from mid-inguinal point to medial femoral condyle. From the\
\ midpoint of this line, another line is drawn to midpoint of superior border\
\ of patella. The perforator is located along this line and marked with Doppler\
\ and used for designing the flap. 43, 44 \n\n As the defects are over or around\
\ the mobile joint, they are immobilized postoperatively for a period of 2 to\
\ 3 weeks to allow the flap or the skin graft to heal well. As there is injury\
\ to the periarticular structures like the ligaments and the capsule, the chances\
\ of stiffness are high. Mobilization is started as soon as healing is achieved.\
\ In case repair or reconstruction of deeper structures like ligamentum patellae\
\ or quadriceps tendon has been done physiotherapy may be done as per the needs\
\ of the repaired structures. The extent of movement gained is dependent on the\
\ injury to deeper structures and scarring.\n\n Defects around the knee require\
\ careful assessment of the size of the defect and the condition of the surrounding\
\ skin and muscles to decide on the type of flap which is chosen to cover them.\
\ The flaps can be raised from the leg or thigh based on the size needed or the\
\ pattern of injury. In case both are injured, we need to use distant flaps in\
\ form of free flaps depending on the condition of the vessels. Prompt and early\
\ coverage of the knee with good pliable flap will go a long way in retaining\
\ good mobility of the joint."
- text: "Several studies suggest that symptomatic and jaundiced patients have a higher\
\ rate of SVC and SVR (8, 9, 18, 23, 25, (34) (35) (36) (37) , possibly associated\
\ with a better response of the immune system (29, 36) .\n\n The monitoring of\
\ viral kinetics can predict the SVC. RNA-HCV undetectable at week 4 in our series\
\ was related to a cure rate of 98%; in these patients treatment with PEG-IFN\
\ is unnecessary (35) . In contrast, only 69% of patients with detectable HCV-RNA\
\ at week 4 were cured. Those treated for persistent viremia at week 4 did not\
\ benefit over those untreated, implying that if there is not spontaneous viral\
\ clearance at week 4 a bad outcome is not improbable, and reinforces the value\
\ of the early determination of HCV-RNA (positive predictive value 95%).\n\n In\
\ patients with undetectable HCV-RNA at week 12 our results showed that 90% were\
\ cured versus only 9% with persistent viremia. Yet these patients did not benefit\
\ from treatment because the cure rate was identical for both those treated and\
\ untreated (91 and 92%). Treatment, therefore, would be unnecessary for patients\
\ with undetectable HCV-RNA at week 12.\n\n A new factor that can help to decide\
\ whether to treat or not to treat is the polymorphism in the IL28B gene region\
\ 8099917. Grebely et al. have found that TT homozygosity, but not GG/GT, was\
\ the only factor associated to SVC in patients with recent HCV. Furthermore,\
\ TT patients were more frequently icteric. These results could not be confirmed\
\ in treated patients. Based in these data, they suggested that this polymorphism\
\ must be investigated in patients with AHC and early treatment could be started\
\ in individuals with non-TT genotypes (38) .\n\n In summary, in Spain the most\
\ likely cause of AHC is nosocomial transmission, followed at a distance by IDUs\
\ and sexual transmission. Nearly 60% of AHC patients received antiviral therapy\
\ with a significantly higher SVR when compared to the untreated group. Given\
\ the high rate of spontaneous recovery, from our results, especially for jaundiced\
\ patients, treatment should not be initiated too early and it can be delayed\
\ until 12 weeks from symptomonset. At week 4, undetectable HCV-RNA was associated\
\ with a satisfactory outcome; thus, treatment could be avoided in these patients.\
\ HCV-RNA negativity at week 12 is also associated with a very high rate of viral\
\ clearance; on the contrary, the persistence of detectable viremia suggests a\
\ bad outcome."
datasets:
- tomaarsen/miriad-4.4M-split
pipeline_tag: feature-extraction
library_name: sentence-transformers
metrics:
- maxsim_accuracy@1
- maxsim_accuracy@10
- maxsim_precision@10
- maxsim_recall@10
- maxsim_ndcg@10
- maxsim_mrr@10
- maxsim_map@10
model-index:
- name: Multi-Vector Encoder
results:
- task:
type: multi-vector-information-retrieval
name: Multi Vector Information Retrieval
dataset:
name: monitor
type: monitor
metrics:
- type: maxsim_accuracy@1
value: 0.966
name: Maxsim Accuracy@1
- type: maxsim_accuracy@10
value: 0.996
name: Maxsim Accuracy@10
- type: maxsim_precision@10
value: 0.0996
name: Maxsim Precision@10
- type: maxsim_recall@10
value: 0.996
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.9818833515858946
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.9772857142857143
name: Maxsim Mrr@10
- type: maxsim_map@10
value: 0.9772857142857143
name: Maxsim Map@10
- task:
type: multi-vector-information-retrieval
name: Multi Vector Information Retrieval
dataset:
name: final
type: final
metrics:
- type: maxsim_accuracy@1
value: 0.929
name: Maxsim Accuracy@1
- type: maxsim_accuracy@10
value: 0.989
name: Maxsim Accuracy@10
- type: maxsim_precision@10
value: 0.09890000000000002
name: Maxsim Precision@10
- type: maxsim_recall@10
value: 0.989
name: Maxsim Recall@10
- type: maxsim_ndcg@10
value: 0.9607570782648661
name: Maxsim Ndcg@10
- type: maxsim_mrr@10
value: 0.9514984126984127
name: Maxsim Mrr@10
- type: maxsim_map@10
value: 0.9514984126984127
name: Maxsim Map@10
---
# Multi-Vector Encoder
This is a [Multi-Vector Encoder](https://www.sbert.net/docs/multi_vector_encoder/usage/usage.html) model finetuned from [lightonai/mLateOn-unsupervised](https://huggingface.co/lightonai/mLateOn-unsupervised) on the [miriad-4.4_m-split](https://huggingface.co/datasets/tomaarsen/miriad-4.4M-split) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps inputs to sequences of 128-dimensional token-level vectors and scores them with late interaction (MaxSim), useful for semantic search with late interaction.
**mLateOn-medical is a domain specialist.** It was trained on 1 million medical question and passage pairs from [MIRIAD](https://huggingface.co/datasets/tomaarsen/miriad-4.4M-split) in 14 hours on a single RTX 3090. On a held-out benchmark of 1,000 medical questions searching 200,000 passages, it beats every general-purpose retrieval model I evaluated, dense, sparse, lexical, and multi-vector alike, including dense models with roughly 30 times as many active parameters.
The full recipe, the evaluation protocol, and the experiments behind each choice are described in [Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers](https://huggingface.co/blog/train-multi-vector-encoder).
## Model Details
### Model Description
- **Model Type:** Multi-Vector Encoder
- **Base model:** [lightonai/mLateOn-unsupervised](https://huggingface.co/lightonai/mLateOn-unsupervised) <!-- at revision bda45579e5aff37abc3f5cf3b64e3e56ac4176d1 -->
- **Maximum Sequence Length:** 8192 tokens
- **Output Dimensionality:** 128 dimensions
- **Similarity Function:** MaxSim
- **Supported Modality:** Text
- **Training Dataset:**
- [miriad-4.4_m-split](https://huggingface.co/datasets/tomaarsen/miriad-4.4M-split)
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Documentation:** [Multi-Vector Encoder Documentation](https://www.sbert.net/docs/multi_vector_encoder/usage/usage.html)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers)
- **Hugging Face:** [Multi-Vector Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=multi-vector)
### Full Model Architecture
```
MultiVectorEncoder(
(0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'ModernBertModel'})
(1): Dense({'in_features': 768, 'out_features': 1536, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'module_input_name': 'token_embeddings', 'module_output_name': 'token_embeddings', 'use_residual': True})
(2): Dense({'in_features': 1536, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'module_input_name': 'token_embeddings', 'module_output_name': 'token_embeddings', 'use_residual': True})
(3): Dense({'in_features': 768, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'module_input_name': 'token_embeddings', 'module_output_name': 'token_embeddings'})
(4): MultiVectorMask({'skiplist_words': ['!', '"', '#', '$', '%', '&', "'", '(', ')', '*', '+', ',', '-', '.', '/', ':', ';', '<', '=', '>', '?', '@', '[', '\\', ']', '^', '_', '`', '{', '|', '}', '~'], 'skiplist_tasks': ['document'], 'keep_only_token_ids': None})
(5): Normalize({'module_input_name': 'token_embeddings', 'module_output_name': 'token_embeddings'})
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import MultiVectorEncoder
# Download from the 🤗 Hub
model = MultiVectorEncoder("multi-vector-encoder/mLateOn-medical")
# Run inference: each input becomes a sequence of per-token vectors (variable length).
queries = [
'How does the polymorphism in the IL28B gene region 8099917 affect the response to treatment in patients with AHC?\n',
]
documents = [
'Several studies suggest that symptomatic and jaundiced patients have a higher rate of SVC and SVR (8, 9, 18, 23, 25, (34) (35) (36) (37) , possibly associated with a better response of the immune system (29, 36) .\n\n The monitoring of viral kinetics can predict the SVC. RNA-HCV undetectable at week 4 in our series was related to a cure rate of 98%; in these patients treatment with PEG-IFN is unnecessary (35) . In contrast, only 69% of patients with detectable HCV-RNA at week 4 were cured. Those treated for persistent viremia at week 4 did not benefit over those untreated, implying that if there is not spontaneous viral clearance at week 4 a bad outcome is not improbable, and reinforces the value of the early determination of HCV-RNA (positive predictive value 95%).\n\n In patients with undetectable HCV-RNA at week 12 our results showed that 90% were cured versus only 9% with persistent viremia. Yet these patients did not benefit from treatment because the cure rate was identical for both those treated and untreated (91 and 92%). Treatment, therefore, would be unnecessary for patients with undetectable HCV-RNA at week 12.\n\n A new factor that can help to decide whether to treat or not to treat is the polymorphism in the IL28B gene region 8099917. Grebely et al. have found that TT homozygosity, but not GG/GT, was the only factor associated to SVC in patients with recent HCV. Furthermore, TT patients were more frequently icteric. These results could not be confirmed in treated patients. Based in these data, they suggested that this polymorphism must be investigated in patients with AHC and early treatment could be started in individuals with non-TT genotypes (38) .\n\n In summary, in Spain the most likely cause of AHC is nosocomial transmission, followed at a distance by IDUs and sexual transmission. Nearly 60% of AHC patients received antiviral therapy with a significantly higher SVR when compared to the untreated group. Given the high rate of spontaneous recovery, from our results, especially for jaundiced patients, treatment should not be initiated too early and it can be delayed until 12 weeks from symptomonset. At week 4, undetectable HCV-RNA was associated with a satisfactory outcome; thus, treatment could be avoided in these patients. HCV-RNA negativity at week 12 is also associated with a very high rate of viral clearance; on the contrary, the persistence of detectable viremia suggests a bad outcome.',
"The ratio is therefore reliable for use in normalizing the excreted urinary cotinine concentration in urine spot samples, when 24-hour samples are not available or not practical. There has been much debate in the scientific community over whether the ratio is useful (38) , and this affirmative finding helps to settle that debate. In addition, the results indicate that cotinine is best used to distinguish recent smokers from nonsmokers. It is not possible to distinguish nonrecent smokers-those that reported being current smokers that did not smoke on the day of urine collection-from nonsmokers. Given the leveling off of the cotinine at higher smoking intensities, it is also difficult to use urinary cotinine to assess a person's smoking intensity at higher levels (greater CPD) of smoking.\n\n In the radiochemical analysis of the urine samples, the median 210 Pb activity concentration in the 24-hour urine samples of smokers (12 mBq) was statistically greater than that in nonsmokers (8 mBq). The median urinary 210 Po activity concentration in smokers (19 mBq) was also close to being statistically greater than that in nonsmokers (15 mBq Po is absorbed (39) , whereas only 15-20% of ingested 210 Pb is absorbed (40) ] to a larger extent than it is influenced by inhalation intakes or internal body stores. Most recently, a group in central Italy published their study of the 210 Po activity concentration in the 24-hour urine samples of 132 residents and also failed to find a statistically significant difference in the urine of smokers versus nonsmokers (41) .\n\n Significant correlations were observed, however, between the total urinary 210 Pb activity and the smoking indices (cotinine, CPD, years). Regression models indicated that CPD is statistically predicted by total urinary 210 Pb activity, although with less variability being accounted for than in the model using total urinary cotinine as a predictor. 210 Pb, however, with its longer apparent half-life, indicates past smoking (months), whereas cotinine, with its short half-life, indicates recent smoking (days). A logistic regression model indicated that a 1-unit increase in total urinary 210 Pb activity is associated with an estimated 25% increase in the odds of being a smoker, indicating that 210 Pb activity can be used to predict the probability of having smoked within the past months. Following validation, practical applications might include both epidemiologic studies and smoking cessation studies. For the latter, repeated intrasubject measurements could be made over time to monitor a person's progress in quitting smoking. In closing, the results of this study indicate a promising new use of 210 Pb as a biomarker in predicting a person's retrospective smoking status.\n\n No potential conflicts of interest were disclosed.",
"Patients with acyanotic CHD (with or without increased pulmonary blood flow) were reported to be more susceptible to developing pulmonary TB. No case of PTB was reported in patients with TOF, which is a cyanotic CHD with decreased pulmonary blood flow. Van der Merwe et al. explained that patients with cyanotic CHD have decreased pulmonary blood flow and cyanosis, which can inhibit the growth of M. tuberculosis whereas in acyanotic CHD, normal or increased pulmonary blood flow and normal pulmonary arterial saturation provides an encouraging environment for bacterial growth [4] . Ifere et al. have also reported two cases of PTB in presence of CHD other than TOF [5] . Olguntürk et al. reported PTB in a pediatric patient with tricuspid valve pathology, ASD, and pulmonary hypertension. The clinical suspicion was only confirmed on autopsy as all other investigations were negative [6] .\n\n To our knowledge, there are only two reported cases of pulmonary TB in the presence of TOF [7] [8] . Radovic et al. [7] reported a middle-aged male patient with sputum positive TB who was investigated for CHD because of deteriorating cyanosis and respiratory insufficiency, even with anti-TB drugs. Echo was consistent with TOF. Gunay et al. [8] presented a young female, a diagnosed case of TOF, with respiratory complains, who was diagnosed with PTB and managed with first-line anti-TB drugs for two months. However, the literature doesn't report any case of TOF with secondary PTB and secondary non-invasive pulmonary aspergilloma as in our reported case.\n\n There are various diagnostic tools for pulmonary aspergilloma, including serum GM level, BAL GM, culture of Aspergillus species in sputum or BAL fluid, and serum Aspergillus precipitating antibodies. BAL GM assays were more sensitive than serum GM assays [9] .\n\n Undiagnosed congenital heart defects rarely present in adults. Physicians dealing with PTB should keep a suspicion of underlying TOF in mind. Similarly, physicians dealing with known cases of TOF presenting with respiratory distress should also keep the differential diagnosis of pulmonary tuberculosis and aspergilloma in mind.\n\n Human subjects: Consent was obtained by all participants in this study.\n\n In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to",
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings[0].shape, document_embeddings[0].shape)
# (35, 128) (516, 128)
# Get the MaxSim similarity scores
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[29.7060, 10.7889, 9.2684]])
```
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
## Evaluation
### Metrics
#### Multi Vector Information Retrieval
* Datasets: `monitor` and `final`
* Evaluated with [<code>MultiVectorInformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/multi_vector_encoder/evaluation.html#sentence_transformers.multi_vector_encoder.evaluation.MultiVectorInformationRetrievalEvaluator)
| Metric | monitor | final |
|:--------------------|:-----------|:-----------|
| maxsim_accuracy@1 | 0.966 | 0.929 |
| maxsim_accuracy@10 | 0.996 | 0.989 |
| maxsim_precision@10 | 0.0996 | 0.0989 |
| maxsim_recall@10 | 0.996 | 0.989 |
| **maxsim_ndcg@10** | **0.9819** | **0.9608** |
| maxsim_mrr@10 | 0.9773 | 0.9515 |
| maxsim_map@10 | 0.9773 | 0.9515 |
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
## Training Details
### Training Dataset
#### miriad-4.4_m-split
* Dataset: [miriad-4.4_m-split](https://huggingface.co/datasets/tomaarsen/miriad-4.4M-split) at [596b9ab](https://huggingface.co/datasets/tomaarsen/miriad-4.4M-split/tree/596b9ab305d52cb73644ed5b5004957c7bfaae40)
* Size: 1,000,000 training samples
* Columns: <code>question</code> and <code>passage_text</code>
* Approximate statistics based on the first 100 samples:
| | question | passage_text |
|:---------|:------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------|
| type | string | string |
| modality | text | text |
| details | <ul><li>min: 48 characters</li><li>mean: 106.6 characters</li><li>max: 317 characters</li></ul> | <ul><li>min: 2837 characters</li><li>mean: 4576.82 characters</li><li>max: 7387 characters</li></ul> |
* Samples:
| question | passage_text |
|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| <code>What factors may contribute to increased pulmonary conduit durability in patients who undergo the Ross operation compared to those with right ventricular outflow tract obstruction?<br></code> | <code>I n 1966, Ross and Somerville 1 reported the first use of an aortic homograft to establish right ventricle-to-pulmonary artery continuity in a patient with tetralogy of Fallot and pulmonary atresia. Since that time, pulmonary position homografts have been used in a variety of right-sided congenital heart lesions. Actuarial 5-year homograft survivals for cryopreserved homografts are reported to range between 55% and 94%, with the shortest durability noted in patients less than 2 years of age. 4 Pulmonary position homografts also are used to replace pulmonary autografts explanted to repair left-sided outflow disease (the Ross operation). Several factors may be likely to favor increased pulmonary conduit durability in Ross patients compared with those with right ventricular outflow tract obstruction, including later age at operation (allowing for larger homografts), more normal pulmonary artery architecture, absence of severe right ventricular hypertrophy, and more natural positioning of ...</code> |
| <code>How does MCAM expression in hMSC affect the growth and maintenance of hematopoietic progenitors?</code> | <code>After culture in a 3-dimensional hydrogel-based matrix, which constitutes hypoxic conditions, MCAM expression is lost. Concordantly, Tormin et al. demonstrated that MCAM is down-regulated under hypoxic conditions. 10 Furthermore, it was shown by others and our group that oxygen tension causes selective modification of hematopoietic cell and mesenchymal stromal cell interactions in co-culture systems as well as influence HSPC metabolism. [44] [45] [46] Thus, the observed differences between Sharma et al. and our data in HSPC supporting capacity of hMSC are likely due to the different culture conditions used. Further studies are required to clarify the influence of hypoxia in our model system. Altogether these findings provide further evidence for the importance of MCAM in supporting HSPC. Furthermore, previous reports have shown that MCAM is down-regulated in MSC after several passages as well as during aging and differentiation. 19, 47 Interestingly, MCAM overexpression in hMSC enhance...</code> |
| <code>What is the relationship between Fanconi anemia and breast and ovarian cancer susceptibility genes?<br></code> | <code>( 31 ) , of which 5% -10 % may be caused by genetic factors ( 32 ) , up to half a million of these patients may be at risk of secondary hereditary neoplasms. The historic observation of twofold to fi vefold increased risks of cancers of the ovary, thyroid, and connective tissue after breast cancer ( 33 ) presaged the later syndromic association of these tumors with inherited mutations of BRCA1, BRCA2, PTEN, and p53 ( 16 ) . By far the largest cumulative risk of a secondary cancer in BRCA mutation carriers is associated with cancer in the contralateral breast, which may reach a risk of 29.5% at 10 years ( 34 ) . The Breast Cancer Linkage Consortium ( 35 , 36 ) also documented threefold to fi vefold increased risks of subsequent cancers of prostate, pancreas, gallbladder, stomach, skin (melanoma), and uterus in BRCA2 mutation carriers and twofold increased risks of prostate and pancreas cancer in BRCA1 mutation carriers; these results are based largely on self-reported family history inf...</code> |
* Loss: [<code>CachedMultiVectorMultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/multi_vector_encoder/losses.html#cachedmultivectormultiplenegativesrankingloss) with these parameters:
```json
{
"scale": 1.0,
"similarity_fct": "colbert_scores",
"mini_batch_size": 16,
"mini_batch_num_tokens": null,
"score_mini_batch_size": 16,
"gather_across_devices": false
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `per_device_train_batch_size`: 128
- `num_train_epochs`: 1
- `learning_rate`: 0.0001
- `warmup_steps`: 0.05
- `bf16`: True
- `per_device_eval_batch_size`: 16
- `prompts`: {'question': '[Q] ', 'passage_text': '[D] '}
- `batch_sampler`: no_duplicates
- `max_length`: 8192
#### All Hyperparameters
<details><summary>Click to expand</summary>
- `per_device_train_batch_size`: 128
- `num_train_epochs`: 1
- `max_steps`: -1
- `learning_rate`: 0.0001
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: None
- `warmup_steps`: 0.05
- `optim`: adamw_torch_fused
- `optim_args`: None
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `optim_target_modules`: None
- `gradient_accumulation_steps`: 1
- `average_tokens_across_devices`: True
- `max_grad_norm`: 1.0
- `label_smoothing_factor`: 0.0
- `bf16`: True
- `fp16`: False
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `use_liger_kernel`: False
- `liger_kernel_config`: None
- `use_cache`: False
- `neftune_noise_alpha`: None
- `torch_empty_cache_steps`: None
- `auto_find_batch_size`: False
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `include_num_input_tokens_seen`: no
- `log_level`: passive
- `log_level_replica`: warning
- `disable_tqdm`: False
- `project`: huggingface
- `trackio_space_id`: None
- `trackio_bucket_id`: None
- `trackio_static_space_id`: None
- `per_device_eval_batch_size`: 16
- `prediction_loss_only`: True
- `eval_on_start`: False
- `eval_do_concat_batches`: True
- `eval_use_gather_object`: False
- `eval_accumulation_steps`: None
- `include_for_metrics`: []
- `batch_eval_metrics`: False
- `save_only_model`: False
- `save_on_each_node`: False
- `enable_jit_checkpoint`: False
- `push_to_hub`: False
- `hub_private_repo`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_always_push`: False
- `hub_revision`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `restore_callback_states_from_checkpoint`: False
- `full_determinism`: False
- `seed`: 42
- `data_seed`: None
- `use_cpu`: False
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `parallelism_config`: None
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `dataloader_prefetch_factor`: None
- `remove_unused_columns`: True
- `label_names`: None
- `train_sampling_strategy`: random
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `ddp_static_graph`: None
- `ddp_backend`: None
- `ddp_timeout`: 1800
- `fsdp`: None
- `fsdp_config`: None
- `deepspeed`: None
- `debug`: []
- `skip_memory_metrics`: True
- `do_predict`: False
- `resume_from_checkpoint`: None
- `warmup_ratio`: None
- `local_rank`: -1
- `prompts`: {'question': '[Q] ', 'passage_text': '[D] '}
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional
- `router_mapping`: {}
- `learning_rate_mapping`: {}
- `max_length`: 8192
</details>
### Training Logs
| Epoch | Step | Training Loss | monitor_maxsim_ndcg@10 | final_maxsim_ndcg@10 |
|:------:|:----:|:-------------:|:----------------------:|:--------------------:|
| 0.0500 | 391 | 0.3596 | - | - |
| 0.1001 | 782 | 0.0226 | 0.9731 | - |
| 0.1501 | 1173 | 0.0196 | - | - |
| 0.2002 | 1564 | 0.0187 | 0.9733 | - |
| 0.2502 | 1955 | 0.0150 | - | - |
| 0.3003 | 2346 | 0.0158 | 0.9752 | - |
| 0.3503 | 2737 | 0.0141 | - | - |
| 0.4004 | 3128 | 0.0135 | 0.9806 | - |
| 0.4504 | 3519 | 0.0123 | - | - |
| 0.5004 | 3910 | 0.0121 | 0.9809 | - |
| 0.5505 | 4301 | 0.0118 | - | - |
| 0.6005 | 4692 | 0.0119 | 0.9776 | - |
| 0.6506 | 5083 | 0.0106 | - | - |
| 0.7006 | 5474 | 0.0103 | 0.9837 | - |
| 0.7507 | 5865 | 0.0085 | - | - |
| 0.8007 | 6256 | 0.0083 | 0.9809 | - |
| 0.8508 | 6647 | 0.0077 | - | - |
| 0.9008 | 7038 | 0.0076 | 0.9835 | - |
| 0.9509 | 7429 | 0.0069 | - | - |
| 1.0 | 7813 | - | 0.9819 | - |
| -1 | -1 | - | - | 0.9608 |
### Training Time
- **Training**: 14.1 hours
- **Evaluation**: 25.3 minutes
- **Total**: 14.5 hours
### Framework Versions
- Python: 3.11.13
- Sentence Transformers: 6.0.0
- Transformers: 5.14.1
- PyTorch: 2.11.0+cu128
- Accelerate: 1.5.2
- Datasets: 3.5.0
- Tokenizers: 0.22.2
## Citation
### BibTeX
#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
```
#### CachedMultiVectorMultipleNegativesRankingLoss
```bibtex
@misc{gao2021scaling,
title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
year={2021},
eprint={2101.06983},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
```
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
--> |