semaj83/ctmatch-retriever-v2

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 384, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("semaj83/ctmatch-retriever-v2")
# Run inference
sentences = [
    "A 39-year-old man came to the clinic with cough and shortness of breath that was not relieved by his inhaler. He had these symptoms for 5 days during the past 2 weeks. He doubled his oral corticosteroids in the past week. He is a chef with a history of asthma for 3 years, suffering from frequent cough, wheezing, and shortness of breath and chest tightness. The symptoms become more bothersome within 1-2 hours of starting work every day and worsen throughout the work week. His symptoms improve within 1-2 hours outside the workplace. Spirometry was performed revealing a forced expiratory volume in the first second (FEV1) of 63% of the predicted. His past medical history is significant for seasonal allergic rhinitis in the summer. He doesn't smoke or use illicit drugs. His family history is significant for asthma in his father and sister. He currently uses inhaled corticosteroid (ICS) and fluticasone 500 mcg/salmeterol 50 mcg, one puff twice daily.",
    'Inclusion Criteria: for all subjects Age between 18 and 45 years old Smoking history ≤2 packyears. Specific for the two groups Group 1. Patients with ongoing asthma Age of onset of asthmatic symptoms: 0 years Documented history of asthma diagnosed according to latest GINA guidelines, i.e. respiratory symptoms and either bronchodilator reversibility (improvement in FEV1 of more than 12% of baseline (and at least 200 mL) after inhalation of 800 µg salbutamol) Use of inhaled corticosteroids or either persistent symptoms of wheeze, cough, or dyspnea or regular use of β2 agonists at least once a week during the last 2 months PC20 methacholine < 8 mg/ml Group 2. Non-asthmatic controls No history of asthma No use of inhaled corticosteroids or β2-agonists for a period longer than 1 month No symptoms of wheeze, nocturnal dyspnea, or bronchial hyperresponsiveness PC20 methacholine > 8 mg/ml, FEV1/FVC > 70% and FEV1 > 80% predicted, Exclusion Criteria: FEV1 <1.2 L Subjects must be able to adhere to the study visit schedule and other protocol requirements A subject is not eligible to enter and participate if he has not signed and dated a written informed consent form prior to participation in the study A subjects is not eligible to enter and participate if he does not agree that we inform his general practitioner Upper respiratory tract infection (e.g. colds), within 6 weeks Serious acute infections (such as hepatitis, pneumonia or pyelonephritis) in the previous 3 months Signs or symptoms of severe, progressive or uncontrolled renal, hepatic, hematologic, endocrine, pulmonary, cardiac, neurologic or cerebral disease Malignancy within the past 5 years (except for squamous or basal cell carcinoma of the skin that has been treated with no evidence of recurrence) Known recent substance abuse (drug or alcohol) Females of childbearing potential without an efficient contraception unless they meet the following definition of post-menopausal: 12 months of natural (spontaneous) amenorrhea or 6 months of spontaneous amenorrhea with serum FSH >40 mIU/mL or the use of one or more of the following acceptable methods of contraception: 1. Surgical sterilization (e.g. bilateral tubal ligation, hysterectomy). 2. Hormonal contraception (implantable, patch, oral, injectable). 3. Barrier methods of contraception: condom or occlusive cap (diaphragm or cervical/vault caps) with spermicidal foam/gel/cream/suppository. 4. Continuous abstinence',
    'Inclusion Criteria: Type 1 diabetes mellitus (T1DM) for at least 1 year and using insulin glargine for at least 6 months with a maximum daily dose of 1 unit per kilogram (U/kg) Hemoglobin A1c (HbA1c) of no greater than 10.5% before randomization Body mass index (BMI) 19 to 45 kilogram per square meter (kg/m²) Capable and willing to prepare and inject insulin with a syringe, monitor own blood glucose, complete the study diary, be receptive to diabetes education, comply with study requirements, and receive telephone calls during treatment Women of childbearing potential must test negative for pregnancy before receiving treatment and agree to use reliable birth control until completing the follow-up, Exclusion Criteria: Twice daily use of insulin glargine within 30 days prior to the study Use of any oral or injectable medication intended for the treatment of diabetes mellitus other than insulins in the 3 months prior to the study Use of an insulin pump More than 1 episode of severe hypoglycemia within 3 months prior to the study, or currently diagnosed as having hypoglycemia unawareness or more emergency room visits or hospitalizations due to poor glucose control in the 6 months preceding the study Known hypersensitivity or allergy to any of the study insulins or their excipients Blood transfusion or severe blood loss within 3 months prior to the study or known hemoglobinopathy, hemolytic anemia, or sickle cell anemia, or any other traits of hemoglobin abnormalities known to interfere with the HbA1c methodology Irregular sleep/wake cycle Pregnant or intend to become pregnant during the study Women who are breastfeeding Use of prescription or over-the-counter medications to promote weight loss within 3 months prior to the study Current participation in a weight loss program or plans to do so during the study Use of chronic (lasting longer than 14 consecutive days) systemic glucocorticoid therapy currently or within 4 weeks prior to the study Cardiac disease with a marked impact on physical functioning Clinically significant electrocardiogram (ECG) abnormalities at screening Fasting triglycerides greater than 500 milligram per deciliter (mg/dL) Liver disease History of renal transplantation, current renal dialysis, or creatinine greater than 2.0 mg/dL (177 micromole per liter [μmol/L]) Malignancy other than basal cell or squamous cell skin cancer, currently or within the last 5 years Treatment with any antibody-based therapy within 6 months prior to the study',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.7314, 0.2627],
#         [0.7314, 1.0000, 0.2874],
#         [0.2627, 0.2874, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 4,830 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 100 samples:

  	sentence_0	sentence_1
  type	string	string
  modality	text	text
  details	min: 57 tokens mean: 192.12 tokens max: 256 tokens	min: 14 tokens mean: 140.79 tokens max: 256 tokens

• Samples:

  sentence_0	sentence_1
  Patient A is a 30-year-old male who was admitted to the hospital after 10 days of cough, profuse nocturnal sweating and loss of appetite. He had traveled to India 1 months ago and has not any positive history of TB vaccination. He is a previously healthy man, working as an engineer in a high tech company. He doesn't smoke o use any illicit drugs. He was febrile (38 c) with heart rate of 115 b/min, respiratory rate of 22, BP of 125/75 mmHg and O2 sat of 97%. Chest X-ray showed infiltrate in the middle of left lung with diameter of 1.8 cm with signs of cavitation. The sputum smear revealed positive sputum culture for Mycobacterium tuberculosis which are sensitive of the first-line TB drugs (isoniazid, streptomycin, rifampicin and ethambutol). Lab study is reported bellow: Hgb: 13 g/dl WBC: 14000 /mm3 Plt: 300000 /ml AST: 13 U/L ALT: 15 U/L Alk P: 53 U/L Bill total: 0.6 mg/dl Na: 137 mEq/l K: 4 mEq/l Creatinine: 0.5 mg/dl BUN: 10 mg/dl ESR: 120 mm/hr	Inclusion Criteria: Any patient requiring Tuberculosis treatment, Exclusion Criteria: Those refusing consent; the healthcare professional deems the patient that will not be able to comply with VOT
  A 44-year-old man was recently in an automobile accident where he sustained a skull fracture. In the emergency room, he noted clear fluid dripping from his nose. The following day he started complaining of severe headache and fever. Nuchal rigidity was found on physical examination.	Inclusion Criteria: Probable bacterial meningitis patients: Clinical manifestation (Any person with sudden onset of fever (> 38.5 °C rectal or 38.0 °C axillary) and one of the following signs: neck stiffness, altered consciousness or other meningeal sign) with cerebrospinal fluid examination showing at least one of the following: A. turbid appearance; B.leukocytosis (> 100 cells/mm3); C.leukocytosis (10-100 cells/ mm3) AND either an elevated protein (> 100 mg/dl) or decreased glucose (< 40 mg/dl) Confirmed bacterial meningitis patients: A case that is laboratory-confirmed by growing (i.e. culturing) or identifying (i.e. by Gram stain or antigen detection methods) a bacterial pathogen (Hib, pneumococcus or meningococcus) in the cerebrospinal fluid or from the blood in a child with a clinical syndrome consistent with bacterial meningitis, Exclusion Criteria: Congenital immunodeficiency patients HIV patients Patients with corticosteroid treatment for long time Patients with disorders in a...
  A 38 year old woman complains of severe premenstrual and menstrual pelvic pain, heavy, irregular periods and occasional spotting between periods. Past medical history remarkable for two years of infertility treatment and an ectopic pregnancy at age 26.	Inclusion Criteria: female >18 years fluent German Cases: diagnosis of endometriosis Control 1: no endometriosis, no chronic pain Control 2: no endometriosis, chronic abdominal/pelvic pain, Exclusion Criteria: male (Except for partner questionnaires) <18years not fluent in German

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• per_device_train_batch_size: 64
• num_train_epochs: 3
• max_steps: -1
• learning_rate: 5e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1
• label_smoothing_factor: 0.0
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 64
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Time

• Training: 55.5 seconds

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 5.6.0
• Transformers: 5.12.1
• PyTorch: 2.11.0+cu128
• Accelerate: 1.14.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}