doaahammud/lora_model_Fine_tuning_Embedding_embeddinggemma_300m

SentenceTransformer based on unsloth/embeddinggemma-300m

This is a sentence-transformers model finetuned from unsloth/embeddinggemma-300m on the generator dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: unsloth/embeddinggemma-300m
• Maximum Sequence Length: 1024 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • generator

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 1024, 'do_lower_case': False, 'architecture': 'PeftModelForFeatureExtraction'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (4): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("doaahammud/lora_model_Fine_tuning_Embedding_embeddinggemma_300m")
# Run inference
queries = [
    "What are the systemic manifestations of Fournier\u0027s syndrome?",
]
documents = [
    "Fournier's syndrome is defined as a suppurative bacterial infection of the perineal, genital, or perianal regions. Those conditions often lead to thrombosis of subcutaneous vessels and with infection, resulting in the development of gangrene of the overlying skin and subcutaneous tissue [1] . This rare syndrome is a rapidly progressive and potentially lethal necrotizing fasciitis caused by invasive infections of the lower part of the genitourinary tract, anorectal soft tissue, and genital skin [1, 2] . The devastating rapidity is typical, as evidenced by the fact that the mean duration of symptoms to become the target of emergency operation is just a few days, and a majority of patients are seriously ill at the time of admission. Anesthetic management of patients with this syndrome is often difficult, due to its devastating nature as well as significant comorbid diseases. However, because of the infrequency of the syndrome, there is limited information regarding the anesthetic management of this disease. We recently encountered the anesthetic management in three cases of patients with Fournier's syndrome. There were three initial emergency and six additional elective operations under general anesthesia, except one spinal anesthesia in an elective case. Therefore, we report these cases and review the relevant literatures.\n\n Immediate and, if required, repetitive operation is important for saving lives in patients with this syndrome [1] [2] [3] . Fournier's syndrome is frequently associated with certain diseases and conditions. Diabetes mellitus is probably the most common comorbid disease, as evidenced by our cases [1] . Even when the patient has diabetes, as in our two patients, Fournier's syndrome might be the first clinical disease to be detected. The second common condition is alcoholism, such as in all our patients, because any disorder that compromises the immunity enhances development of a severe infection [1, 2] . The other associated clinical features are malnutrition, prolonged hospitalization, radiation therapy, chemotherapy, neurologic deficits, cirrhosis, leukemia, renal failure, organic heart disease, vasculitis, intravenous drug abuse, lupus, cirrhosis, AIDS and steroid medications. In obstetric anesthesia, cervical or pudendal nerve block can induce the syndrome as well [1] .\n\n Abnormal laboratory results include hyperglycemia, hypocalcemia, anemia, leukocytosis and thrombocytopenia, as evidenced by our patients [1] . Most of those abnormalities are due to sepsis. The systemic manifestations include fever, tachycardia, and volume depletion similar to those of severe peritonitis [1] . All our patients also had sepsis in terms of the preoperative definition. Two patients looked to be in late distributive shock and the other patient in early distributive shock, respectively. In the case of no active bleeding, delayed or inadequate volume resuscitation is a significant error that would have detrimental effects on the patients's outcome in septic shock. If initial crystalloid fluid resuscitation is insufficient to raise the mean arterial pressure to 65 mmHg and the CVP to 8 to 12 mmHg, then vasopressors and inotropes are needed as the second step in the guidelines of early goal-directed therapy [4] . It is rational to use a blood transfusion when the hematocrit is below 30% when invasive monitoring might be indicated [4] . Among two patients in late septic shock, one patient fortunately responded to our initial fluid resuscitation, whereas the other patient needed dopamine for hypotension. In another patient, early shock occurred, and a blood transfusion and dopamine and norepinephrine were required to achieve an adequate cardiac output and oxygen delivery to maintain vital organ function were needed, because his affected area including www.ekja.org\n\n Vol. 61, No. 2, August 2011 lower extremity was wide and bleeding was ongoing. His septic manifestations reoccurred sporadically over four months of hospitalization and progressed into cardiorespiratory collapse and death after five debridements under general anesthesia. The preanesthetic investigation of the extent of the lesion is also important, because the ambiguity of the region involved could influence the choice of the anesthetic technique. Koitabashi and colleagues [5] suggested the avoidance of spinal anesthesia in the presence of lumbar subcutaneous gas. Sato and associates [3] recommended that using general anesthesia is preferable for controlling physiologic homeostasis. Fournier's syndrome in particular originateed in anorectal disease, which is the usual subject of regional anesthetic procedures, is known to be aggressive, produces marked systemic toxicity and myonecrosis as in our mortal case, and can be connected with higher mortality [1] . Hence, particular attention should be necessary for the choice of the anesthetic procedures. The reason we performed spinal anesthesia at the secondary wound closure one month after initial debridement in one patient was that he underwent computer tomography to depict the accurate extent of the lesion. Among gravely ill patients it seems wise to not waste precious time doing a lot of investigation to perform regional anesthesia.\n\n The degree of debridement for Fournier's syndrome is variable from simple incision to wide excision with massive bleeding [2] . Many surviving patients require secondary wound closure, skin graft or a reconstructive flap procedure. In spite of appropriate therapy, the mortality rate in Fournier's syndrome is reported to exceed 40% in many studies [1] [2] [3] . Prolonged sepsis manifested by fever or hypotension and lasting for more than 48 hours was experienced among about 40% of patients, as in our expired patient [2] . Some reports have associated older age, female gender, anorectal causes, delayed admission, the presence of debilitating conditions such as renal failure and hepatic dysfunction with high mortality. Laboratory parameters on admission statistically related to fatality include low hematocrit, calcium, albumin, and cholesterol, and high BUN and alkaline phosphatase levels. The syndrome could rapidly progress into prolonged sepsis, DIC, pneumonia, respiratory failure, diabetic ketoacidosis, renal failure, and heart failure.\n\n In conclusion, our experiences emphasize that Fournier's syndrome has a fatal potential, so optimal conduct of anesthesia requires forethought and sound management as well as an understanding of the pathophysiology of this syndrome for successful anesthesia.",
    'Nucleotides can be synthesized de novo or recycled through a salvage pathway in vivo. In the salvage pathway, nucleotides are synthesized from extracellular nucleosides and/or nucleobases. The plasma membrane transport of nucleosides is concerned with both the physiology and pharmacology of mammalian cells. Most mammalian cells simultaneously express several nucleoside transporters (NTs) in the plasma membrane. NTs possess certain differences in Na + -dependency, permeation selectivity, and inhibitor sensitivity. NTs can be divided into two major classes: concentrative Na + -dependent nucleoside transporters (CNTs) and equilibrative Na + -independent nucleoside transporters (ENTs). CNTs are nucleoside/Na + symporters that transport nucleosides against their concentration * To whom correspondence should be addressed: Laboratory of Chemical Toxicology and Environmental Health, Showa Pharmaceutical University, 3-3165 Higashi-Tamagawagakuen, Machida, Tokyo 194-8543, Japan. Tel. & Fax: +81-42-721-1563; E-mail: ogra@ac.shoyaku.ac.jp gradients. In contrast, ENTs transport nucleosides by facilitated diffusion. ENTs can be further divided into two subclasses depending on their sensitivity to nitrobenzylthioinosine (NBTI). NBTI-sensitive and NBTI-insensitive forms are coded by ENT1 and ENT2 genes, respectively. ENT1 and ENT2 are inhibited by dipyridamole and dilazep. 1) On the other hand, CNTs can be divided into three forms. No specific pharmacological inhibitors have been identified for any CNTs so far. It was reported that dipyridamole, a classic NT inhibitor, was useful in enhancing the effectiveness of cancer chemotherapeutic agents, in particular, antimetabolites, based upon the inhibition of the nucleoside salvage pathway. 2, 3) Hence, the combination of NT inhibitors and antimetabolites is expected to provide more effective and safer cancer chemotherapy in the clinical setting.\n\n Cimicifugoside, a triterpenoid originating from the rhizomes of Cimicifuga simplex (C. simplex), has been used in traditional Chinese medicine as the so-called Cimicifugae rhizoma (Fig. 1) . The medicine is prescribed because of its anti-C 2011 The Pharmaceutical Society of Japan inflammatory, analgesic, and anti-pyretic effects. 4) In addition, it has been reported that cimicifugoside selectively inhibits the uptake of nucleosides into phytohematoagglutinin-stimulated human lymphocytes and several malignant cell lines. 5, 6) Indeed, the uptake of nucleosides, such as uridine, thymidine, and adenosine, but not nucleobases was inhibited by cimicifugoside and its analogs, such as cimicifugenin and bugbanosides A and B, in a leukemia cell line. 7) As the mechanism underlying the inhibition of nucleoside transport by cimicifugoside, it is speculated that cimicifugoside has weak affinity for the binding site of NBTI in ENTs, although the detailed inhibitory mechanism is still unclear. Furthermore, cimicifugoside and its analogs potentiated the cytotoxicity of methotrexate, a folic acid antimetabolite. In our previous study, we demonstrated that the synergic effect of methotrexate and cimicifugoside. 7) In this study, we intended to clarify the mechanisms underlying the cell-specific synergic effect of cimicifugoside on the cytotoxicity of methotrexate. We focused on the involvement of NT expression and activity in the cell lines. Cell Culture --The human promonocytic leukemia cell line U937 and the chronic myelogenetic leukemia cell line K562 were provided by the Cell Resource Center for Biomedical Research (Institute of Development, Aging and Cancer, Tohoku University, Sendai, Japan). Cells were cultured in RPMI 1640 medium supplemented with 5% (v/v) heat-inactivated fetal bovine serum (FBS), 60 µg/ml kanamycin and 2 mM L-glutamine (culture medium) at 37 • C in a humidified atmosphere of 5% CO 2 and 95% air.',
    'tissue infiltration of polymorphonuclear neutrophils (PMN). In response to bacterial infection, neutrophil activation in the lungs leads to ALI (53) and PMN infiltration represents a primary mechanism for sepsis-induced pulmonary dysfunction and injury (10) .\n\n In the normal lung, continuous fluid clearance by the lung lymphatics is essential for the maintenance of dry alveolar surfaces (60) . Ion pumps and channels positioned on alveolar epithelial cell surfaces generate transepithelial osmotic gradients that drive water movement from the alveolar space into the lung interstitium. The key pumps and channels involved in alveolar fluid transport include aquaporin 5 (Aqp5), cystic fibrosis transmembrane conductance regulator (CFTR), epithelial sodium channel (ENaC), and Na ϩ -K ϩ -ATPase (37) . In ALI, the lung endothelial barrier is damaged, resulting in abnormal capillary permeability and pulmonary edema as the lymphatic clearance is overwhelmed (60) . In contrast to the endothelium, the alveolar epithelium is often spared in sepsisinduced ALI, and therefore active ion and fluid clearance is preserved (56) . Recent reports, however, suggest that ion pump and channel functions are affected early during sepsis (37) .\n\n To maintain a "dry" alveolar space and normal lung function, it is essential that the milieu within the alveolar space remain distinct from that of the subepithelial compartment (30) . The maintenance mechanism depends on the formation and proper functioning of specialized molecular structures between adjacent cells comprising the epithelial sheet, the so-called tight junctions (TJ). The alveolar epithelial TJ is a complex of integral membrane proteins that firmly interact with the epithelial cytoskeleton (27) . TJs serve as a regulated semipermeable barrier that limits passive diffusion of solutes across paracellular pathways between adjacent cells (1). Han et al. (30) recently showed that ALI was associated with diminished expression and function of TJ proteins in lung epithelium.\n\n Emerging evidence indicates that inflammation and coagulation are connected (34) . This is especially important in sepsis as inflammatory cytokines activate the coagulation cascade and inhibit fibrinolysis, thereby shifting normal hemostasis toward a prothrombotic state. Sepsis-driven coagulation induces consumption of coagulation factors leading to disseminated intravascular coagulation (DIC), a phenomenon frequently associated with sepsis-induced ALI. Indeed, impairment of capillary blood flow during sepsis has been observed in human tissues by orthogonal polarization spectral imaging and sidestream dark-field imaging (17) . It is currently estimated that as many as 50% of all sepsis patients develop DIC (25) . Bastarache et al. (6) have recently reported that in ALI, the alveolar compartment contains high levels of tissue factor (TF) procoagulant activity that favor fibrin deposition in the air spaces. TF activation results in thrombin formation, which augments permeability and enhances inflammation (14) .\n\n Vitamin C is a small, water-soluble molecule that readily acts as a one-or two-electron reducing agent for many radicals and oxidants. Vitamin C is bioavailable equally as either dehydro-L-ascorbic acid (DHA) or L-ascorbic acid (AscA). Specialized cells can take up reduced vitamin C (AscA) through Na ϩ -dependent ascorbate cotransporters (SVCT1 and SVCT2). Most other cells take up vitamin C in its oxidized form (DHA) via facilitative glucose transporters (48) . Sepsis lowers plasma AscA concentrations (57) and, importantly, low vitamin C levels correlate inversely with multiple organ failure and directly with survival (9) . Studies using animal models show that vitamin C prevents endotoxin-induced hypotension and improves arteriolar responsiveness, arterial blood pressure, capillary blood flow, liver function, and survival in experimental sepsis (4, 59) .\n\n We recently showed that vitamin C, administered after the onset of endotoxemia, attenuates proinflammatory and procoagulant states that induce lung vascular injury and improved survival in an animal model of sepsis (23) . In the present study we show that vitamin C attenuates sepsis-induced ALI by enhancing alveolar epithelial barrier integrity. Furthermore, vitamin C induced the expression of ion channels and pumps, which play critical roles in improving alveolar fluid clearance. In addition, we also observed marked changes in the viscoelastic clot properties of septic mice blood.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.4607, 0.0097, 0.1365]], dtype=torch.float16)

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.774
cosine_accuracy@3	0.883
cosine_accuracy@5	0.9145
cosine_accuracy@10	0.943
cosine_precision@1	0.774
cosine_precision@3	0.2943
cosine_precision@5	0.1829
cosine_precision@10	0.0943
cosine_recall@1	0.774
cosine_recall@3	0.883
cosine_recall@5	0.9145
cosine_recall@10	0.943
cosine_ndcg@10	0.8617
cosine_mrr@10	0.8353
cosine_map@100	0.8375

Training Details

Training Dataset

generator

• Dataset: generator
• Size: 10,000 training samples
• Columns: question and passage_text
• Approximate statistics based on the first 1000 samples:

  	question	passage_text
  type	string	string
  details	min: 7 tokens mean: 20.79 tokens max: 60 tokens	min: 481 tokens mean: 945.6 tokens max: 1024 tokens

• Samples:

  question	passage_text
  What factors may contribute to increased pulmonary conduit durability in patients who undergo the Ross operation compared to those with right ventricular outflow tract obstruction?	I n 1966, Ross and Somerville 1 reported the first use of an aortic homograft to establish right ventricle-to-pulmonary artery continuity in a patient with tetralogy of Fallot and pulmonary atresia. Since that time, pulmonary position homografts have been used in a variety of right-sided congenital heart lesions. Actuarial 5-year homograft survivals for cryopreserved homografts are reported to range between 55% and 94%, with the shortest durability noted in patients less than 2 years of age. 4 Pulmonary position homografts also are used to replace pulmonary autografts explanted to repair left-sided outflow disease (the Ross operation). Several factors may be likely to favor increased pulmonary conduit durability in Ross patients compared with those with right ventricular outflow tract obstruction, including later age at operation (allowing for larger homografts), more normal pulmonary artery architecture, absence of severe right ventricular hypertrophy, and more natural positioning of ...
  How does MCAM expression in hMSC affect the growth and maintenance of hematopoietic progenitors?	After culture in a 3-dimensional hydrogel-based matrix, which constitutes hypoxic conditions, MCAM expression is lost. Concordantly, Tormin et al. demonstrated that MCAM is down-regulated under hypoxic conditions. 10 Furthermore, it was shown by others and our group that oxygen tension causes selective modification of hematopoietic cell and mesenchymal stromal cell interactions in co-culture systems as well as influence HSPC metabolism. [44] [45] [46] Thus, the observed differences between Sharma et al. and our data in HSPC supporting capacity of hMSC are likely due to the different culture conditions used. Further studies are required to clarify the influence of hypoxia in our model system. Altogether these findings provide further evidence for the importance of MCAM in supporting HSPC. Furthermore, previous reports have shown that MCAM is down-regulated in MSC after several passages as well as during aging and differentiation. 19, 47 Interestingly, MCAM overexpression in hMSC enhance...
  What is the relationship between Fanconi anemia and breast and ovarian cancer susceptibility genes?	( 31 ) , of which 5% -10 % may be caused by genetic factors ( 32 ) , up to half a million of these patients may be at risk of secondary hereditary neoplasms. The historic observation of twofold to fi vefold increased risks of cancers of the ovary, thyroid, and connective tissue after breast cancer ( 33 ) presaged the later syndromic association of these tumors with inherited mutations of BRCA1, BRCA2, PTEN, and p53 ( 16 ) . By far the largest cumulative risk of a secondary cancer in BRCA mutation carriers is associated with cancer in the contralateral breast, which may reach a risk of 29.5% at 10 years ( 34 ) . The Breast Cancer Linkage Consortium ( 35 , 36 ) also documented threefold to fi vefold increased risks of subsequent cancers of prostate, pancreas, gallbladder, stomach, skin (melanoma), and uterus in BRCA2 mutation carriers and twofold increased risks of prostate and pancreas cancer in BRCA1 mutation carriers; these results are based largely on self-reported family history inf...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

generator

• Dataset: generator
• Size: 2,000 evaluation samples
• Columns: question and passage_text
• Approximate statistics based on the first 1000 samples:

  	question	passage_text
  type	string	string
  details	min: 7 tokens mean: 20.91 tokens max: 61 tokens	min: 465 tokens mean: 943.1 tokens max: 1024 tokens

• Samples:

  question	passage_text
  What are some hereditary cancer syndromes that can result in various forms of cancer?	Hereditary Cancer Syndromes, including Hereditary Breast and Ovarian Cancer (HBOC) and Lynch Syndrome (LS), can result in various forms of cancer due to germline mutations in cancer predisposition genes. While the major contributory genes for these syndromes have been identified and well-studied (BRCA1/ BRCA2 for HBOC and MSH2/MSH6/MLH1/PMS2/ EPCAM for LS), there remains a large percentage of associated cancer cases that are negative for germline mutations in these genes, including 80% of women with a personal or family history of breast cancer who are negative for BRCA1/2 mutations [1] . Similarly, between 30 and 50% of families fulfill stringent criteria for LS and test negative for germline mismatch repair gene mutations [2] . Adding complexity to these disorders is the significant overlap in the spectrum of cancers observed between various hereditary cancer syndromes, including many cancer susceptibility syndromes. Some that contribute to elevated breast cancer risk include Li-Frau...
  How do MAK-4 and MAK-5 exert their antioxidant properties?	Hybrid F1 mice were injected with urethane (300 mg/kg) at 8 days of age. A group was then put on a MAK-supplemented diet, another group was fed a standard pellet diet. At 36 weeks of age the mice were sacrificed and the livers examined for the presence of tumors mouse (Panel A) and for the number of nodules per mouse (Panel B) (* p < 0.05, ** P < 0.001). Statistical analysis was performed by Two Way ANOVA Test followed by Post Hoc Bonferroni analysis. We than measured the influence of the MAK-4+5 combination on the expression of the three liver-specific connexins (cx26, cx32, and cx43). The level of cx26 expression was similar in all the groups of mice treated with the MAK-supplemented diet and in the control (Figure 4, Panel A) . A significant, time-dependent increase in cx32 was observed in the liver of all the groups of MAK treated mice compared to the normal diet-fed controls. Cx32 expression increased 2-fold after 1 week of treatment, and 3-to 4-fold at 3 months (Figure 4, Pane...
  What are the primary indications for a decompressive craniectomy, and what role does neurocritical care play in determining the suitability of a patient for this procedure?	Decompressive craniectomy is a valid neurosurgical strategy now a day as an alternative to control an elevated intracranial pressure (ICP) and controlling the risk of uncal and/or subfalcine herniation, in refractory cases to the postural, ventilator, and pharmacological measures to control it. The neurocritical care and the ICP monitorization are key determinants to identify and postulate the inclusion criteria to consider a patient as candidate to this procedure, as it is always considered a rescue surgical technique. Head trauma and ischemic or hemorrhagic cerebrovascular disease with progressive deterioration due to mass effect are some of the cases that may require a decompressive craniectomy with its different variants. However, this procedure per se can have complications described in the postcraniectomy syndrome and may occur in short, medium, or even long term. 1,2 The paradoxical herniation is a condition in which there is a deviation of the midline with mass effect, even t...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• gradient_accumulation_steps: 2
• learning_rate: 2e-05
• num_train_epochs: 10
• max_steps: 30
• warmup_ratio: 0.03
• prompts: {'question': '', 'passage_text': ''}
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 2
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 10
• max_steps: 30
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.03
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: {'question': '', 'passage_text': ''}
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	cosine_ndcg@10
-1	-1	-	-	0.4325
0.0637	5	1.041	0.5049	0.7713
0.1274	10	0.438	0.2788	0.8315
0.1911	15	0.2167	0.2166	0.8483
0.2548	20	0.185	0.1893	0.8564
0.3185	25	0.1928	0.1743	0.8604
0.3822	30	0.1944	0.1691	0.8617
-1	-1	-	-	0.8617

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.2.2
• Transformers: 4.56.2
• PyTorch: 2.9.0+cu126
• Accelerate: 1.12.0
• Datasets: 4.3.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}