dgwon/e5-small-v2.nfcorpus.meta-llama-Llama-3.2-3B-Instruct.confilt

SentenceTransformer based on intfloat/e5-small-v2

This is a sentence-transformers model finetuned from intfloat/e5-small-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: intfloat/e5-small-v2
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'query: How Do Fried Oils Affect Heart Health?',
    'passage: BACKGROUND AND AIM: Currently, more than 30% of the caloric intake in the Colombian population comes from vegetable oil consumption mainly by the ingestion of deep-fried foods. Recently, it has been reported that unsaturated fatty acid rich oils have a beneficial effect on the endothelial function. Nevertheless, it is well know that the deep-frying process alters the chemical composition of vegetable oils and can produce adverse effects in the endothelial function. OBJECTIVE: To evaluate the acute effect of the ingestion of large amounts of olive, soybean and palm oils, fresh and at two different deep-fry levels, on the glucose and lipid profiles and the endothelial function. METHODS AND RESULTS: Ten healthy young volunteers were included in the study. After performing a baseline evaluation of cardiovascular risk factors and drawing a fasting blood sample, subjects were exposed to a randomly assigned potato soup meal containing 60 mL of one of three different vegetable oils (olive, soybean and palm), either fresh or at one of two different deep-fry levels (10 and 20 fries, respectively). Flow-mediated vasodilation (FMD) was performed in fasting conditions and 3h after the intake of the oil rich meal. Furthermore, blood samples were taken at these stages for the lipid profiles and plasma glucose determinations. All the meals resulted in a similar acute endothelial impairment (FMD decrease of 32.1%, confidence interval [CI] 95%, 28.0-36.2) and postprandial increase in triglycerides (27.03%, CI 95%, 20.5-33.3), independently of the type of oil ingested (p=0.44) and regardless of its deep-fry level (p=0.62). No correlation was found between endothelial impairment and postprandial triglyceride increment (r=-0.22, p=0.09). CONCLUSIONS: No difference was found in the acute adverse effect of the ingestion of different vegetable oils on the endothelial function. All the vegetable oils, fresh and deep-fried, produced an increase in the triglyceride plasma levels in healthy subjects.',
    'passage: This case-control study examined different food groups in relation to breast cancer. Between 2002 and 2004, 437 cases and 922 controls matched according to age and area of residence were interviewed. Diet was measured by a validated food frequency questionnaire. Adjusted odds ratios (Ors) were computed across levels of various dietary intakes identified by two methods: the "classical" and the "spline" methods. Neither of the 2 methods found an association between total fruit and vegetable consumption and breast cancer. Results of the 2 methods showed a nonsignificant decreased association with cooked vegetables intake as well as legumes and fish consumption. Whereas the spline method showed no association, the classical method showed significant associations related to the lowest consumption of raw vegetables or dairy products and breast cancer risk: Adjusted OR for raw vegetable consumption between (67.4 and 101.3 g/day) vs. (< 67.4 g/day) was 0.63 [95% confidence interval (CI) = 0.43-0.93]. Adjusted OR for dairy consumption between (134.3 and 271.2 g/day) vs. (< 134.3 g/day) was 1.57 (95% CI = 1.06-2.32). However, the overall results were not consistent. Compared to the classical method, the use of the spline method showed a significant association for cereal, meat, and olive oil. Cereal and olive oil were inversely associated with breast cancer risk. Breast cancer risk increased by 56% for each additional 100 g/day of meat consumption. Studies using novel methodological techniques are needed to confirm the dietary threshold responsible for changes in breast cancer risk. New approaches that consist in analyzing dietary patterns rather than dietary food are necessary.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.7551, 0.0129],
#         [0.7551, 1.0000, 0.0260],
#         [0.0129, 0.0260, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 20,291 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 7 tokens mean: 13.06 tokens max: 27 tokens	min: 23 tokens mean: 326.45 tokens max: 512 tokens

• Samples:

  anchor	positive
  query: Can the Mediterranean Diet Really Increase My Life Expectancy?	passage: Longevity is a very complex phenomenon, because many environmental, behavioral, socio-demographic and dietary factors influence the physiological pathways of aging and life-expectancy. Nutrition has been recognized to have an important impact on overall mortality and morbidity; and its role in extending life expectancy has been the object of extensive scientific research. This paper reviews the pathophysiological mechanisms that potentially link aging with diet and the scientific evidence supporting the anti-aging effect of the traditional Mediterranean diet, as well as of some specific foods. The diet and several of its components have additionally been shown to have beneficial effects on the co-morbidities typical of elderly populations. Furthermore, the epigenetic effects of diet on the aging process - through calorie restriction and the consumption of foods like red wine, orange juice, probiotics and prebiotics - have attracted scientific interest. Some, such as dark choco...
  query: Does Eating Chocolate Make You Fat?	passage: Objective Habitual chocolate intake was recently found to be associated with lower body weight in three cross-sectional epidemiological studies. Our objective was to assess whether these cross-sectional results hold up in a more rigorous prospective analysis. Methods We used data from the Atherosclerosis Risk in Communities cohort. Usual dietary intake was assessed by questionnaire at baseline (1987–98), and after six years. Participants reported usual chocolate intake as the frequency of eating a 1-oz (∼28 g) serving. Body weight and height were measured at the two visits. Missing data were replaced by multiple imputation. Linear mixed-effects models were used to evaluate cross-sectional and prospective associations between chocolate intake and adiposity. Results Data were from 15,732 and 12,830 participants at the first and second visit, respectively. More frequent chocolate consumption was associated with a significantly greater prospective weight gain over time, in a dose-...
  query: Can Algal DHA Cure Breast Cancer?	passage: Docosahexaenoic acid (DHA) is an omega-3 fatty acid that comprises 22 carbons and 6 alternative double bonds in its hydrocarbon chain (22:6omega3). Previous studies have shown that DHA from fish oil controls the growth and development of different cancers; however, safety issues have been raised repeatedly about contamination of toxins in fish oil that makes it no longer a clean and safe source of the fatty acid. We investigated the cell growth inhibition of DHA from the cultured microalga Crypthecodinium cohnii (algal DHA [aDHA]) in human breast carcinoma MCF-7 cells. aDHA exhibited growth inhibition on breast cancer cells dose-dependently by 16.0% to 59.0% of the control level after 72-h incubations with 40 to 160 microM of the fatty acid. DNA flow cytometry shows that aDHA induced sub-G(1) cells, or apoptotic cells, by 64.4% to 171.3% of the control levels after incubations with 80 mM of the fatty acid for 24, 48, and 72 h. Western blot studies further show that aDHA did no...

• Loss: CachedMultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "mini_batch_size": 16,
      "gather_across_devices": false
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 2,255 evaluation samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 8 tokens mean: 13.21 tokens max: 26 tokens	min: 23 tokens mean: 324.9 tokens max: 512 tokens

• Samples:

  anchor	positive
  query: What Are Antimutagens?	passage: In this review recent publications are cited for a number of antimutagens. The molecules surveyed are potential or proven "desmutagens" or "interceptors." These are biologically prevalent or synthetic molecules that are most often small metabolites proficient in binding to, or reacting with, mutagenic chemicals and free radicals. Many of this class of "blocking agents" are "soft" and "hard" nucleophiles with consequently varying abilities to react with particular classes of electrophiles, the major classes of direct-acting mutagens. Although they serve as a first line of defense against mutagens and carcinogens, many interceptor molecules are under-investigated with regard to their spectra of activity and their possible relevance to prophylaxis or treatment of human disease states.
  query: How Does Not Breastfeeding Affect Health Later In Life?	passage: OBJECTIVE: To assess the public health significance of premature weaning of infants from breast milk on later-life risk of chronic illness. DESIGN: A review and summary of recent meta-analyses of studies linking premature weaning from breast milk with later-life chronic disease risk is presented followed by an estimation of the approximate exposure in a developed Western country, based on historical breast-feeding prevalence data for Australia since 1927. The population-attributable proportion of chronic disease associated with current patterns of artificial feeding in infancy is estimated. RESULTS: After adjustment for major confounding variables, current research suggests that the risks of chronic disease are 30-200 % higher in those who were not breast-fed compared to those who were breast-fed in infancy. Exposure to premature weaning ranges from 20 % to 90 % in post-World War II age cohorts. Overall, the attributable proportion of chronic disease in the population is estim...
  query: Do Jerte Valley Cherries Help You Sleep Better?	passage: Tryptophan, serotonin, and melatonin, present in Jerte Valley cherries, participate in sleep regulation and exhibit antioxidant properties. The effect of the intake of seven different Jerte Valley cherry cultivars on the sleep-wake cycle, 6-sulfatoxymelatonin levels, and urinary total antioxidant capacity in middle-aged and elderly participants was evaluated. Volunteers were subjected to actigraphic monitoring to record and display the temporal patterns of their nocturnal activity and rest. 6-sulfatoxymelatonin and total antioxidant capacity were quantified by enzyme-linked immunosorbent assay and colorimetric assay kits, respectively. The intake of each of the cherry cultivars produced beneficial effects on actual sleep time, total nocturnal activity, assumed sleep, and immobility. Also, there were significant increases in 6-sulfatoxymelatonin levels and total antioxidant capacity in urine after the intake of each cultivar. These findings suggested that the intake of Jerte Va...

• Loss: CachedMultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "mini_batch_size": 16,
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 128
• learning_rate: 2e-05
• num_train_epochs: 30
• warmup_ratio: 0.1
• fp16: True
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 128
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 30
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss
1.0	159	1.4525	0.0076
2.0	318	0.0854	0.0056
3.0	477	0.061	0.0042
4.0	636	0.0465	0.0039
5.0	795	0.0377	0.0036
6.0	954	0.0366	0.0029
7.0	1113	0.0325	0.0030
8.0	1272	0.0296	0.0026
9.0	1431	0.0254	0.0029
10.0	1590	0.0263	0.0024
11.0	1749	0.0247	0.0027
12.0	1908	0.0252	0.0022
13.0	2067	0.0224	0.0023
14.0	2226	0.0205	0.0023
15.0	2385	0.0213	0.0024

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.1.2
• Transformers: 4.57.1
• PyTorch: 2.8.0+cu126
• Accelerate: 1.11.0
• Datasets: 4.0.0
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CachedMultipleNegativesRankingLoss

@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}