vaktibabat/e5-grid-search

StrictSentenceTransformer based on intfloat/e5-base-v2

This is a sentence-transformers model finetuned from intfloat/e5-base-v2 on the bge-multilingual-gemma2-data dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: intfloat/e5-base-v2
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • bge-multilingual-gemma2-data

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

StrictSentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("vaktibabat/e5-grid-search")
# Run inference
sentences = [
    'query: normal oximetry range',
    'passage: How Do I Interpret Oximeter Readings The pulse oximetry normal values are dependent on the patient`s health condition, respiratory rate, percentage of oxygen in the air, activities of the patients and other factors. Handheld Pulse Oximeter UT100. For a regular healthy human body, the normal blood oxygen saturation level (SpO2) should be around 94% to 99%.Patients who have mild respiratory diseases, the SpO2 should be 90% or above.or a regular healthy human body, the normal blood oxygen saturation level (SpO2) should be around 94% to 99%. Patients who have mild respiratory diseases, the SpO2 should be 90% or above.',
    "passage: Corkscrew Vine Propagation Cutting The blooms of the corkscrew vine resemble a snail's shell when viewed from the side. The corkscrew or snail vine (Vigna caracalla), a tender perennial vine, produces fragrant, spiral blooms. Apart from the unusual white or yellow and purple blooms, the plant has beanlike foliage that is evergreen in U.S. Department of Agriculture plant hardiness zones 9 through 12.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.7384, 0.1191],
#         [0.7384, 1.0000, 0.1328],
#         [0.1191, 0.1328, 1.0000]])

Training Details

Training Dataset

bge-multilingual-gemma2-data

• Dataset: bge-multilingual-gemma2-data at ef165e1
• Size: 484,365 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 6 tokens mean: 10.89 tokens max: 34 tokens	min: 24 tokens mean: 86.9 tokens max: 215 tokens

• Samples:

  anchor	positive
  query: how many days in lent	passage: - 40 Days of Lent. Lent starts on Ash Wednesday, lasts 40 days not counting Sundays and ends on Holy Saturday, the day before Easter. There are 46 days because Sundays are not included during Lent. Some say that Sundays aren't counted in the 40 days of Lent, because every Sunday carries with it a part of the glow of Easter Sunday.
  query: interest rates the last 30 days	passage: Mortgage Interest Rates Last 30 Days (Valuable Insights) The big news for mortgage interest rates last 30 days came during the second half of May as rates continued to tumble to new lows, falling first to 4.29 percent, then to 4.21 percent and finally closing out with Freddie Mac reporting rates of 4.12 percent for a 30-year fixed rate as of May 29.ong-term mortgage rates are still likely to creep up to around the 6 percent mark as 2015 draws to a close, but don’t expect that to shake the housing market or the economy in general.
  query: definition of echocardiogram	passage: Definitions &Translations Freebase(0.00 / 0 votes)Rate this definition: Echocardiography. Echocardiogram, often referred to cardiac echo or simply an echo is a sonogram of the heart. Echocardiography uses standard two-dimensional, three-dimensional, and Doppler ultrasound to create images of the heart.

• Loss: training.train_utils.loss.MNRLWithMeanNegativeTwoTerms

Evaluation Dataset

bge-multilingual-gemma2-data

• Dataset: bge-multilingual-gemma2-data at ef165e1
• Size: 1,458 evaluation samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 6 tokens mean: 11.04 tokens max: 87 tokens	min: 21 tokens mean: 88.46 tokens max: 228 tokens

• Samples:

  anchor	positive
  query: what was the significance of the second battle of the marne quizlet	passage: What Significance of the second battle of the Marne? The significance of this second battle was that it showed a close end to the First World war. As the fresh Americans came in to fight we pushed Germany back a second time. The significance of this second battle was that it showed a close end to the First World war. As the fresh Americans came in to fight we pushed Germany back a second time.
  query: how long do i pct	passage: How Long Does It Take To Hike The Pacific Crest Trail? To complete the PCT in two months (or less) you will need to average approximately 43.6 mi / 70.2 km per day (this was my longest day on trail).
  query: average water temperature in december in st. john	passage: Climate & Weather â When to visit St. John Climate & Weather – When to visit St. John. It is warm year-round on St. John. The average max temp. is 84-89°F, with August being the hottest month and Dec-Jan the coldest. The average min temp is 73°F-80°F. You will rarely need anything long, except for supermarkets, where it is freezing or to protect yourself from sunburn or bug bites. The average water temperature is 82°F.

• Loss: training.train_utils.loss.MNRLWithMeanNegativeTwoTerms

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 96
• learning_rate: 4e-05
• weight_decay: 0.01
• num_train_epochs: 1
• warmup_ratio: 0.1
• dataloader_num_workers: 15
• eval_on_start: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 96
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 4e-05
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 15
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: True
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss
0	0	-	0.1050
0.0002	1	0.9836	-
0.0020	10	1.3375	-
0.0040	20	1.387	-
0.0059	30	1.3184	-
0.0079	40	1.2462	-
0.0099	50	1.0014	-
0.0119	60	1.0369	-
0.0139	70	0.975	-
0.0159	80	0.7796	-
0.0178	90	1.0021	-
0.0198	100	0.8907	-
0.0218	110	0.7629	-
0.0238	120	0.945	-
0.0258	130	0.7851	-
0.0277	140	0.7527	-
0.0297	150	0.9915	-
0.0317	160	0.7314	-
0.0337	170	0.9877	-
0.0357	180	0.9458	-
0.0377	190	0.8852	-
0.0396	200	0.6089	-
0.0416	210	0.8883	-
0.0436	220	1.1748	-
0.0456	230	0.7057	-
0.0476	240	0.6926	-
0.0495	250	0.9279	-
0.0515	260	0.9746	-
0.0535	270	0.663	-
0.0555	280	1.0344	-
0.0575	290	0.9686	-
0.0595	300	0.4997	-
0.0614	310	0.8318	-
0.0634	320	0.8989	-
0.0654	330	0.8827	-
0.0674	340	0.5811	-
0.0694	350	0.8668	-
0.0713	360	0.6718	-
0.0733	370	0.8139	-
0.0753	380	0.693	-
0.0773	390	1.0242	-
0.0793	400	0.8008	-
0.0813	410	0.9236	-
0.0832	420	1.0689	-
0.0852	430	1.027	-
0.0872	440	0.8557	-
0.0892	450	0.9058	-
0.0912	460	1.0211	-
0.0931	470	0.9113	-
0.0951	480	0.8473	-
0.0971	490	0.9376	-
0.0991	500	0.6026	-
0.1011	510	0.94	-
0.1031	520	0.7082	-
0.1050	530	1.0271	-
0.1070	540	0.8847	-
0.1090	550	0.8367	-
0.1110	560	0.6785	-
0.1130	570	0.6599	-
0.1149	580	0.7667	-
0.1169	590	0.7403	-
0.1189	600	0.8816	-
0.1209	610	1.1463	-
0.1229	620	0.7646	-
0.1249	630	0.7321	-
0.1268	640	0.9198	-
0.1288	650	0.864	-
0.1308	660	0.7541	-
0.1328	670	0.9667	-
0.1348	680	0.9442	-
0.1367	690	0.5636	-
0.1387	700	0.6658	-
0.1407	710	0.6024	-
0.1427	720	0.97	-
0.1447	730	0.855	-
0.1467	740	0.7716	-
0.1486	750	0.6797	-
0.1506	760	0.7534	-
0.1526	770	0.9356	-
0.1546	780	0.7488	-
0.1566	790	0.7739	-
0.1585	800	0.8565	-
0.1605	810	0.9057	-
0.1625	820	0.8434	-
0.1645	830	0.8065	-
0.1665	840	0.6384	-
0.1685	850	1.0452	-
0.1704	860	0.7133	-
0.1724	870	0.6577	-
0.1744	880	0.7132	-
0.1764	890	0.9297	-
0.1784	900	0.5956	-
0.1803	910	0.648	-
0.1823	920	0.972	-
0.1843	930	1.0241	-
0.1863	940	0.9475	-
0.1883	950	1.039	-
0.1902	960	0.6771	-
0.1922	970	0.5779	-
0.1942	980	0.7125	-
0.1962	990	0.9354	-
0.1982	1000	0.7346	-
0.2002	1010	0.7185	0.0032
0.2021	1020	0.7416	-
0.2041	1030	0.9688	-
0.2061	1040	0.7711	-
0.2081	1050	0.945	-
0.2101	1060	0.7326	-
0.2120	1070	0.8058	-
0.2140	1080	0.7557	-
0.2160	1090	0.6534	-
0.2180	1100	0.7883	-
0.2200	1110	0.8062	-
0.2220	1120	0.6192	-
0.2239	1130	0.7172	-
0.2259	1140	0.6389	-
0.2279	1150	0.5044	-
0.2299	1160	0.9025	-
0.2319	1170	0.589	-
0.2338	1180	0.8662	-
0.2358	1190	0.7707	-
0.2378	1200	0.8708	-
0.2398	1210	0.437	-
0.2418	1220	0.6173	-
0.2438	1230	0.8496	-
0.2457	1240	0.7202	-
0.2477	1250	0.532	-
0.2497	1260	0.5777	-
0.2517	1270	0.7224	-
0.2537	1280	0.5855	-
0.2556	1290	0.8911	-
0.2576	1300	0.773	-
0.2596	1310	0.3215	-
0.2616	1320	0.4938	-
0.2636	1330	0.6604	-
0.2656	1340	0.6648	-
0.2675	1350	0.3865	-
0.2695	1360	0.6923	-
0.2715	1370	0.4671	-
0.2735	1380	0.5518	-
0.2755	1390	0.506	-
0.2774	1400	0.6273	-
0.2794	1410	0.5701	-
0.2814	1420	0.698	-
0.2834	1430	0.8443	-
0.2854	1440	0.5466	-
0.2874	1450	0.6025	-
0.2893	1460	0.6741	-
0.2913	1470	0.8025	-
0.2933	1480	0.6032	-
0.2953	1490	0.5131	-
0.2973	1500	0.6024	-
0.2992	1510	0.3469	-
0.3012	1520	0.6386	-
0.3032	1530	0.4665	-
0.3052	1540	0.6506	-
0.3072	1550	0.6024	-
0.3092	1560	0.4678	-
0.3111	1570	0.3724	-
0.3131	1580	0.4537	-
0.3151	1590	0.5408	-
0.3171	1600	0.4541	-
0.3191	1610	0.546	-
0.3210	1620	0.6713	-
0.3230	1630	0.5569	-
0.3250	1640	0.4741	-
0.3270	1650	0.5648	-
0.3290	1660	0.6074	-
0.3310	1670	0.5242	-
0.3329	1680	0.6622	-
0.3349	1690	0.79	-
0.3369	1700	0.2706	-
0.3389	1710	0.4091	-
0.3409	1720	0.4265	-
0.3428	1730	0.5644	-
0.3448	1740	0.4754	-
0.3468	1750	0.5008	-
0.3488	1760	0.4613	-
0.3508	1770	0.4062	-
0.3528	1780	0.6566	-
0.3547	1790	0.4417	-
0.3567	1800	0.5888	-
0.3587	1810	0.5818	-
0.3607	1820	0.5467	-
0.3627	1830	0.5819	-
0.3646	1840	0.5291	-
0.3666	1850	0.4268	-

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 5.1.2
• Transformers: 4.57.3
• PyTorch: 2.6.0+cu126
• Accelerate: 1.12.0
• Datasets: 4.4.1
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}