jangedoo/sanolm-v2

SentenceTransformer based on jangedoo/sanolm-base-v2

This is a sentence-transformers model finetuned from jangedoo/sanolm-base-v2 on the hard_negatives and replay datasets. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: jangedoo/sanolm-base-v2
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text
• Training Datasets:
  • hard_negatives
  • replay

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'ModernBertModel'})
  (1): Pooling({'embedding_dimension': 384, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    'डेङ्गी रोग विश्वका कति देशहरूमा व्याप्त छ?',
]
documents = [
    'डेङ्गी दोस्रो विश्वयुद्धको समयदेखि विश्वव्यापी समस्या बनेको हो। हाल यो रोग विश्वका ११० देशहरूमा व्याप्त छ। प्रत्येक वर्ष यो रोगका कारण करीब ५ करोडदेखि ५ करोड २८ लाख मानिसहरू बिरामी पर्ने गरेका छन्। जसमध्ये करीब दसहजारदेखि बीसहजार मानिसहरूको मृत्यु हुने गरेको छ। डेङ्गीको महामारीको बारेमा सन् १७७९ लिखित तथ्य भेटिएको छ। यो रोग भाइरसका कारणले लाग्ने र लामखुट्टेका कारणले मानिसमा सर्ने गरेको तथ्य बीसौँ सताब्दीमै पत्ता लागेको थियो। लामखुट्टेको निर्मुल पार्न असम्भव जस्तै भएकोले सिँधै भाइरसलाई नियन्त्रण गर्ने औषधिहरूको बारेमा पनि अध्ययन भइरहेको छ।',
    "Despite some changes, China's one-child family planning program remains a source of coercion, forced abortions, infanticide and perilously imbalanced boy-girl ratios, State Department officials said Tuesday.",
    'बलभद्र कुँवर नेपाली सेना (तत्कालिन गोरखाली सेना)को पुरानो गोरख गणको कप्तान थिए। उनले देहरादुनको बाहिर नालापानीको युद्धमा गोरखाली सेनाका नेतृत्व गर्दै लडेका थिए।',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 384] [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.8878, 0.7303, 0.6111]])

Evaluation

Metrics

Information Retrieval

• Datasets: hard_negative, indic_rag, balanced_news, en_ne_alignment, hard_negative, indic_rag, balanced_news and en_ne_alignment
• Evaluated with InformationRetrievalEvaluator with these parameters:

  {
      "query_prompt_name": "query",
      "corpus_prompt_name": "passage"
  }
  

Metric	hard_negative	indic_rag	balanced_news	en_ne_alignment
cosine_accuracy@1	0.369	0.655	0.6283	0.0106
cosine_accuracy@3	0.593	0.742	0.775	0.6567
cosine_accuracy@5	0.683	0.771	0.815	0.7981
cosine_accuracy@10	0.768	0.814	0.8517	0.8944
cosine_precision@1	0.369	0.655	0.6283	0.0106
cosine_precision@3	0.1977	0.2473	0.2583	0.2189
cosine_precision@5	0.1366	0.1542	0.163	0.1596
cosine_precision@10	0.0768	0.0814	0.0852	0.0894
cosine_recall@1	0.369	0.655	0.6283	0.0106
cosine_recall@3	0.593	0.742	0.775	0.6567
cosine_recall@5	0.683	0.771	0.815	0.7981
cosine_recall@10	0.768	0.814	0.8517	0.8944
cosine_ndcg@10	0.564	0.7324	0.744	0.489
cosine_mrr@10	0.4992	0.7065	0.7091	0.355
cosine_map@100	0.5055	0.7105	0.7138	0.3599

Training Details

Training Datasets

hard_negatives

• Dataset: hard_negatives
• Size: 8,000 training samples
• Columns: anchor, positive, negative_1, negative_2, and negative_3
• Approximate statistics based on the first 100 samples:

  	anchor	positive	negative_1	negative_2	negative_3
  type	string	string	string	string	string
  modality	text	text	text	text	text
  details	min: 8 tokens mean: 16.62 tokens max: 34 tokens	min: 20 tokens mean: 57.35 tokens max: 141 tokens	min: 8 tokens mean: 22.19 tokens max: 51 tokens	min: 6 tokens mean: 21.65 tokens max: 44 tokens	min: 10 tokens mean: 21.35 tokens max: 44 tokens

• Samples:

  anchor	positive	negative_1	negative_2	negative_3
  Baby lai night time ma wake garyo bhane ke fix garnu parcha?	यदि बच्चा राति उठेर रोइरहेको छ भने, सबैभन्दा पहिले उसलाई आरामदायी वातावरण दिनुहोस्। प्रकाश कम राख्नुहोस्, मधुर आवाजमा कुरा गर्नुहोस् र बिस्तारै उसलाई सुत्न लगाउनुहोस्। यदि समस्या जारी रहेमा बालरोग विशेषज्ञ (Pediatrician) सँग consult गर्नु राम्रो हुन्छ।	Bataune kura garna saknu hunchha, tara thik fix pani hunna sakcha.	Baby lai garma garma chautari ma rakhnu parcha, yeta le help garcha.	Sadae thik bhanda lai 10 minute samma wait garnuhos, pachi pani wake bhane pani bhetna pardaina.
  पासपोर्ट बनाउन कुन अफिसमा जानुपर्छ?	नेपाली नागरिकले नयाँ पासपोर्टको लागि सम्बन्धित जिल्ला प्रशासन कार्यालय (District Administration Office) मा निवेदन दिनुपर्ने हुन्छ।	पासपोर्टको लागि सबैभन्दा नजिकको बैंक शाखामा मात्र जाने आवश्यकता पर्दछ।	तपाईँले पासपोर्ट नवीकरणको लागि मात्र 'विदेश सेवा केन्द्र' मा जानु पर्छ।	पासपोर्ट बनाउनको लागि सरकारी सरकारी निकायको सट्टा निजी फोटो स्टुडियोमा भेट्नुपर्छ।
  Online certificate renewal ko regards ma ke samasya aaunchha?	Online portal ma certificate renewal ko lagi 'verification failed' error aaune bhaye, sabai bhanda pahile ta aapno mobile number ra email address check garnuhos. Yadi tyahi thik chha bhane, government helpline ma contact garera support ma sodhna parchha.	Certificate renewal ko lagi lagi naya application form bharnu paryo bhane tapai le online process ma aaphai nai try garnu hos.	Certificate ko renewal ko time frame 30 din ho, tesaile jaldikai garera pani samasya aune chhaina.	Yo process lai government office ma personal javera bharnu parne ho, online ma dinu chhaina.

• Loss: main.CachedGISTEmbedLoss with these parameters:

  {
      "guide": "SentenceTransformer('intfloat/multilingual-e5-small')",
      "temperature": 0.01,
      "mini_batch_size": 16,
      "margin_strategy": "relative",
      "margin": 0.05,
      "contrast_anchors": true,
      "contrast_positives": false,
      "gather_across_devices": false
  }
  

replay

• Dataset: replay
• Size: 8,000 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 100 samples:

  	anchor	positive
  type	string	string
  modality	text	text
  details	min: 5 tokens mean: 14.9 tokens max: 39 tokens	min: 4 tokens mean: 35.41 tokens max: 266 tokens

• Samples:

  anchor	positive
  निश्चयले जिते कार्ल्सबर्ग गल्फ	कार्ल्सबर्ग गल्फ सिरिज २०१९ मा मेजर निश्चय रायमाझी कुल २ सय ७५ अंक बटुल्दै नेट च्याम्पियन भएका छन् । विजेता भएवापत् उनले नोभेम्बर पहिलो साता मलेसियामा हुने गल्फ प्रतियोगितामा भाग लिने छन् । टासी छिरिङ २ सय १० अंक प्राप्त गर्दै ग्रस विजेता बनेका छन् ।
  A leader with a leader	नेतासँगै नेता
  रौतहट प्रहरीले कुटपिटको घटनामा जाहेरी लिन मानेन, मिलापत्र गर्न पीडित पक्षलाई दबाब	Rautahat police refused to take a complaint in the case of assault, pressure on the victim to settle

• Loss: main.CachedGISTEmbedLoss with these parameters:

  {
      "guide": "SentenceTransformer('intfloat/multilingual-e5-small')",
      "temperature": 0.01,
      "mini_batch_size": 16,
      "margin_strategy": "relative",
      "margin": 0.05,
      "contrast_anchors": true,
      "contrast_positives": false,
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 128
• num_train_epochs: 2.0
• learning_rate: 5e-06
• warmup_steps: 0.1
• bf16: True
• torch_compile: True
• torch_compile_backend: inductor
• per_device_eval_batch_size: 16
• load_best_model_at_end: True
• data_seed: 42
• dataloader_num_workers: 8
• dataloader_persistent_workers: True
• dataloader_prefetch_factor: 4
• prompts: {'anchor': 'query: ', 'positive': 'passage: ', 'negative_1': 'passage: ', 'negative_2': 'passage: ', 'negative_3': 'passage: '}
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• per_device_train_batch_size: 128
• num_train_epochs: 2.0
• max_steps: -1
• learning_rate: 5e-06
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: True
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: True
• torch_compile_backend: inductor
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 16
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: True
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: 42
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 8
• dataloader_pin_memory: True
• dataloader_persistent_workers: True
• dataloader_prefetch_factor: 4
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: {'anchor': 'query: ', 'positive': 'passage: ', 'negative_1': 'passage: ', 'negative_2': 'passage: ', 'negative_3': 'passage: '}
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	hard_negative_cosine_ndcg@10	indic_rag_cosine_ndcg@10	balanced_news_cosine_ndcg@10	en_ne_alignment_cosine_ndcg@10
0.0502	31	3.2558	-	-	-	-
0.1003	62	1.5447	0.3483	0.6433	0.6051	0.3638
0.1505	93	0.8522	-	-	-	-
0.2006	124	0.6679	0.3726	0.6900	0.6713	0.4115
0.2508	155	0.6209	-	-	-	-
0.3010	186	0.5758	0.3844	0.7045	0.6988	0.4335
0.3511	217	0.5351	-	-	-	-
0.4013	248	0.5234	0.3887	0.7111	0.7144	0.4434
0.4515	279	0.4474	-	-	-	-
0.5016	310	0.4444	0.3907	0.7193	0.7287	0.4507
0.5518	341	0.4595	-	-	-	-
0.6019	372	0.4143	0.3923	0.7261	0.7371	0.4561
0.6521	403	0.3946	-	-	-	-
0.7023	434	0.4026	0.3909	0.7302	0.7440	0.4620
0.7524	465	0.3900	-	-	-	-
0.8026	496	0.3825	0.3916	0.7314	0.7473	0.4637
0.8528	527	0.3980	-	-	-	-
0.9029	558	0.3711	0.3928	0.7326	0.7508	0.4661
0.9531	589	0.3678	-	-	-	-
1.0	618	-	0.3930	0.7329	0.7525	0.4670
0.1032	13	0.7110	-	-	-	-
0.2063	26	0.6710	0.4261	0.7335	0.7489	0.4706
0.3095	39	0.6519	-	-	-	-
0.4127	52	0.5455	0.4754	0.7301	0.7487	0.4780
0.5159	65	0.4912	-	-	-	-
0.6190	78	0.5774	0.5007	0.7320	0.7448	0.4830
0.7222	91	0.5217	-	-	-	-
0.8254	104	0.5394	0.5272	0.7314	0.7432	0.4838
0.9286	117	0.4898	-	-	-	-
1.0317	130	0.4209	0.5419	0.7305	0.7420	0.4845
1.1349	143	0.4120	-	-	-	-
1.2381	156	0.4448	0.5556	0.7306	0.7412	0.4849
1.3413	169	0.3246	-	-	-	-
1.4444	182	0.3570	0.5603	0.7313	0.7420	0.4865
1.5476	195	0.4110	-	-	-	-
1.6508	208	0.2963	0.5626	0.7321	0.7451	0.4881
1.7540	221	0.3628	-	-	-	-
1.8571	234	0.3137	0.5639	0.7320	0.7440	0.4885
1.9603	247	0.3308	-	-	-	-
2.0	252	-	0.564	0.7324	0.744	0.489

• The bold row denotes the saved checkpoint.

Training Time

• Training: 3.7 minutes
• Evaluation: 1.3 minutes
• Total: 5.0 minutes

Framework Versions

• Python: 3.12.11
• Sentence Transformers: 5.6.0
• Transformers: 5.13.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.13.0
• Datasets: 4.3.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}