BjarneNPO/BjarneNPO-27_08_2025_10_58_39

SentenceTransformer based on Snowflake/snowflake-arctic-embed-m-v2.0

This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-m-v2.0. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Snowflake/snowflake-arctic-embed-m-v2.0
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'GteModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("BjarneNPO-27_08_2025_10_58_39")
# Run inference
queries = [
    "Userin hinterlegt Email-Adresse im Benutzerkonto und speichert. Aber die Adresse wird trotz Best\u00e4tigung nicht gespeichert. \r\nEMA ist notwendig f\u00fcr 2FA\r\n\r\n Roesler =  jil.roesler@cse.ruhr",
]
documents = [
    'N',
    'Unter dem Namen der Dame gibt es nur einen Login. Vielleicht schaut sie mit dem Login einer anderen Kollegin auf die zweite Einrichtung? Oder sie hat einen Login als Träger? Dies klärt sie mit der Einrichtung ab.',
    'Userin hat die Rolle "Mitarbeiter".',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.3811, 0.1504, 0.1198]])

Evaluation

Metrics

Information Retrieval

• Dataset: Snowflake/snowflake-arctic-embed-m-v2.0
• Evaluated with scripts.InformationRetrievalEvaluatorCustom.InformationRetrievalEvaluatorCustom with these parameters:

  {
      "query_prompt_name": "query",
      "corpus_prompt_name": "document"
  }
  

Metric	Value
cosine_accuracy@1	0.3377
cosine_accuracy@3	0.5584
cosine_accuracy@5	0.6364
cosine_accuracy@10	0.7143
cosine_precision@1	0.3377
cosine_precision@3	0.2944
cosine_precision@5	0.2597
cosine_precision@10	0.1779
cosine_recall@1	0.0377
cosine_recall@3	0.0982
cosine_recall@5	0.1329
cosine_recall@10	0.1775
cosine_ndcg@10	0.238
cosine_mrr@10	0.4674
cosine_map@100	0.1695

Training Details

Training Dataset

Unnamed Dataset

• Size: 86,218 training samples
• Columns: query and answer
• Approximate statistics based on the first 1000 samples:

  	query	answer
  type	string	string
  details	min: 5 tokens mean: 80.67 tokens max: 5231 tokens	min: 3 tokens mean: 25.16 tokens max: 238 tokens

• Samples:

  query	answer
  Nun ist die Monatsmeldung erfolgt, aber rote Ausrufezeichen tauchen auf.	Userin an das JA verwiesen, diese müssten ihr die Schloss-Monate zur Überarbeitung im Kibiz.web zurückgeben. Userin dazu empfohlen, die Kinder die nicht in kitaplus sind, aber in Kibiz.web - im KiBiz.web zu entfernen, wenn diese nicht vorhanden sind.
  Die Feiertage in den Stammdaten stimmen nicht.	Es besteht bereits ein Ticket dafür.
  Abrechnung kann nicht final freigegeben werden, es wird aber keiner Fehlermeldung angeziegt	im Hintergrund ist eine Fehlermeldung zu sehen. An Entwickler weitergeleitet.
  Korrektur vorgenommen.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• gradient_accumulation_steps: 4
• learning_rate: 4e-05
• weight_decay: 0.01
• warmup_ratio: 0.08
• bf16: True
• tf32: True
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 4
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 4e-05
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.08
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: True
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Snowflake/snowflake-arctic-embed-m-v2.0_cosine_ndcg@10
0.0297	10	2.8914	-
0.0593	20	2.8359	-
0.0890	30	2.4573	-
0.1187	40	2.3298	-
0.1484	50	2.215	-
0.1780	60	2.0003	-
0.2077	70	1.8714	-
0.2374	80	1.7492	-
0.2671	90	1.6268	-
0.2967	100	1.6434	-
0.3264	110	1.5872	-
0.3561	120	1.5221	-
0.3858	130	1.4166	-
0.4154	140	1.4093	-
0.4451	150	1.4323	-
0.4748	160	1.3748	-
0.5045	170	1.3443	-
0.5341	180	1.3358	-
0.5638	190	1.3118	-
0.5935	200	1.2791	-
0.6231	210	1.2576	-
0.6528	220	1.2493	-
0.6825	230	1.2586	-
0.7122	240	1.2468	-
0.7418	250	1.2017	-
0.7715	260	1.177	-
0.8012	270	1.1899	-
0.8309	280	1.161	-
0.8605	290	1.1743	-
0.8902	300	1.1568	-
0.9199	310	1.1422	-
0.9496	320	0.0	-
0.9792	330	0.0	-
1.0	337	-	0.2375
1.0089	340	0.331	-
1.0386	350	0.9826	-
1.0682	360	0.9872	-
1.0979	370	0.9697	-
1.1276	380	0.9763	-
1.1573	390	1.0233	-
1.1869	400	0.9827	-
1.2166	410	0.9754	-
1.2463	420	0.986	-
1.2760	430	0.9342	-
1.3056	440	0.9685	-
1.3353	450	0.9699	-
1.3650	460	0.906	-
1.3947	470	0.9959	-
1.4243	480	0.9386	-
1.4540	490	0.9565	-
1.4837	500	0.9308	-
1.5134	510	0.9325	-
1.5430	520	0.9232	-
1.5727	530	0.9413	-
1.6024	540	0.9183	-
1.6320	550	0.9651	-
1.6617	560	0.9034	-
1.6914	570	0.8517	-
1.7211	580	0.923	-
1.7507	590	0.8351	-
1.7804	600	0.858	-
1.8101	610	0.8404	-
1.8398	620	0.9191	-
1.8694	630	0.8746	-
1.8991	640	0.8732	-
1.9288	650	0.5662	-
1.9585	660	0.0	-
1.9881	670	0.0	-
2.0	674	-	0.2252
2.0178	680	0.4717	-
2.0475	690	0.7903	-
2.0772	700	0.7363	-
2.1068	710	0.7626	-
2.1365	720	0.7836	-
2.1662	730	0.7634	-
2.1958	740	0.7843	-
2.2255	750	0.8229	-
2.2552	760	0.7876	-
2.2849	770	0.7467	-
2.3145	780	0.7461	-
2.3442	790	0.7687	-
2.3739	800	0.7353	-
2.4036	810	0.7721	-
2.4332	820	0.7392	-
2.4629	830	0.7698	-
2.4926	840	0.7876	-
2.5223	850	0.7493	-
2.5519	860	0.7775	-
2.5816	870	0.717	-
2.6113	880	0.6827	-
2.6409	890	0.7727	-
2.6706	900	0.7433	-
2.7003	910	0.725	-
2.7300	920	0.7344	-
2.7596	930	0.7822	-
2.7893	940	0.7131	-
2.8190	950	0.7894	-
2.8487	960	0.7286	-
2.8783	970	0.7635	-
2.9080	980	0.7814	-
2.9377	990	0.2642	-
2.9674	1000	0.0	-
2.9970	1010	0.0	-
3.0	1011	-	0.2380

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.11
• Sentence Transformers: 5.1.0
• Transformers: 4.55.2
• PyTorch: 2.8.0+cu129
• Accelerate: 1.10.0
• Datasets: 3.6.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}