BjarneNPO/BjarneNPO-28_08_2025_09_52_32

SentenceTransformer based on Snowflake/snowflake-arctic-embed-m-v2.0

This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-m-v2.0. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Snowflake/snowflake-arctic-embed-m-v2.0
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'GteModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("BjarneNPO-28_08_2025_09_52_32")
# Run inference
queries = [
    "Ein Vater taucht nicht auf bei den Eltern im Elternbeirat \r\n\r\nAu\u00dferdem auf die Kinder mit archivierten Angeh\u00f6rigen hingewiesen und ihr gezeigt",
]
documents = [
    'Weil er keinen Zugang zur EAPP hat, Außerdem auf die Kinder mit archivierten Angehörigen hingewiesen und ihr gezeigt wie sie das lösen kann',
    '1. Vorlage da. Userin auch gezeigt wie sie die verwanden kann\r\n2. Als Wunsch weitergegeben.',
    'In der Kinderliste haben Kinder gefehlt. Userin muss die Daten in der Kinderliste hinterlegen.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.5587, 0.0609, 0.0386]])

Evaluation

Metrics

Information Retrieval

• Dataset: Snowflake/snowflake-arctic-embed-m-v2.0
• Evaluated with scripts.InformationRetrievalEvaluatorCustom.InformationRetrievalEvaluatorCustom with these parameters:

  {
      "query_prompt_name": "query",
      "corpus_prompt_name": "document"
  }
  

Metric	Value
cosine_accuracy@1	0.3333
cosine_accuracy@3	0.5652
cosine_accuracy@5	0.6522
cosine_accuracy@10	0.7246
cosine_precision@1	0.3333
cosine_precision@3	0.3043
cosine_precision@5	0.2754
cosine_precision@10	0.1797
cosine_recall@1	0.0285
cosine_recall@3	0.0825
cosine_recall@5	0.1107
cosine_recall@10	0.1414
cosine_ndcg@10	0.2387
cosine_mrr@10	0.4743
cosine_map@100	0.139

Training Details

Training Dataset

Unnamed Dataset

• Size: 72,349 training samples
• Columns: query and answer
• Approximate statistics based on the first 1000 samples:

  	query	answer
  type	string	string
  details	min: 6 tokens mean: 49.26 tokens max: 4339 tokens	min: 6 tokens mean: 28.66 tokens max: 238 tokens

• Samples:

  query	answer
  Nun ist die Monatsmeldung erfolgt, aber rote Ausrufezeichen tauchen auf.	Userin an das JA verwiesen, diese müssten ihr die Schloss-Monate zur Überarbeitung im Kibiz.web zurückgeben. Userin dazu empfohlen, die Kinder die nicht in kitaplus sind, aber in Kibiz.web - im KiBiz.web zu entfernen, wenn diese nicht vorhanden sind.
  Die Feiertage in den Stammdaten stimmen nicht.	Es besteht bereits ein Ticket dafür.
  Abrechnung kann nicht final freigegeben werden, es wird aber keiner Fehlermeldung angeziegt	im Hintergrund ist eine Fehlermeldung zu sehen. An Entwickler weitergeleitet.
  Korrektur vorgenommen.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• gradient_accumulation_steps: 4
• learning_rate: 4e-05
• weight_decay: 0.01
• warmup_ratio: 0.08
• bf16: True
• tf32: True
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 4
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 4e-05
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.08
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: True
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Snowflake/snowflake-arctic-embed-m-v2.0_cosine_ndcg@10
0.0354	10	2.7715	-
0.0707	20	2.5556	-
0.1061	30	2.3266	-
0.1415	40	2.2795	-
0.1768	50	1.9574	-
0.2122	60	1.8775	-
0.2476	70	1.6964	-
0.2829	80	1.6731	-
0.3183	90	1.5234	-
0.3537	100	1.5026	-
0.3890	110	1.4455	-
0.4244	120	1.3997	-
0.4598	130	1.2861	-
0.4951	140	1.3341	-
0.5305	150	1.2485	-
0.5659	160	1.2433	-
0.6012	170	1.2227	-
0.6366	180	1.1542	-
0.6720	190	1.1898	-
0.7073	200	1.1421	-
0.7427	210	1.1524	-
0.7781	220	1.1649	-
0.8134	230	1.1481	-
0.8488	240	1.1145	-
0.8842	250	1.1199	-
0.9195	260	1.0708	-
0.9549	270	1.0884	-
0.9903	280	1.0438	-
1.0	283	-	0.2342
1.0248	290	0.9415	-
1.0601	300	0.9413	-
1.0955	310	0.9291	-
1.1309	320	0.92	-
1.1662	330	0.8903	-
1.2016	340	0.8868	-
1.2370	350	0.8619	-
1.2723	360	0.899	-
1.3077	370	0.8975	-
1.3431	380	0.9212	-
1.3784	390	0.8578	-
1.4138	400	0.8616	-
1.4492	410	0.857	-
1.4845	420	0.8721	-
1.5199	430	0.8747	-
1.5553	440	0.8554	-
1.5906	450	0.7959	-
1.6260	460	0.9174	-
1.6614	470	0.8515	-
1.6967	480	0.8732	-
1.7321	490	0.8501	-
1.7675	500	0.8426	-
1.8028	510	0.8304	-
1.8382	520	0.8354	-
1.8736	530	0.8078	-
1.9089	540	0.7801	-
1.9443	550	0.7839	-
1.9797	560	0.8351	-
2.0	566	-	0.2357
2.0141	570	0.7665	-
2.0495	580	0.7086	-
2.0849	590	0.7283	-
2.1202	600	0.7306	-
2.1556	610	0.7029	-
2.1910	620	0.6897	-
2.2263	630	0.6793	-
2.2617	640	0.7545	-
2.2971	650	0.6831	-
2.3324	660	0.7055	-
2.3678	670	0.7356	-
2.4032	680	0.6705	-
2.4385	690	0.7016	-
2.4739	700	0.6664	-
2.5093	710	0.7096	-
2.5447	720	0.7248	-
2.5800	730	0.7214	-
2.6154	740	0.6597	-
2.6508	750	0.6873	-
2.6861	760	0.6788	-
2.7215	770	0.7003	-
2.7569	780	0.6794	-
2.7922	790	0.657	-
2.8276	800	0.7099	-
2.8630	810	0.6948	-
2.8983	820	0.7311	-
2.9337	830	0.7373	-
2.9691	840	0.7017	-
3.0	849	-	0.2387

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.11
• Sentence Transformers: 5.1.0
• Transformers: 4.55.2
• PyTorch: 2.8.0+cu129
• Accelerate: 1.10.0
• Datasets: 3.6.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}