TdelaSelle/PatriRetriever-lat

SentenceTransformer based on bowphs/LaBerta

This is a sentence-transformers model finetuned from bowphs/LaBerta. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: bowphs/LaBerta
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'RobertaModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("TdelaSelle/PatriLaRS-dev")
# Run inference
queries = [
    'aut si adhuc parum uidetur esse monstratum, quod sententiam istam propter inmutationem, quae futura est, apostolus dixerit, cum ait: caro et sanguis regnum dei hereditate possidere non possunt neque corruptio incorruptionem hereditate possidebit, adtendite, quid continuo subiciat et adiungat: ecce mysterium dico uobis:',
]
documents = [
    'Ecce mysterium vobis dico: omnes quidem resurgemus, sed non omnes inmutabimur.',
    'Hoc autem dico, fratres, quoniam caro et sanguis regnum Dei possidere non possunt, neque corruptio incorruptelam possidebit.',
    'Hoc autem dico, fratres, quoniam caro et sanguis regnum Dei possidere non possunt, neque corruptio incorruptelam possidebit.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.4887, 0.6675, 0.6675]])

Evaluation

Metrics

Information Retrieval

• Dataset: laberta-mpmnrl-closed
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.7123
cosine_accuracy@3	0.8107
cosine_accuracy@5	0.8405
cosine_accuracy@10	0.8773
cosine_precision@1	0.7123
cosine_precision@3	0.3183
cosine_precision@5	0.2041
cosine_precision@10	0.1091
cosine_recall@1	0.5821
cosine_recall@3	0.7178
cosine_recall@5	0.7567
cosine_recall@10	0.8001
cosine_ndcg@10	0.7381
cosine_mrr@10	0.7688
cosine_map@100	0.699

Training Details

Training Dataset

Unnamed Dataset

• Size: 89,904 training samples
• Columns: anchor, positive_1, positive_2, positive_3, negative_1, negative_2, negative_3, negative_4, negative_5, negative_6, negative_7, negative_8, negative_9, and negative_10
• Approximate statistics based on the first 100 samples:

  	anchor	positive_1	positive_2	positive_3	negative_1	negative_2	negative_3	negative_4	negative_5	negative_6	negative_7	negative_8	negative_9	negative_10
  type	string	string	string	string	string	string	string	string	string	string	string	string	string	string
  modality	text	text	text	text	text	text	text	text	text	text	text	text	text	text
  details	min: 18 tokens mean: 39.19 tokens max: 69 tokens	min: 10 tokens mean: 23.11 tokens max: 47 tokens	min: 10 tokens mean: 22.83 tokens max: 47 tokens	min: 10 tokens mean: 22.9 tokens max: 47 tokens	min: 10 tokens mean: 22.36 tokens max: 58 tokens	min: 7 tokens mean: 22.62 tokens max: 52 tokens	min: 10 tokens mean: 22.73 tokens max: 46 tokens	min: 10 tokens mean: 23.75 tokens max: 42 tokens	min: 10 tokens mean: 22.52 tokens max: 43 tokens	min: 9 tokens mean: 22.07 tokens max: 47 tokens	min: 11 tokens mean: 24.19 tokens max: 50 tokens	min: 9 tokens mean: 22.72 tokens max: 45 tokens	min: 10 tokens mean: 21.94 tokens max: 46 tokens	min: 6 tokens mean: 22.36 tokens max: 40 tokens

• Samples:

  anchor	positive_1	positive_2	positive_3	negative_1	negative_2	negative_3	negative_4	negative_5	negative_6	negative_7	negative_8	negative_9	negative_10
  nam mihi credite, uel potius illi credite, qui ait: quaerite et inuenietis, nec cognitionem desperandam esse et manifestiorem futuram, quam sunt illi numeri.	Petite et dabitur vobis, quaerite et invenietis, pulsate et aperietur vobis.	Petite et dabitur vobis, quaerite et invenietis, pulsate et aperietur vobis.	Petite et dabitur vobis, quaerite et invenietis, pulsate et aperietur vobis.	Verumtamen quaerite regnum Dei et haec omnia adicientur vobis.	Quaerite autem primum regnum et iustitiam eius, et omnia haec adicientur vobis.	Quaerite Dominum dum inveniri potest, invocate eum dum prope est.	Quaerite Dominum et vivite, ne forte conburatur ut ignis domus Ioseph et devorabit et non erit qui extinguat Bethel.	Et ego vobis dico: petite et dabitur vobis, quaerite et invenietis, pulsate et aperietur vobis.	Et invocabitis me, et ibitis, et orabitis me, et exaudiam vos.	Quaeretis me et invenietis, cum quaesieritis me in toto corde vestro.	Omnis enim qui petit accipit, et qui quaerit invenit, et pulsanti aperietur.	Quia haec dicit Dominus domui Israhel: quaerite me et vivetis.	Quaerite Dominum et confirmamini, quaerite faciem eius semper.
  de eo quod scriptum est: in principio fecit deus caelum et terram usque ad id quod scriptum est: et factum est uespere et factum est mane dies una.	Appelavitque lucem diem et tenebras noctem. Factumque est vespere et mane dies unus.	In principio creavit Deus caelum et terram.	Appelavitque lucem diem et tenebras noctem. Factumque est vespere et mane dies unus.	Et fecit Deus firmamentum, divisitque aquas quae erant sub firmamento ab his quae erant super firmamentum. Et factum est ita.	Viditque Deus cuncta quae fecit, et erant valde bona. Et factum est vespere et mane dies sextus.	Vocavitque Deus firmamentum caelum. Et factum est vespere et mane dies secundus.	Et erit dies una quae nota est Domino, non dies neque nox, et in tempore vesperae erit lux.	Istae generationes caeli et terrae quando creatae sunt, in die quo fecit Dominus Deus caelum et terram,	Et benedixit diei septimo et sanctificavit illum, quia in ipso cessaverat ab omni opere suo quod creavit Deus ut faceret.	et praeessent diei ac nocti et dividerent lucem ac tenebras. Et vidit Deus quod esset bonum.	Et vidit Deus lucem quod esset bona, et divisit lucem ac tenebras.	Dixitque Deus: fiat lux. Et facta est lux.	Terra autem erat inanis et vacua, et tenebrae super faciem abyssi, et spiritus Dei ferebatur super aquas.
  hoc capitulum legis aduersum esse euangelio stultissimi manichaei arbitrantur dicentes in genesi scriptum esse, quod deus per se ipsum fecerit caelum et terram et lucem, in euangelio autem scriptum esse per dominum nostrum iesum christum fabricatum esse mundum, ubi dictum est: et mundus factus est per ipsum, et mundus illum non cognouit.	In mundo erat et mundus per ipsum factus est et mundus eum non cognovit.	In mundo erat et mundus per ipsum factus est et mundus eum non cognovit.	In mundo erat et mundus per ipsum factus est et mundus eum non cognovit.	et ipse est ante omnes, et omnia in ipso constant,	Non potest mundus odisse vos, me autem odit, quia ego testimonium perhibeo de illo quia opera eius mala sunt.	Non enim misit Deus Filium suum in mundum ut iudicet mundum, sed ut salvetur mundus per ipsum.	In principio creavit Deus caelum et terram.	Si mundus vos odit, scitote quia me priorem vobis odio habuit.	Pater iuste, et mundus te non cognovit, ego autem te cognovi et hii cognoverunt quia tu me misisti.	Invisibilia enim ipsius a creatura mundi per ea quae facta sunt intellecta conspiciuntur, sempiterna quoque eius virtus et divinitas, ut sint inexcusabiles.	Omnia per ipsum facta sunt et sine ipso factum est nihil quod factum est.	Nam quia in Dei sapientia non cognovit mundus per sapientiam Deum, placuit Deo per stultitiam praedicationis salvos facere credentes.	In principio erat Verbum et Verbum erat apud Deum et Deus erat Verbum.

• Loss: mpmnrl_loss.MultiplePositiveMultipleNegativeRankingLoss with these parameters:

  {
      "scale": 20.0,
      "n_positives": 3,
      "mini_batch_size": 128,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 256
• learning_rate: 2e-05
• warmup_steps: 0.1
• bf16: True
• per_device_eval_batch_size: 64
• dataloader_drop_last: True

All Hyperparameters

Click to expand

• per_device_train_batch_size: 256
• num_train_epochs: 3.0
• max_steps: -1
• learning_rate: 2e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: True
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 64
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: True
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• dataloader_multiprocessing_context: None
• dataloader_in_order: True
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}
• warmup_ratio: None

Training Logs

Epoch	Step	Training Loss	laberta-mpmnrl-closed_cosine_ndcg@10
-1	-1	-	0.3202
0.0570	20	3.6829	-
0.1140	40	2.5612	-
0.1709	60	2.1213	-
0.2279	80	1.9791	-
0.2849	100	1.8775	-
0.3419	120	1.7494	-
0.3989	140	1.7140	-
0.4558	160	1.6059	-
0.5128	180	1.6469	-
0.5698	200	1.6590	-
0.6268	220	1.5901	-
0.6838	240	1.5561	-
0.7407	260	1.5477	-
0.7977	280	1.5401	-
0.8547	300	1.4709	-
0.9117	320	1.5277	-
0.9687	340	1.4457	-
1.0256	360	1.3532	-
1.0826	380	1.2586	-
1.1396	400	1.2786	-
1.1966	420	1.2156	-
1.2536	440	1.2563	-
1.3105	460	1.2898	-
1.3675	480	1.1908	-
1.4245	500	1.2010	-
1.4815	520	1.2143	-
1.5385	540	1.1404	-
1.5954	560	1.2330	-
1.6524	580	1.2099	-
1.7094	600	1.2006	-
1.7664	620	1.1738	-
1.8234	640	1.1662	-
1.8803	660	1.1719	-
1.9373	680	1.2119	-
1.9943	700	1.1572	-
2.0513	720	1.0015	-
2.1083	740	1.0585	-
2.1652	760	1.0623	-
2.2222	780	1.0220	-
2.2792	800	1.0203	-
2.3362	820	1.0171	-
2.3932	840	1.0690	-
2.4501	860	0.9702	-
2.5071	880	0.9934	-
2.5641	900	1.0266	-
2.6211	920	0.9917	-
2.6781	940	0.9868	-
2.7350	960	1.0152	-
2.7920	980	1.0991	-
2.8490	1000	1.0358	-
2.9060	1020	1.0061	-
2.9630	1040	1.0297	-
-1	-1	-	0.7381

Training Time

• Training: 20.9 minutes

Framework Versions

• Python: 3.10.20
• Sentence Transformers: 5.7.0
• Transformers: 5.17.0
• PyTorch: 2.11.0+cu128
• Accelerate: 1.14.0
• Datasets: 4.6.0
• Tokenizers: 0.23.2

Additional Resources

• Training and Finetuning Embedding Models with Sentence Transformers: the end-to-end guide for training or finetuning Sentence Transformer models.
• Introduction to Matryoshka Embedding Models: variable-size embeddings that can be truncated with minimal quality loss.
• Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval: post-training compression of embedding vectors.
• Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video models through the same API.
• Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: train multimodal embedding models, with a Visual Document Retrieval walkthrough.

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

Paramètres BibliReuse

Calibration e6d2353488e1 du 2026-10-13 (37 œuvres, 1774 occurrences annotées). Ces valeurs sont lues par l'outil BibliReuse dans l'en-tête de cette carte.

paramètre	valeur
seuil de détection (règle OU)	0.48905356610240647