| | --- |
| | tags: |
| | - sentence-transformers |
| | - sentence-similarity |
| | - feature-extraction |
| | - dense |
| | - generated_from_trainer |
| | - dataset_size:70323 |
| | - loss:CosineSimilarityLoss |
| | base_model: intfloat/e5-base-v2 |
| | widget: |
| | - source_sentence: 'Birth of Cainan | participants: cainan_534, enos_1193' |
| | sentences: |
| | - The mother of Sisera looked out at a window, and cried through the lattice, Why |
| | is his chariot so long in coming? why tarry the wheels of his chariots? |
| | - 'Therefore, behold, the days come, that I will do judgment upon the graven images |
| | of Babylon: and her whole land shall be confounded, and all her slain shall fall |
| | in the midst of her.' |
| | - Which was the son of Mathusala, which was the son of Enoch, which was the son |
| | of Jared, which was the son of Maleleel, which was the son of Cainan, |
| | - source_sentence: 'Jerusalem Council | participants: silas_2740, judas_1759, james_719, |
| | peter_2745, barnabas_1722, paul_2479' |
| | sentences: |
| | - What ailed thee, O thou sea, that thou fleddest? thou Jordan, that thou wast driven |
| | back? |
| | - We have sent therefore Judas and Silas, who shall also tell you the same things |
| | by mouth. |
| | - 'The Spirit itself beareth witness with our spirit, that we are the children of |
| | God:' |
| | - source_sentence: But he that is married careth for the things that are of the world, |
| | how he may please his wife. |
| | sentences: |
| | - But she had brought them up to the roof of the house, and hid them with the stalks |
| | of flax, which she had laid in order upon the roof. |
| | - And their whole body, and their backs, and their hands, and their wings, and the |
| | wheels, were full of eyes round about, even the wheels that they four had. |
| | - 'There is difference also between a wife and a virgin. The unmarried woman careth |
| | for the things of the Lord, that she may be holy both in body and in spirit: but |
| | she that is married careth for the things of the world, how she may please her |
| | husband.' |
| | - source_sentence: And the little owl, and the cormorant, and the great owl, |
| | sentences: |
| | - And the swan, and the pelican, and the gier eagle, |
| | - Take Aaron and his sons with him, and the garments, and the anointing oil, and |
| | a bullock for the sin offering, and two rams, and a basket of unleavened bread; |
| | - 'And his power shall be mighty, but not by his own power: and he shall destroy |
| | wonderfully, and shall prosper, and practise, and shall destroy the mighty and |
| | the holy people.' |
| | - source_sentence: John's Witness |
| | sentences: |
| | - And they asked him, and said unto him, Why baptizest thou then, if thou be not |
| | that Christ, nor Elias, neither that prophet? |
| | - Then I took Jaazaniah the son of Jeremiah, the son of Habaziniah, and his brethren, |
| | and all his sons, and the whole house of the Rechabites; |
| | - 'But he turned, and said unto Peter, Get thee behind me, Satan: thou art an offence |
| | unto me: for thou savourest not the things that be of God, but those that be of |
| | men.' |
| | pipeline_tag: sentence-similarity |
| | library_name: sentence-transformers |
| | --- |
| | |
| | # SentenceTransformer based on intfloat/e5-base-v2 |
| |
|
| | This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [intfloat/e5-base-v2](https://huggingface.co/intfloat/e5-base-v2). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. |
| |
|
| | ## Model Details |
| |
|
| | ### Model Description |
| | - **Model Type:** Sentence Transformer |
| | - **Base model:** [intfloat/e5-base-v2](https://huggingface.co/intfloat/e5-base-v2) <!-- at revision f52bf8ec8c7124536f0efb74aca902b2995e5bcd --> |
| | - **Maximum Sequence Length:** 128 tokens |
| | - **Output Dimensionality:** 768 dimensions |
| | - **Similarity Function:** Cosine Similarity |
| | <!-- - **Training Dataset:** Unknown --> |
| | <!-- - **Language:** Unknown --> |
| | <!-- - **License:** Unknown --> |
| |
|
| | ### Model Sources |
| |
|
| | - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) |
| | - **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers) |
| | - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) |
| |
|
| | ### Full Model Architecture |
| |
|
| | ``` |
| | SentenceTransformer( |
| | (0): Transformer({'max_seq_length': 128, 'do_lower_case': False, 'architecture': 'BertModel'}) |
| | (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) |
| | (2): Normalize() |
| | ) |
| | ``` |
| |
|
| | ## Usage |
| |
|
| | ### Direct Usage (Sentence Transformers) |
| |
|
| | First install the Sentence Transformers library: |
| |
|
| | ```bash |
| | pip install -U sentence-transformers |
| | ``` |
| |
|
| | Then you can load this model and run inference. |
| | ```python |
| | from sentence_transformers import SentenceTransformer |
| | |
| | # Download from the 🤗 Hub |
| | model = SentenceTransformer("sentence_transformers_model_id") |
| | # Run inference |
| | sentences = [ |
| | "John's Witness", |
| | 'And they asked him, and said unto him, Why baptizest thou then, if thou be not that Christ, nor Elias, neither that prophet?', |
| | 'Then I took Jaazaniah the son of Jeremiah, the son of Habaziniah, and his brethren, and all his sons, and the whole house of the Rechabites;', |
| | ] |
| | embeddings = model.encode(sentences) |
| | print(embeddings.shape) |
| | # [3, 768] |
| | |
| | # Get the similarity scores for the embeddings |
| | similarities = model.similarity(embeddings, embeddings) |
| | print(similarities) |
| | # tensor([[1.0000, 0.7469, 0.7488], |
| | # [0.7469, 1.0000, 0.8236], |
| | # [0.7488, 0.8236, 1.0000]]) |
| | ``` |
| |
|
| | <!-- |
| | ### Direct Usage (Transformers) |
| |
|
| | <details><summary>Click to see the direct usage in Transformers</summary> |
| |
|
| | </details> |
| | --> |
| |
|
| | <!-- |
| | ### Downstream Usage (Sentence Transformers) |
| |
|
| | You can finetune this model on your own dataset. |
| |
|
| | <details><summary>Click to expand</summary> |
| |
|
| | </details> |
| | --> |
| |
|
| | <!-- |
| | ### Out-of-Scope Use |
| |
|
| | *List how the model may foreseeably be misused and address what users ought not to do with the model.* |
| | --> |
| |
|
| | <!-- |
| | ## Bias, Risks and Limitations |
| |
|
| | *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* |
| | --> |
| |
|
| | <!-- |
| | ### Recommendations |
| |
|
| | *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* |
| | --> |
| |
|
| | ## Training Details |
| |
|
| | ### Training Dataset |
| |
|
| | #### Unnamed Dataset |
| |
|
| | * Size: 70,323 training samples |
| | * Columns: <code>sentence_0</code>, <code>sentence_1</code>, and <code>label</code> |
| | * Approximate statistics based on the first 1000 samples: |
| | | | sentence_0 | sentence_1 | label | |
| | |:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------| |
| | | type | string | string | float | |
| | | details | <ul><li>min: 3 tokens</li><li>mean: 53.54 tokens</li><li>max: 128 tokens</li></ul> | <ul><li>min: 5 tokens</li><li>mean: 35.99 tokens</li><li>max: 85 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.99</li><li>max: 1.0</li></ul> | |
| | * Samples: |
| | | sentence_0 | sentence_1 | label | |
| | |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------| |
| | | <code>Prophecies of Jeremiah \| participants: jeremiah_853</code> | <code>In his days Judah shall be saved, and Israel shall dwell safely: and this is his name whereby he shall be called, The Lord Our Righteousness.</code> | <code>1.0</code> | |
| | | <code>God: (A.S. and Dutch God; Dan. Gud; Ger. Gott), the name of the Divine Being. It is the rendering (1) of the Hebrew <i> 'El</i> , from a word meaning to be strong; (2) of <i> 'Eloah_, plural _'Elohim</i> . The singular form, <i> Eloah</i> , is used only in poetry. The plural form is more commonly used in all parts of the Bible, The Hebrew word Jehovah (q.v.), the only other word generally employed to denote the Supreme Being, is uniformly rendered in the Authorized Version by "LORD," printed in small capitals. The existence of God is taken for granted in the Bible. There is nowhere any argument to prove it. He who disbelieves this truth is spoken of as one devoid of understanding ( Psalms 14:1 ). The arguments generally adduced by theologians in proof of the being of God are: <li> The a priori argument, which is the testimony afforded by reason. <li> The a posteriori argument, by which we proceed logically from the facts of experience to causes. These arguments are, (a) T...</code> | <code>And if ye offer the blind for sacrifice, is it not evil? and if ye offer the lame and sick, is it not evil? offer it now unto thy governor; will he be pleased with thee, or accept thy person? saith the Lord of hosts.</code> | <code>1.0</code> | |
| | | <code>Holy Week</code> | <code>For in those days shall be affliction, such as was not from the beginning of the creation which God created unto this time, neither shall be.</code> | <code>1.0</code> | |
| | * Loss: [<code>CosineSimilarityLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosinesimilarityloss) with these parameters: |
| | ```json |
| | { |
| | "loss_fct": "torch.nn.modules.loss.MSELoss" |
| | } |
| | ``` |
| | |
| | ### Training Hyperparameters |
| | #### Non-Default Hyperparameters |
| | |
| | - `num_train_epochs`: 1 |
| | - `max_steps`: 50 |
| | - `multi_dataset_batch_sampler`: round_robin |
| | |
| | #### All Hyperparameters |
| | <details><summary>Click to expand</summary> |
| | |
| | - `overwrite_output_dir`: False |
| | - `do_predict`: False |
| | - `eval_strategy`: no |
| | - `prediction_loss_only`: True |
| | - `per_device_train_batch_size`: 8 |
| | - `per_device_eval_batch_size`: 8 |
| | - `per_gpu_train_batch_size`: None |
| | - `per_gpu_eval_batch_size`: None |
| | - `gradient_accumulation_steps`: 1 |
| | - `eval_accumulation_steps`: None |
| | - `torch_empty_cache_steps`: None |
| | - `learning_rate`: 5e-05 |
| | - `weight_decay`: 0.0 |
| | - `adam_beta1`: 0.9 |
| | - `adam_beta2`: 0.999 |
| | - `adam_epsilon`: 1e-08 |
| | - `max_grad_norm`: 1 |
| | - `num_train_epochs`: 1 |
| | - `max_steps`: 50 |
| | - `lr_scheduler_type`: linear |
| | - `lr_scheduler_kwargs`: None |
| | - `warmup_ratio`: 0.0 |
| | - `warmup_steps`: 0 |
| | - `log_level`: passive |
| | - `log_level_replica`: warning |
| | - `log_on_each_node`: True |
| | - `logging_nan_inf_filter`: True |
| | - `save_safetensors`: True |
| | - `save_on_each_node`: False |
| | - `save_only_model`: False |
| | - `restore_callback_states_from_checkpoint`: False |
| | - `no_cuda`: False |
| | - `use_cpu`: False |
| | - `use_mps_device`: False |
| | - `seed`: 42 |
| | - `data_seed`: None |
| | - `jit_mode_eval`: False |
| | - `bf16`: False |
| | - `fp16`: False |
| | - `fp16_opt_level`: O1 |
| | - `half_precision_backend`: auto |
| | - `bf16_full_eval`: False |
| | - `fp16_full_eval`: False |
| | - `tf32`: None |
| | - `local_rank`: 0 |
| | - `ddp_backend`: None |
| | - `tpu_num_cores`: None |
| | - `tpu_metrics_debug`: False |
| | - `debug`: [] |
| | - `dataloader_drop_last`: False |
| | - `dataloader_num_workers`: 0 |
| | - `dataloader_prefetch_factor`: None |
| | - `past_index`: -1 |
| | - `disable_tqdm`: False |
| | - `remove_unused_columns`: True |
| | - `label_names`: None |
| | - `load_best_model_at_end`: False |
| | - `ignore_data_skip`: False |
| | - `fsdp`: [] |
| | - `fsdp_min_num_params`: 0 |
| | - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} |
| | - `fsdp_transformer_layer_cls_to_wrap`: None |
| | - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} |
| | - `parallelism_config`: None |
| | - `deepspeed`: None |
| | - `label_smoothing_factor`: 0.0 |
| | - `optim`: adamw_torch_fused |
| | - `optim_args`: None |
| | - `adafactor`: False |
| | - `group_by_length`: False |
| | - `length_column_name`: length |
| | - `project`: huggingface |
| | - `trackio_space_id`: trackio |
| | - `ddp_find_unused_parameters`: None |
| | - `ddp_bucket_cap_mb`: None |
| | - `ddp_broadcast_buffers`: False |
| | - `dataloader_pin_memory`: True |
| | - `dataloader_persistent_workers`: False |
| | - `skip_memory_metrics`: True |
| | - `use_legacy_prediction_loop`: False |
| | - `push_to_hub`: False |
| | - `resume_from_checkpoint`: None |
| | - `hub_model_id`: None |
| | - `hub_strategy`: every_save |
| | - `hub_private_repo`: None |
| | - `hub_always_push`: False |
| | - `hub_revision`: None |
| | - `gradient_checkpointing`: False |
| | - `gradient_checkpointing_kwargs`: None |
| | - `include_inputs_for_metrics`: False |
| | - `include_for_metrics`: [] |
| | - `eval_do_concat_batches`: True |
| | - `fp16_backend`: auto |
| | - `push_to_hub_model_id`: None |
| | - `push_to_hub_organization`: None |
| | - `mp_parameters`: |
| | - `auto_find_batch_size`: False |
| | - `full_determinism`: False |
| | - `torchdynamo`: None |
| | - `ray_scope`: last |
| | - `ddp_timeout`: 1800 |
| | - `torch_compile`: False |
| | - `torch_compile_backend`: None |
| | - `torch_compile_mode`: None |
| | - `include_tokens_per_second`: False |
| | - `include_num_input_tokens_seen`: no |
| | - `neftune_noise_alpha`: None |
| | - `optim_target_modules`: None |
| | - `batch_eval_metrics`: False |
| | - `eval_on_start`: False |
| | - `use_liger_kernel`: False |
| | - `liger_kernel_config`: None |
| | - `eval_use_gather_object`: False |
| | - `average_tokens_across_devices`: True |
| | - `prompts`: None |
| | - `batch_sampler`: batch_sampler |
| | - `multi_dataset_batch_sampler`: round_robin |
| | - `router_mapping`: {} |
| | - `learning_rate_mapping`: {} |
| | |
| | </details> |
| | |
| | ### Framework Versions |
| | - Python: 3.11.14 |
| | - Sentence Transformers: 5.2.0 |
| | - Transformers: 4.57.6 |
| | - PyTorch: 2.10.0+cpu |
| | - Accelerate: 1.12.0 |
| | - Datasets: 4.5.0 |
| | - Tokenizers: 0.22.2 |
| | |
| | ## Citation |
| | |
| | ### BibTeX |
| | |
| | #### Sentence Transformers |
| | ```bibtex |
| | @inproceedings{reimers-2019-sentence-bert, |
| | title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", |
| | author = "Reimers, Nils and Gurevych, Iryna", |
| | booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", |
| | month = "11", |
| | year = "2019", |
| | publisher = "Association for Computational Linguistics", |
| | url = "https://arxiv.org/abs/1908.10084", |
| | } |
| | ``` |
| | |
| | <!-- |
| | ## Glossary |
| |
|
| | *Clearly define terms in order to be accessible across audiences.* |
| | --> |
| |
|
| | <!-- |
| | ## Model Card Authors |
| |
|
| | *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* |
| | --> |
| |
|
| | <!-- |
| | ## Model Card Contact |
| |
|
| | *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* |
| | --> |
