--- base_model: distilbert/distilbert-base-uncased library_name: sentence-transformers pipeline_tag: sentence-similarity tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:676193 - loss:MultipleNegativesRankingLoss widget: - source_sentence: which type of tides have the largest range sentences: - 'Your BMI is based on your height and weight. It''s one way to see if you''re at a healthy weight. Underweight: Your BMI is less than 18.5. Healthy weight: Your BMI is 18.5 to 24.9. Overweight: Your BMI is 25 to 29.9. Obese: Your BMI is 30 or higher. The chart below shows examples of body mass indexes. The figure at which your height corresponds with your weight is your body mass index.' - 'For example, a slight color change in the test pad for protein may indicate a small amount of protein present in the urine whereas a deep color change may indicate a large amount. The most frequently performed chemical tests using reagent test strips are: 1 Specific gravity.' - When the moon is full or new, the gravitational pull of the moon and sun are combined. At these times, the high tides are very high and the low tides are very low. This is known as a spring high tide. Spring tides are especially strong tides (they do not have anything to do with the season Spring). They occur when the Earth, the Sun, and the Moon are in a line. The gravitational forces of the Moon and the Sun both contribute to the tides. Spring tides occur during the full moon and the new moon. - source_sentence: what is the mexican hat dance what are the moves sentences: - 'You’ve probably heard about the mis-selling of payment protection insurance, the “reclaim PPI” adverts, and the refunds people have received. Because of the high payouts, a lot of claims management companies have sprung up, trying to earn commissions from claiming refunds on behalf of their clients. ' - These symptoms could be signs of a bacterial infection, such as strep throat. Taking antibiotics won’t help at all if your sore throat is caused by viruses, but they’re essential for fighting bacterial infections like strep. Strep is the most common bacterial throat infection. Although it can occur in adults, strep throat is more common in children between ages 5 and 15. Riddle says strep can be harder to detect in younger children, because it can cause a runny nose and other symptoms that make it seem like a cold. Another fairly common throat infection is tonsillitis, which occurs when you have sore, swollen tonsils. It’s caused by many of the same viruses and bacteria that cause sore throats. If you have frequent bouts of tonsillitis or strep throat, you may need surgery (called a tonsillectomy) to have your tonsils removed. - 'Jarabe Tapatio (Mexican Hat Dance) -- April 2010. To learn the dance often considered the national dance of Mexico. To learn words from the Spanish language and facts about the country of Mexico. ' - source_sentence: where is murchison location sentences: - Share. The cerebral cortex is the layer of the brain often referred to as gray matter. The cortex (thin layer of tissue) is gray because nerves in this area lack the insulation that makes most other parts of the brain appear to be white. The cortex covers the outer portion (1.5mm to 5mm) of the cerebrum and cerebellum. The portion of the cortex that covers the cerebrum is called the cerebral cortex. The cerebral cortex consists of folded bulges called gyri that create deep furrows or fissures called sulci. - Murchison is a small riverside rural village located on the Goulburn River in Victoria, Australia. Murchison is located 167 kilometres from Melbourne and is just to the west of the Goulburn Valley Highway between Shepparton and Nagambie. The surrounding countryside contains orchards, vineyards and dairy farms and also HM Prison Dhurringile. At the 2011 census, Murchison had a population of 1,047 - Medicare beneficiary means an individual who is entitled to benefits under medicare part A plan and enrolled under medicare part B plan or enrolled in both medicare part A and part B plan and who resides in the U.S. Medicare beneficiaries pay deductibles and 20 percent coinsurance for most services and equipment. Whenever admitted to a hospital for a new spell of illness or benefit period, a beneficiary is entitled to another 90 days of Part A coverage. In addition, each Medicare beneficiary has a lifetime reserve of 60 days that the beneficiary may elect to use toward one or more hospital stays. 42 C.F.R. § 409.61 [a] [2]. However, if the beneficiary has elected to apply the 60 reserve days to a previous hospital stay, the lifetime reserve is exhausted - source_sentence: is hpv a std sentences: - 'HPV is the most common sexually transmitted infection (STI). HPV is a different virus than HIV and HSV (herpes). HPV is so common that nearly all sexually active men and women get it at some point in their lives. There are many different types of HPV. ' - Hibiscus plants reach a wide variety of heights due to the diversity of the species. Grown as annuals, perennials or shrubs, the height range includes dwarf varieties as well taller plants that grow up to 15 feet tall. Red leaf hibiscus (H. acetosella) is an annual tropical shrub that grows to a height of 5 feet and displays deep red leaves. Great rose mallow (Hibiscus grandiflorus) is a perennial species that displays light pink blooms at a height of 8 feet, according to the Clemson University Extension. Additionally, hollyhocks (Alcea rosea) often reach 8 feet in height and display flowers in vivid colors - Snake bites to people tend to be warning bites, and as such contain little venom. The most common venomous snake in the eastern states, copperheads are considered pit vipers, but unlike most other vipers, the copperhead does not flee when it is caught unawares. Instead, the snake will freeze in its current position. Of all the pit vipers, copperhead venom is the least toxic. Breeding does not take place every year, but a female snake will give birth to live young. Litters can consist of up to twenty young, though fewer than ten is most common. As with the majority of reptiles, the babies are on their own once they are born. - source_sentence: how long crocodile live without food sentences: - 'Copper is a chemical element with symbol Cu (from Latin: cuprum) and atomic number 29. It is a ductile metal with very high thermal and electrical conductivity. Pure copper is soft and malleable; a freshly exposed surface has a reddish-orange color. It is used as a conductor of heat and electricity, a building material, and a constituent of various metal alloys.' - Watercress, a slightly sweet and spicy green that you won’t find at every market, is an amazingly delicious green to enjoy when you get the chance. Reminiscent of arugula and spinach combined, you’ll find it often still with the roots attached or even sold in small water pots at stores like Whole Foods. The cruciferous veggies like watercress, kale, broccoli, cabbage, etc., all topped the list while other leafy greens such as spinach, romaine, and beet greens also ranked high on the list. - 'Share to: about 1 week actually, but most people say 2 weeks, but that is a long time if you think about it. New answer; People who deliberatley stop eating can go for about 2 weeks, an … d it tends to be skinny people who do this. You can go a long time without food but not even 2 days or so without water....' --- # SentenceTransformer based on distilbert/distilbert-base-uncased This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 768 tokens - **Similarity Function:** Cosine Similarity ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: DistilBertModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("aryanmagoon/ms_marco_bi_encoder") # Run inference sentences = [ 'how long crocodile live without food', 'Share to: about 1 week actually, but most people say 2 weeks, but that is a long time if you think about it. New answer; People who deliberatley stop eating can go for about 2 weeks, an … d it tends to be skinny people who do this. You can go a long time without food but not even 2 days or so without water....', 'Copper is a chemical element with symbol Cu (from Latin: cuprum) and atomic number 29. It is a ductile metal with very high thermal and electrical conductivity. Pure copper is soft and malleable; a freshly exposed surface has a reddish-orange color. It is used as a conductor of heat and electricity, a building material, and a constituent of various metal alloys.', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 676,193 training samples * Columns: sentence_0, sentence_1, and label * Approximate statistics based on the first 1000 samples: | | sentence_0 | sentence_1 | label | |:--------|:---------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:---------------------------------------------------------------| | type | string | string | float | | details | | | | * Samples: | sentence_0 | sentence_1 | label | |:--------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------| | what airport is closest to rinteln germany | What is the closest airport to Berlin, Germany? The closest international and non-international airports to Berlin, Germany are listed below in order of increasing distance. | 0.0 | | what is javaone | JavaOne™ coffee pods are individually engineered with the precise roast level, grind setting, blending and dosage to achieve the best tasting pods. Starting with only the finest quality Arabica coffee beans, we roast our beans using hot air for a consistent, even roast throughout the entire bean. While traditional drum roasting can overcook the outside of the bean and undercook the inside, our beans are evenly roasted for a smoother, richer taste. | 0.0 | | what does watercress taste like | Watercress, a slightly sweet and spicy green that you won’t find at every market, is an amazingly delicious green to enjoy when you get the chance. Reminiscent of arugula and spinach combined, you’ll find it often still with the roots attached or even sold in small water pots at stores like Whole Foods. The cruciferous veggies like watercress, kale, broccoli, cabbage, etc., all topped the list while other leafy greens such as spinach, romaine, and beet greens also ranked high on the list. | 0.0 | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim" } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `per_device_train_batch_size`: 64 - `per_device_eval_batch_size`: 64 - `multi_dataset_batch_sampler`: round_robin #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: no - `prediction_loss_only`: True - `per_device_train_batch_size`: 64 - `per_device_eval_batch_size`: 64 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 5e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1 - `num_train_epochs`: 3 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.0 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: True - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `eval_use_gather_object`: False - `batch_sampler`: batch_sampler - `multi_dataset_batch_sampler`: round_robin
### Training Logs | Epoch | Step | Training Loss | |:------:|:----:|:-------------:| | 0.1893 | 500 | 1.2126 | | 0.3786 | 1000 | 0.2246 | | 0.5680 | 1500 | 0.1542 | | 0.7573 | 2000 | 0.1332 | | 0.9466 | 2500 | 0.115 | | 1.1359 | 3000 | 0.1025 | | 1.3253 | 3500 | 0.0929 | | 1.5146 | 4000 | 0.081 | | 1.7039 | 4500 | 0.074 | | 1.8932 | 5000 | 0.0669 | | 2.0825 | 5500 | 0.0605 | | 2.2719 | 6000 | 0.0563 | | 2.4612 | 6500 | 0.047 | | 2.6505 | 7000 | 0.0433 | | 2.8398 | 7500 | 0.0391 | ### Framework Versions - Python: 3.10.14 - Sentence Transformers: 3.1.1 - Transformers: 4.45.1 - PyTorch: 2.4.1+cu121 - Accelerate: 0.34.2 - Datasets: 3.0.1 - Tokenizers: 0.20.0 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```