--- tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:849 - loss:MultipleNegativesRankingLoss base_model: sentence-transformers/all-MiniLM-L12-v2 widget: - source_sentence: Graphic designer who specializes in creating visual content for brands, including logos, marketing materials, and user interfaces. Focuses on aesthetics, user experience, and brand identity. sentences: - 'user_1: I''m looking to refresh my company''s brand image but don''t know where to start. user_2: You should consult a brand manager.' - 'user_1: I need help designing a logo for my new business. user_2: Have you thought about hiring a graphic designer? user_1: Yes, I want something that really represents my brand.' - 'user_1: My car''s making a weird noise, and I don''t know what to do. user_2: You should take it to a mechanic.' - source_sentence: Nutritionist who specializes in dietary planning and nutritional counseling. Helps clients achieve their health goals through personalized meal plans and education. sentences: - 'user_1: I''m trying to lose weight but I don''t know what to eat. user_2: Have you considered talking to a nutritionist?' - 'user_1: Our database is running slow, and I don''t know why. user_2: Have you checked the indexing?' - 'user_1: I need help fixing my car''s engine; it''s making a weird noise. user_2: Have you checked the oil level?' - source_sentence: 'user_2: Sure, what problem are you working on?' sentences: - Gardening expert specializing in vegetable gardening techniques and plant care. - Event planner focusing on corporate events and wedding coordination. - Math tutor specializing in teaching and clarifying mathematical concepts and problem-solving. - source_sentence: 'user_2: Have you thought about getting some storage bins?' sentences: - Web developer focused on software engineering and application design. - Professional organizer specializing in home organization and decluttering strategies. - Pet behavior specialist who provides advice on dog breeds and training for small living spaces. - source_sentence: 'user_1: Maybe the national parks, I want to see some nature.' sentences: - Mental health counselor specializing in stress management and coping strategies. - Data analyst focusing on market trends and business intelligence. - Travel consultant specializing in road trip planning and national park itineraries. pipeline_tag: sentence-similarity library_name: sentence-transformers --- # SentenceTransformer based on sentence-transformers/all-MiniLM-L12-v2 This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2) on the semantic_triplets_round1 and inverse_semantic_triplets datasets. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [sentence-transformers/all-MiniLM-L12-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2) - **Maximum Sequence Length:** 128 tokens - **Output Dimensionality:** 384 dimensions - **Similarity Function:** Cosine Similarity - **Training Datasets:** - semantic_triplets_round1 - inverse_semantic_triplets ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("sentence_transformers_model_id") # Run inference sentences = [ 'user_1: Maybe the national parks, I want to see some nature.', 'Travel consultant specializing in road trip planning and national park itineraries.', 'Data analyst focusing on market trends and business intelligence.', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 384] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Training Details ### Training Datasets #### semantic_triplets_round1 * Dataset: semantic_triplets_round1 * Size: 422 training samples * Columns: anchor, positive, and negative * Approximate statistics based on the first 422 samples: | | anchor | positive | negative | |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------| | type | string | string | string | | details | | | | * Samples: | anchor | positive | negative | |:--------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------| | user_1: Can anyone recommend a good app for tracking my expenses? | Personal finance advisor specializing in budgeting tools and expense tracking applications. | Fitness instructor focusing on workout plans and nutrition. | | user_1: Can anyone recommend a good workout routine for beginners? | Fitness trainer who specializes in creating beginner workout plans and exercise coaching. | Financial advisor focused on investment strategies and retirement planning. | | user_2: What kind of vegetables are you thinking of planting? | Gardening expert who provides guidance on vegetable gardening techniques and plant care. | Investment advisor specializing in stock market strategies and financial planning. | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim" } ``` #### inverse_semantic_triplets * Dataset: inverse_semantic_triplets * Size: 427 training samples * Columns: anchor, positive, and negative * Approximate statistics based on the first 427 samples: | | anchor | positive | negative | |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------| | type | string | string | string | | details | | | | * Samples: | anchor | positive | negative | |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------| | UX researcher specializing in user experience design and user testing. Conducts research to understand user needs and improve product usability. | user_1: I'm looking for ways to improve the usability of our app.
user_2: Have you considered conducting user interviews? | user_1: I need to plan a trip to Europe next summer.
user_2: What countries are you thinking about visiting? | | Software developer specializing in web applications, proficient in various programming languages and frameworks. I design, develop, and maintain software solutions, focusing on user experience and functionality. | user_1: I'm trying to build a web application, but I'm stuck on how to integrate the backend with the frontend.
user_2: What technologies are you using for both?
user_1: I’m using Node.js for the backend and React for the frontend. | user_1: I'm looking for a good recipe for chocolate chip cookies.
user_2: I can share my favorite one! | | Marketing strategist who focuses on developing comprehensive marketing plans to drive brand engagement and sales growth. Specializes in digital marketing and content strategy. | user_1: I'm launching a new product and need a marketing strategy.
user_2: Have you set any goals for your campaign? | user_1: I'm looking for a new pair of running shoes.
user_2: What brand do you prefer? | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim" } ``` ### Evaluation Datasets #### semantic_triplets_round1 * Dataset: semantic_triplets_round1 * Size: 47 evaluation samples * Columns: anchor, positive, and negative * Approximate statistics based on the first 47 samples: | | anchor | positive | negative | |:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------| | type | string | string | string | | details | | | | * Samples: | anchor | positive | negative | |:----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------| | user_1: What's the best way to train my puppy to stop barking? | Dog training specialist focused on behavioral issues and obedience training. | Financial advisor who specializes in investment strategies and wealth management. | | user_2: What vegetables do you want to grow? | Gardening expert specializing in vegetable gardening and sustainable practices. | Real estate agent focusing on home buying and selling. | | user_1: Anyone have tips on how to improve my running time for a 5k? | Running coach specializing in training plans and performance improvement. | Financial advisor focusing on investment strategies and retirement planning. | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim" } ``` #### inverse_semantic_triplets * Dataset: inverse_semantic_triplets * Size: 48 evaluation samples * Columns: anchor, positive, and negative * Approximate statistics based on the first 48 samples: | | anchor | positive | negative | |:--------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------| | type | string | string | string | | details | | | | * Samples: | anchor | positive | negative | |:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------| | Graphic designer who specializes in creating visual content for brands, including logos, marketing materials, and user interfaces. Focuses on aesthetics, user experience, and brand identity. | user_1: I need help designing a logo for my new business.
user_2: Have you thought about hiring a graphic designer?
user_1: Yes, I want something that really represents my brand. | user_1: My car's making a weird noise, and I don't know what to do.
user_2: You should take it to a mechanic. | | Physical therapist specializing in rehabilitation for sports injuries, pain management, and improving mobility through tailored exercise programs. | user_1: I twisted my ankle playing basketball, and it's really swollen.
user_2: Have you seen a doctor about it? | user_1: I'm thinking of redecorating my living room.
user_2: What style are you going for? | | An accountant who specializes in financial record-keeping, tax preparation, and business consulting. Provides services to help clients manage their finances effectively and ensure compliance with tax regulations. | user_1: I need help with my taxes this year.
user_2: Are you looking for someone to prepare them for you? | user_1: I'm thinking about getting a puppy. | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim" } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: steps - `per_device_train_batch_size`: 16 - `per_device_eval_batch_size`: 16 - `learning_rate`: 2e-05 - `num_train_epochs`: 1 - `warmup_ratio`: 0.1 - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: steps - `prediction_loss_only`: True - `per_device_train_batch_size`: 16 - `per_device_eval_batch_size`: 16 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 1 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: None - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `eval_use_gather_object`: False - `average_tokens_across_devices`: False - `prompts`: None - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional
### Framework Versions - Python: 3.12.9 - Sentence Transformers: 4.1.0 - Transformers: 4.52.4 - PyTorch: 2.7.1 - Accelerate: 1.8.1 - Datasets: 3.6.0 - Tokenizers: 0.21.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```