Ermiyas/embedding-1

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2 on the csv dataset. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • csv

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Ermiyas/embedding-1")
# Run inference
sentences = [
    " We're working on a story about negotiating a raise at work during a time of rising inflation. We need these queries answered for the story.\n\n1. What impact, if any, is skyrocketing inflation having on salaries and compensation on the job?\n\n2. Is it reasonable for an employee to ask for a raise at a time of high inflation? Or should job responsibilities be the only factor when asking for a raise?\n\n3. What are your best tips for negotiating a raise at a time of high inflation? What strategies works and what doesn't?\n\nThanks.  |  Looking for negotiation, career and financial experts for an article on negotiating a raise at work at a time of high inflation ",
    ' ',
    'Hey Amy Walters, Thank you for reaching out. Dr. Sandeep Nayak, a Surgical Oncologist (DNB, MRCS UK, Fellowship in Laparoscopic and Robotic Surgical Oncology), provides his professional insight into the claim of curing cancer through dietary changes: "A claim like this needs to be examined carefully. While diet and nutrition play a crucial role in overall health and can support recovery during and after cancer treatment, there is no scientific evidence to suggest that dietary changes alone can cure cancer. The case mentioned, where surgery was performed and a plant-based diet followed, likely reflects the combined effect of medical intervention and lifestyle improvements. Surgery is often a curative treatment for localized cancers, but attributing the cure solely to dietary changes is misleading. What diet does offer is an enhancement to the body’s ability to heal and recover. For example, a diet rich in vegetables, legumes, and whole foods provides antioxidants and anti-inflammatory properties that can support the immune system and potentially reduce the risk of cancer recurrence. However, this should complement, not replace, proven cancer therapies like surgery, chemotherapy, or immunotherapy. It’s important for patients to understand that cancer is multifactorial, involving genetic, environmental, and lifestyle factors. While maintaining a healthy diet is essential for reducing risks and supporting recovery, it is not a standalone cure. Bold claims like this can deter patients from seeking evidence-based treatments, which could have life-saving outcomes." Dr. Nayak emphasizes that diet, while a powerful tool in promoting health, is one part of a larger treatment plan. "The best approach combines advanced medical treatments with lifestyle modifications for long-term well-being and prevention. Patients should always consult their oncologist or healthcare provider before making decisions about cancer care based on anecdotal reports." If you’d like additional commentary or specific examples, please feel free to reach out. Here are Dr. Nayak’s details for attribution: Dr. Sandeep Nayak DNB (General Surgery), DNB (Surgical Oncology), MRCS (UK), MNAMS (General Surgery) Fellowship in Laparoscopic and Robotic Surgical Oncology Profile: Dr. Sandeep Nayak Website: MACS for Cancer',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[     1.0000,     -0.0021,      0.4980],
#         [    -0.0021,      1.0000,      0.0003],
#         [     0.4980,      0.0003,      1.0000]])

Evaluation

Metrics

Semantic Similarity

• Dataset: sts-dev
• Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	0.9238
spearman_cosine	0.8171

Training Details

Training Dataset

csv

• Dataset: csv
• Size: 1,900 training samples
• Columns: request, pitch, and score
• Approximate statistics based on the first 1000 samples:

  	request	pitch	score
  type	string	string	float
  details	min: 16 tokens mean: 120.15 tokens max: 256 tokens	min: 2 tokens mean: 160.7 tokens max: 256 tokens	min: 0.0 mean: 0.75 max: 85.0

• Samples:

  request	pitch	score
  What can be done to improve diversity for financial advisors?	What are tangible ways that the financial advisory industry can increase diversity?	What can be done to improve diversity for financial advisors? What are tangible ways that the financial advisory industry can increase diversity? Even as an eternal optimist, I'll start with the business case: diverse teams are more likely to outperform on the bottom line. McKinsey can give you the stats. With that out of the way, let's talk about action. "Diversity & Inclusion" is often taken to mean hiring statistics, but increasing diversity in the financial advisory industry requires work to build a talent pipeline for advisors and to build better services from advisors. A history of exclusionary practices has resulted in our status quo; today, financial advisors are often assumed to be well-off themselves and to serve a well-off clientele. Changing this history is going to take time, and improving diversity for financial advisors and for the industry won't happen overnight. The industry should start by investing time to build a diverse customer base and to meet their needs. On ...
  UPDATE: Medical Doctors who are also Geriatricians	Topic: Baby Boomers I want to speak with US medical experts (for 15 min) support the medical needs of those born from 1946 and 1964 AKA "Boomers" Previous: or those in professions that support their healthcare and retirement concerns. Aging life Specialists and Care Managers and Geriatricians to the front, please. Someone who can chat with me about the nuance in this demographic and what that are currently facing. -Please don't send ready-made quotes for this one. -Looking for experts who can speak to the health and social concerns of this demographic. It would be ideal to connect with an expert who can speak for 15 minutes on the 11 or 12 of next week. Here are my primary questions about Baby Boomers: What Role do Baby Boomers play in family dynamics today and why does it matter? What are challenges baby boomers are facing when it comes to their health? What are challenges baby boomers are facing when it comes to senio...	Hi Yolande, I'd like to introduce you to Ann Lilly as a potential source for you. Ann is the Brand Lead for House Doctors, a leading home improvement and handyman service, and just last year, she helped launch the Aging in Place program. This program prioritizes fall prevention for the aging population, helping so seniors can live independently and safely in their homes with a solution that addresses their unique needs. One in Four senior citizens in the U.S. suffer a fall each year. With nearly 90% of seniors expressing a desire to remain in their own homes as they age, Ann is helping to ensure that seniors can continue living in their homes with the comfort and safety they deserve. KEY STATS: • 1 in 4 seniors in the U.S. falls each year • Falls result in 3 million emergency department visits annually. • 76% of remodelers have seen a significant or moderate increase in requests for aging-in-place features in the last five years. • The most common aging-i...
  Looking to speak to a researcher that has been in academia 20+ years	Looking to have a quick conversation with a researcher that has been in academia 20+ years and uses Twitter to share their research.

• Loss: CosineSimilarityLoss with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss"
  }
  

Evaluation Dataset

csv

• Dataset: csv
• Size: 316 evaluation samples
• Columns: request, pitch, and score
• Approximate statistics based on the first 316 samples:

  	request	pitch	score
  type	string	string	float
  details	min: 14 tokens mean: 120.31 tokens max: 256 tokens	min: 2 tokens mean: 161.22 tokens max: 256 tokens	min: 0.0 mean: 0.66 max: 0.95

• Samples:

  request	pitch	score
  I am working on an article for USA Today's Blueprint (Business Section) on the best onboarding practices. Looking for an HR professional to weigh in on the following: 1. What are some top onboarding practices? For instance, keep new hires engaged before their joining date, set up team meet and greets, set clear goals and objectives, check in often, collect feedback to refine onboarding practices. 2. Why is it important to implement these best onboarding practices? How does it help with productivity, retention, general morale and well-being in the workplace? 3. What are some challenges to look out for during onboarding? 4. What are some tools to help with the onboarding process?	Human resources professional needed for insights on best onboarding and worker retention practices	Hey Jackie, I work with Keith Harper, the founder and CEO of NY-based Above & Beyond Talent Acquisition. A&B is on a mission to increase executive diversity among the Fortune 500 and along the way Keith has helped hundreds of companies find and successfully onboard candidates. They actually covered some onboarding strategies that maximize retention in a recent blog post here: https://aandbtalent.com/importance-of-mentorship-during-new-employee-onboarding/ I think Keith would be a fantastic resource and I'd be happy to connect you both if you'd like to schedule some time to speak with him. Very best, Bruce
  I’m working on a TravelSavvy feature on the Best Spring Break Destinations for Families With Young Kids—places where parents can relax, kids can play, and everyone gets the most out of their vacation. Whether it's a beach escape with calm waters (for newbie swimmers), a resort packed with kid-friendly activities, or an off-the-beaten-path gem perfect that's safe for family adventures, I want to hear about it! If you rep a destination (hotel, resort, tourism board) that offers an unforgettable spring break experience for families, send me your pitch! Please include links to book, off-peak and peak price ranges, and details on what makes it kid-friendly! I'll respond to those I plan to include.	Best Spring Break Destinations for Families With Young Kids	Hi Ysolt, Hope you're doing well! For your story on the best Spring Break Destinations for Families with Young Kids, I wanted to put forward Cornwall, England for potential consideration. Destination: Cornwall, England Why: Cornwall is the ultimate British beach holiday location in the Spring as the weather gets noticeably warmer here much earlier than in other parts of Britain. The golden sand and turquoise water rivals that of the Caribbean. The quaint fishing villages and stunning beaches make for the perfect family break. During the summer months there will be ice cream on almost every corner, restaurants with delicious locally sourced fish dishes and coves to explore at low tide. It is beautiful corner of England that also offers fantastic car free biking trails (The Camel Trail) ideal for all ages and abilities. Coastal walks providing dramatic cliff tops, film locations from Poldark and more miles of sandy beaches for sandcastles or sunbathing! Quote from Gaby Cecil, Commercial ...
  Hi there, I'm looking for a telecoms engineer to discuss ways to connect to a satellite during a service outage, and share are more similar tips.	Looking for a telecoms engineer	Hi Maria, Cool story you're working on here. I work with Benchmark Electronics, an advanced manufacturing and engineering company based in Arizona. They do A LOT of telecom work (https://www.bench.com/next-gen-communications) and I'd love to find an engineer for you to speak with there. I have one question for you on this though, is the listed deadline you have for collecting sources or is it for getting answers to the questions you have? If its the former, I can package these up for some folks and work to get them to you but if its the latter the turn around may be too tight. Please let me know either way.

• Loss: CosineSimilarityLoss with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 64
• gradient_accumulation_steps: 2
• learning_rate: 0.0001
• num_train_epochs: 7
• warmup_ratio: 0.2
• load_best_model_at_end: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 2
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 0.0001
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 7
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.2
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	sts-dev_spearman_cosine
-1	-1	-	-	0.7685
0.3333	10	11.2084	0.0593	0.7714
0.6667	20	0.0316	0.0308	0.7794
1.0	30	0.0245	0.0334	0.7870
1.3333	40	0.0136	0.0246	0.7978
1.6667	50	11.0605	0.0233	0.8154
2.0	60	0.0155	0.0234	0.8161
2.3333	70	0.0125	0.0244	0.8331
2.6667	80	11.0853	0.0230	0.8264
3.0	90	0.0116	0.0225	0.8261
3.3333	100	0.0071	0.0226	0.8279
3.6667	110	11.0368	0.0227	0.8165
4.0	120	0.0072	0.0226	0.8145
4.3333	130	0.005	0.0222	0.8194
4.6667	140	0.0054	0.0225	0.8193
5.0	150	11.034	0.0226	0.8103
5.3333	160	0.0036	0.0225	0.8150
5.6667	170	0.0038	0.0228	0.8193
6.0	180	11.0309	0.0224	0.8168
6.3333	190	11.0297	0.0224	0.8163
6.6667	200	0.0029	0.0225	0.8165
7.0	210	0.0026	0.0225	0.8171

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.11.13
• Sentence Transformers: 5.0.0
• Transformers: 4.55.0
• PyTorch: 2.6.0+cu124
• Accelerate: 1.9.0
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}