Fatin757/ssf-retriever-modernbert-v8-cleaned

SentenceTransformer based on nomic-ai/modernbert-embed-base

This is a sentence-transformers model finetuned from nomic-ai/modernbert-embed-base on the ssf-train-valid-v8-cleaned dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: nomic-ai/modernbert-embed-base
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • ssf-train-valid-v8-cleaned

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Fatin757/ssf-retriever-modernbert-v8-cleaned")
# Run inference
sentences = [
    'The Learning Facilitator delivers learning products and services in a variety of environments, using multiple learning delivery modes and methods. He/She assesses learning needs and adapts the facilitation approach to reflect desired learning outcomes and learner needs. He is responsible for knowledge and skills transfer by delivering learning content, facilitating group discussions and responding to queries. He drives learner development and commitment to continuous learning by actively providing feedback and learner support. He evaluates curriculum effectiveness and recommends improvement areas by collecting learner feedback as well as analysing learning delivery approaches and materials. He is a strong communicator who builds trusted relationships and creates a cooperative and engaging learning environment. He is adaptable and adept at managing multiple stakeholders. He works in multiple different environments, including different learning venues and client sites, and regularly interacts with digital systems.',
    'The Learning Facilitator is responsible for delivering educational programs across diverse settings, employing various instructional methods and delivery formats. They evaluate learner requirements and tailor their teaching strategies to meet specific learning goals and individual needs. Their duties include transferring knowledge and skills by presenting course materials, leading group discussions, and addressing participant inquiries. The facilitator promotes learner growth and encourages ongoing education by providing constructive feedback and support. They assess the effectiveness of training curricula and suggest enhancements through gathering participant feedback and reviewing instructional methods and resources. Strong communication skills enable them to foster trust and create an engaging and collaborative learning atmosphere. They demonstrate flexibility and skillfully manage relationships with multiple stakeholders. Their work spans numerous environments, including different training venues and client locations, with frequent use of digital platforms.',
    'The Retail Store Manager oversees daily retail operations, manages inventory levels, supervises sales staff, and ensures excellent customer service standards are maintained throughout the store. They handle merchandising, coordinate promotional activities, and analyze sales performance data to optimize store profitability. The manager works closely with suppliers and coordinates with the marketing team to drive store traffic and customer engagement.\n\nThe Software Developer designs, codes, and tests software applications based on user requirements. They collaborate with cross-functional teams to develop new features, debug issues, and maintain existing systems. Proficiency in programming languages and software development methodologies is essential. The developer also documents code and participates in code reviews to ensure high-quality deliverables.\n\nThe Human Resources Assistant provides administrative support to the HR department by maintaining employee records, assisting with recruitment processes, and coordinating onboarding activities. They handle scheduling interviews, managing employee benefits documentation, and supporting employee relations initiatives. Strong organizational skills and confidentiality are critical in this role.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9345, 0.1172],
#         [0.9345, 1.0000, 0.1495],
#         [0.1172, 0.1495, 1.0000]])

Training Details

Training Dataset

ssf-train-valid-v8-cleaned

• Dataset: ssf-train-valid-v8-cleaned at 884a157
• Size: 681 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 681 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 58 tokens mean: 167.15 tokens max: 380 tokens	min: 54 tokens mean: 126.93 tokens max: 278 tokens	min: 72 tokens mean: 132.96 tokens max: 292 tokens

• Samples:

  anchor	positive	negative
  The Audit Associate/Audit Assistant Associate undertakes specific stages of audit work under supervision. He/She begins to appreciate the underlying principles behind the tasks assigned to him as part of the audit plan. He is also able to make adjustments to the application of skills to improve the work tasks or solve non-complex issues. The Audit Associate/Audit Assistant Associate operates in a structured work environment. He is able to build relationships, work in a team and identify ethical issues with reference to the code of professional conduct and ethics. He is able to select and apply from a range of known solutions to familiar problems and takes responsibility for his own learning and performance. He is a trustworthy and meticulous individual.	The Audit Associate/Audit Assistant Associate performs designated audit tasks under guidance, gaining an understanding of the fundamental principles behind assigned duties within the audit plan. They adapt their skills to enhance task execution and resolve straightforward problems. Working within a structured environment, they collaborate effectively with team members, recognize ethical considerations according to professional conduct codes, and apply established solutions to routine issues. This role requires a dependable and detail-oriented professional committed to continuous learning and accountability.	The Retail Store Manager is responsible for overseeing daily store operations, managing inventory levels, and leading staff to deliver exceptional customer service. They ensure the store meets sales targets, maintain visual merchandising standards, and handle customer inquiries and complaints efficiently. The Software Developer designs, codes, and tests software applications based on project requirements. They collaborate with cross-functional teams to develop scalable solutions, troubleshoot issues, and maintain documentation throughout the software development lifecycle. The Human Resources Coordinator provides administrative support to the HR team, assists with recruitment and onboarding processes, maintains employee records, and coordinates training programs to enhance workforce development.
  The Audit Senior Manager/Audit Manager manages a portfolio of engagements to deliver high quality audit services. He/she also provides leadership on audit engagements which includes client acceptance process, engagement planning, execution and finalisation of an audit engagement. He is fully accountable for the audit engagement and ensures that the engagement progress against budget and timeline is closely monitored. He also serves to develop and maintain long-term client relationships and value-add to the audit firm by identifying new business development opportunities. The Audit Senior Manager/Audit Manager reviews and provides key technical expertise to ensure the quality of audit work performed is in compliance with professional standards and requirements. He contributes towards continuous improvement in audit methodology and process. He will also assume a greater role in professional development activities such as training, staff recruitment and resource planning.	The Audit Senior Manager/Audit Manager oversees multiple audit engagements to ensure delivery of superior audit services. They lead all phases of the audit process, including client acceptance, planning, execution, and completion, maintaining full accountability for meeting budget and timeline targets. This role involves fostering strong client relationships and identifying opportunities to grow the firm’s business. The Audit Senior Manager/Audit Manager provides technical guidance to uphold audit quality in line with professional standards and actively participates in enhancing audit methodologies. Additionally, they play a significant role in staff development, recruitment, and resource allocation.	The Retail Store Manager is responsible for managing daily store operations, supervising staff, maintaining inventory levels, and ensuring excellent customer service. They coordinate merchandising strategies and promotional activities to drive sales growth and enhance the shopping experience. The Software Developer designs, codes, and tests software applications based on client requirements. They collaborate with cross-functional teams to develop efficient and scalable solutions, maintain documentation, and troubleshoot technical issues. The Human Resources Coordinator supports recruitment efforts, manages employee records, coordinates training programs, and assists with employee engagement initiatives to promote a positive workplace culture.
  The Audit Partner/Audit Director is a transformational leader who steers the organisation to achieve its business goals and objectives by formulating technical and strategic directions to drive change. He/She provides strategic vision and leadership to the organisation in order to develop and strengthen organisational capabilities and culture. The Audit Partner/Audit Director is expected to promote new ideas and business solutions that result in extended services to existing clients. He constantly seeks to expand client base and support business development activities. He also establishes consistent and rigorous quality and risk management processes and procedures. The Audit Partner/Audit Director uses a multitude of controls and procedures consisting professional, regulatory, business, economic, social and environmental conditions to manage risk exposure.	The Audit Partner/Audit Director serves as a visionary leader who guides the organisation toward achieving its business objectives by setting both technical and strategic directions to foster transformation. This role involves providing strong leadership and strategic insight to enhance organisational capabilities and culture. The Audit Partner/Audit Director encourages innovative ideas and business solutions to broaden services offered to current clients, actively pursues client base growth, and supports business development efforts. Additionally, they implement robust quality and risk management frameworks, utilizing a comprehensive set of controls and procedures that consider professional, regulatory, economic, social, and environmental factors to effectively manage risk exposure.	The Retail Store Manager oversees daily store operations, manages inventory levels, and leads a team to ensure excellent customer service and sales performance. This role requires effective staff training and scheduling to maintain smooth store functioning and maximize customer satisfaction. The Software Developer designs, codes, and tests software applications according to user requirements. They collaborate with cross-functional teams to develop technical solutions and ensure system functionality and performance. The Human Resources Coordinator assists with recruitment, onboarding, and employee relations, maintaining personnel records and supporting HR initiatives to promote a positive workplace environment.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

ssf-train-valid-v8-cleaned

• Dataset: ssf-train-valid-v8-cleaned at 884a157
• Size: 171 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 171 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 72 tokens mean: 162.46 tokens max: 337 tokens	min: 34 tokens mean: 97.4 tokens max: 196 tokens	min: 19 tokens mean: 88.67 tokens max: 257 tokens

• Samples:

  anchor	positive	negative
  The Chartering Manager handles all aspects of chartering activities and ensures the profitable employment of a fleet of vessels, based on vessel types and/or area of deployment, while monitoring adherence to the organisations risk management procedures. He/She analyses market research, identifies business development opportunities for the business unit and has a sound understanding of the ship chartering market with a strong drive to succeed. He has excellent analytical and problem-solving skills, with the ability to communicate with various stakeholders.	The Chartering Manager oversees all chartering operations, ensuring the effective and profitable deployment of vessels according to their types and operational regions. This role involves conducting market analysis, spotting new business opportunities, and maintaining compliance with the company’s risk management policies. The manager possesses deep knowledge of the ship chartering industry, demonstrates strong analytical and problem-solving abilities, and communicates effectively with diverse stakeholders.	The Retail Store Manager is responsible for managing daily store activities, supervising sales staff, maintaining inventory levels, and ensuring customers receive excellent service. The Software Developer designs, codes, and tests software applications, collaborates with cross-functional teams to develop new features, and troubleshoots technical issues. The Human Resources Coordinator supports recruitment processes, organizes employee training sessions, maintains personnel records, and assists with employee relations initiatives.
  The Crewing Executive provides operational support to the recruitment and management of seafarers for vessels. He/She handles the administration of compliance requirements for crew onboard vessels and supports the deployment of crew, in accordance to vessel requirements, organisational standards, International Maritime Organisation (IMO) regulations, Standards for Training, Certification and Watchkeeping for Seafarers (STCW) conventions and the Maritime Labour Convention. He also helps to ensure that crewing tasks are performed in adherence to the organisation's health, safety, security, environment and quality (HSSEQ) procedures, and alerts senior management, protection and indemnity (P&I) clubs and relevant authorities in the event that accidents and/or incidents occur. He possesses knowledge of sea-going crew administration and has interpersonal skills to support engagements with internal and external stakeholders for crewing needs.	The Crewing Executive is responsible for supporting the recruitment and management of seafarers for vessels, ensuring compliance with International Maritime Organisation (IMO) regulations, STCW conventions, and the Maritime Labour Convention. This role includes administrating crew deployment according to vessel needs and organisational standards, adhering to HSSEQ procedures, and coordinating with senior management, P&I clubs, and relevant authorities in case of accidents or incidents. The Crewing Executive must have strong knowledge of sea-going crew administration and effective interpersonal skills to collaborate with internal and external stakeholders.	The Retail Store Manager oversees daily store operations, manages inventory levels, supervises retail staff, and ensures excellent customer service standards are met. This role involves coordinating sales promotions, handling customer inquiries, and maintaining visual merchandising to enhance the shopping experience. The Human Resources Coordinator supports recruitment processes, maintains employee records, assists with onboarding activities, and facilitates employee engagement initiatives within the organisation. The Software Developer designs, codes, and tests software applications, collaborates with cross-functional teams to develop new features, and troubleshoots technical issues to improve system performance.
  The Demurrage Analyst/Laytime Analyst/Post Fixture Executive monitors a ship schedule and its status before arrival at the ports, the delivery and re-delivery notices for ships and arranges for freight/hire payments. He/She calculates, negotiates and ensures timely processing of payables/receivables associated with the voyage or hire (e.g. demurrage, third party claims, commissions, port services). He has strong organisational skills and possesses strong analytical and numerical skills, complemented with good communication skills.	The Demurrage Analyst/Laytime Analyst/Post Fixture Executive is responsible for tracking ship schedules and statuses prior to port arrivals, managing delivery and re-delivery notices, and coordinating freight and hire payment arrangements. This role involves calculating and negotiating charges such as demurrage, third-party claims, commissions, and port services, ensuring timely processing of all voyage-related payables and receivables. The candidate must demonstrate excellent organizational abilities, strong analytical and numerical skills, and effective communication capabilities.	The Retail Store Manager oversees daily retail operations, manages stock levels, supervises sales staff, and ensures customers receive excellent service. The Human Resources Coordinator supports recruitment efforts, assists with employee onboarding, maintains personnel records, and coordinates staff training programs. The Software Developer designs, codes, and tests software applications, collaborates with cross-functional teams, and troubleshoots technical issues to enhance system performance.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 16
• learning_rate: 2e-05
• num_train_epochs: 5
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• tf32: False
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 16
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: False
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss
1.0	2	0.1876	0.0948
2.0	4	0.0128	0.0524
3.0	6	0.0028	0.0474
4.0	8	0.0039	0.0476
5.0	10	0.0045	0.0473

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.11
• Sentence Transformers: 5.1.1
• Transformers: 4.56.2
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.0
• Datasets: 4.0.0
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}