Fatin757/ssf-retriever-modernbert

SentenceTransformer based on answerdotai/ModernBERT-base

This is a sentence-transformers model finetuned from answerdotai/ModernBERT-base on the ssf-train-valid-v3-cleaned dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: answerdotai/ModernBERT-base
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • ssf-train-valid-v3-cleaned

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Fatin757/ssf-retriever-modernbert")
# Run inference
sentences = [
    'The Procurement Manager leads the procurement function and is responsible for establishing procurement plans, policies and processes required to obtain the materials, equipment and services to deliver on marine manufacturing projects. He/She leads improvement initiatives to enhance the procurement process and timeliness of providing the necessary resources to complete projects on schedule. He has good communication and negotiation skills for engaging vendors and other external parties, and is able to balance the organisations needs, as the job encompasses a strategic role in selecting new vendors, fostering relationships and managing vendor performance and contract-related processes by liaising with legal and other key stakeholders.',
    'The Purchasing Director oversees the purchasing operations and is tasked with developing comprehensive purchasing strategies, policies, and procedures essential for acquiring materials, equipment, and services necessary for successful marine manufacturing projects. This role involves spearheading initiatives aimed at optimizing the procurement process and ensuring timely delivery of resources to meet project deadlines. The ideal candidate possesses excellent communication and negotiation abilities to effectively engage with suppliers and other external partners, ensuring a balance between organizational requirements and vendor capabilities. This strategic position includes selecting new suppliers, nurturing relationships, and managing vendor performance and contractual agreements in collaboration with legal and other critical stakeholders.',
    "The Supply Chain Coordinator manages the supply chain operations and is responsible for implementing logistics plans, policies, and processes necessary to transport materials, equipment, and services for various manufacturing projects. This position focuses on enhancing the logistics process and ensuring timely delivery of supplies to meet production schedules. The candidate should have strong communication and negotiation skills for interacting with transporters and other external parties, while balancing the company's logistics needs. The role includes selecting new transport partners, maintaining relationships, and overseeing transportation performance and contract-related matters by working closely with compliance and other relevant departments.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9002, 0.5999],
#         [0.9002, 1.0000, 0.6487],
#         [0.5999, 0.6487, 1.0000]])

Training Details

Training Dataset

ssf-train-valid-v3-cleaned

• Dataset: ssf-train-valid-v3-cleaned at b829f41
• Size: 2,316 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 57 tokens mean: 176.53 tokens max: 403 tokens	min: 73 tokens mean: 170.94 tokens max: 299 tokens	min: 68 tokens mean: 127.18 tokens max: 257 tokens

• Samples:

  anchor	positive	negative
  The Audit Associate/Audit Assistant Associate undertakes specific stages of audit work under supervision. He/She begins to appreciate the underlying principles behind the tasks assigned to him as part of the audit plan. He is also able to make adjustments to the application of skills to improve the work tasks or solve non-complex issues. The Audit Associate/Audit Assistant Associate operates in a structured work environment. He is able to build relationships, work in a team and identify ethical issues with reference to the code of professional conduct and ethics. He is able to select and apply from a range of known solutions to familiar problems and takes responsibility for his own learning and performance. He is a trustworthy and meticulous individual.	The Audit Trainee plays a crucial role in assisting with various stages of the audit process while working under the guidance of experienced auditors. This position offers the opportunity to deepen understanding of audit principles and methodologies as the trainee engages in the execution of the audit plan. The Audit Trainee is encouraged to adapt and refine their skills to enhance task efficiency and address straightforward challenges. Operating within a structured environment, the Audit Trainee collaborates effectively with team members and develops strong professional relationships. They are also expected to recognize ethical dilemmas in line with the professional code of conduct. Taking ownership of their learning journey and performance, the Audit Trainee is a diligent and reliable individual committed to excellence.	The Junior Risk Analyst is responsible for supporting the risk management team in identifying and assessing potential risks within the organization. This entry-level position involves gathering data, conducting preliminary analyses, and assisting in the development of risk mitigation strategies. The Junior Risk Analyst operates in a dynamic environment where teamwork and communication are essential. They will learn to navigate ethical considerations relevant to risk assessment and are expected to contribute to discussions on known solutions to common risk-related issues. While they are still developing their expertise, the Junior Risk Analyst is encouraged to take initiative in their professional growth and demonstrate attention to detail in their work.
  The Audit Partner/Audit Director is a transformational leader who steers the organisation to achieve its business goals and objectives by formulating technical and strategic directions to drive change. He/She provides strategic vision and leadership to the organisation in order to develop and strengthen organisational capabilities and culture. The Audit Partner/Audit Director is expected to promote new ideas and business solutions that result in extended services to existing clients. He constantly seeks to expand client base and support business development activities. He also establishes consistent and rigorous quality and risk management processes and procedures. The Audit Partner/Audit Director uses a multitude of controls and procedures consisting professional, regulatory, business, economic, social and environmental conditions to manage risk exposure.	The Chief Audit Officer is a visionary leader who guides the organization in achieving its strategic objectives by establishing comprehensive technical and strategic frameworks to foster transformation. This role involves providing a clear strategic direction and leadership to enhance the organization’s capabilities and culture. The Chief Audit Officer is responsible for encouraging innovative ideas and business solutions that enhance service offerings to current clients while actively pursuing opportunities to expand the client portfolio. Additionally, this leader ensures the implementation of robust quality and risk management processes and practices. Utilizing a range of controls and procedures that consider professional, regulatory, business, economic, social, and environmental factors, the Chief Audit Officer effectively manages risk exposure within the organization.	The Junior Risk Analyst is responsible for assisting in the identification and assessment of potential risks within the organization. This entry-level position involves supporting senior analysts in evaluating risk management strategies and implementing basic risk assessment processes. The Junior Risk Analyst will work closely with various departments to gather data and analyze trends that may impact the organization’s operations. This role also includes maintaining documentation and reports related to risk assessments and ensuring compliance with established regulations. The Junior Risk Analyst is expected to contribute to the development of risk mitigation strategies, although they may not have the authority to make strategic decisions independently.
  The Audit Senior is expected to team lead various sizes of audit engagements which include planning, performing, reviewing of audit work performed by less experienced personnel and concluding the engagement. He/She works closely with team members to execute the audit plan and ensures the timely completion of the audit engagement. He also works closely with clients and manages their expectations to further nurture on-going client relationships. He is able to identify unethical behaviour and react based on the applicable ethical principles. The Audit Senior operates in work contexts that are less standardised and unanticipated changes can be frequent due to changes in the external environment. He is a trustworthy and meticulous individual.	The Audit Manager is responsible for leading a variety of audit engagements, overseeing the planning, execution, and review of audit activities conducted by junior staff members. This role involves collaborating with team members to implement the audit strategy effectively and ensuring that all audit projects are completed within the designated timelines. The Audit Manager also plays a crucial role in maintaining strong relationships with clients by managing their expectations and addressing their concerns promptly. With a keen eye for identifying unethical practices, he/she adheres to established ethical standards in all situations. The Audit Manager operates in dynamic environments where adaptability to unexpected changes is essential, showcasing a high level of trustworthiness and attention to detail.	The Junior Financial Analyst will support the finance team by conducting data analysis and assisting in the preparation of financial reports. This role involves gathering and organizing financial information, as well as helping to monitor budgetary compliance. The Junior Financial Analyst will work under the supervision of senior analysts, ensuring that all tasks are completed accurately and on time. Additionally, this position requires basic knowledge of financial principles and the ability to work collaboratively within a team. The Junior Financial Analyst must demonstrate strong organizational skills and attention to detail, while also being open to learning and developing new skills in the financial domain.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

ssf-train-valid-v3-cleaned

• Dataset: ssf-train-valid-v3-cleaned at b829f41
• Size: 580 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 580 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 73 tokens mean: 161.41 tokens max: 349 tokens	min: 75 tokens mean: 155.73 tokens max: 296 tokens	min: 65 tokens mean: 140.06 tokens max: 261 tokens

• Samples:

  anchor	positive	negative
  The Pricing Actuarial Executive is responsible for gathering market and statistical data in developing pricing models for insurance products. He/She collaborates with other stakeholders to ensure successful launches of products. He also helps to evaluate product risks and reviews reinsurance programmes. He facilitates reinsurance administration to ensure it is completed in an accurate and timely manner. The Pricing Actuarial Executive works with relevant key stakeholders and handling data and information. He is attentive to details and possesses good time management and strong people skills. He also has strong technical skills and is able to convey his idea in an effective manner.	The Insurance Pricing Analyst is tasked with collecting and analyzing market and statistical data to create effective pricing models for various insurance products. This role involves close collaboration with cross-functional teams to ensure the successful launch of new products. Additionally, the Insurance Pricing Analyst assesses product risks and reviews reinsurance strategies to optimize coverage. They play a crucial role in managing reinsurance processes, ensuring accuracy and timeliness in administration. The ideal candidate will be detail-oriented, possess excellent time management abilities, and demonstrate strong interpersonal skills. Furthermore, they should have robust technical expertise and the ability to communicate ideas clearly and effectively.	The Risk Management Specialist is responsible for collecting and analyzing market and statistical data to develop risk assessment models for financial products. He/She collaborates with various departments to ensure the effective implementation of risk management strategies. Additionally, he helps to evaluate potential risks and reviews compliance programs. He facilitates risk administration to ensure it is executed in an accurate and timely manner. The Risk Management Specialist works with relevant key stakeholders and manages data and information. He is attentive to details and possesses good time management and strong communication skills. He also has strong analytical skills and is able to convey his findings in an effective manner.
  The Reserving Actuarial Executive is responsible for analysing, monitoring and reporting insurance products in the market as well as managing internal product pricing and reinsurance negotiations. He/She conducts periodic reviews of reserving parameters and prepares reporting in adherence to regulatory requirements. He facilitates reinsurance administration to ensure it is completed in an accurate and timely manner. The Reserving Actuarial Executive works with relevant key stakeholders and handling data and information. He possesses strong analytical abilities with a good understanding of actuarial techniques, standards and assumptions. He also has strong knowledge of insurance operations and is able to convey complex information clearly and concisely.	The Actuarial Pricing Specialist is tasked with evaluating, overseeing, and reporting on insurance products available in the market while also managing internal pricing strategies and negotiating reinsurance agreements. This role involves conducting regular assessments of reserving parameters and preparing reports in compliance with regulatory standards. The Actuarial Pricing Specialist ensures that reinsurance administration is executed accurately and promptly. Collaborating with key stakeholders, this individual handles data and information proficiently. A strong analytical skill set is essential, along with a solid grasp of actuarial methods, standards, and assumptions. Additionally, they must possess extensive knowledge of insurance operations and be capable of communicating intricate information in a clear and concise manner.	The Risk Management Analyst is responsible for evaluating, overseeing, and reporting on potential risks associated with financial products in the market while also managing internal risk assessment strategies and negotiating insurance agreements. This role involves conducting regular assessments of risk parameters and preparing reports in adherence to compliance standards. The Risk Management Analyst ensures that risk administration is completed accurately and in a timely manner. Collaborating with key stakeholders, this individual handles data and information efficiently. A strong analytical skill set is essential, along with a solid understanding of risk management techniques, standards, and assumptions. Additionally, they must possess extensive knowledge of financial operations and be able to communicate complex information clearly and concisely.
  The Reserving and Pricing Actuarial Manager is responsible for reviewing the pricing of insurance products, handling financial reporting and analysing statistical data for the organisation. He/She prepares presentations and reports to explain findings to management. He conducts pricing reviews for new and existing insurance products as well as valuations for cash reserves. He oversees risk management and reinsurance administration. The Reserving and Pricing Actuarial Manager manages and works with a diverse team. He possesses strong problem-solving, analytical and people management skills. He is detail-oriented and diligent. He possesses strong mathematical skills in order to determine risks accurately and estimate probabilities.	The Insurance Pricing and Reserving Actuary is tasked with evaluating the pricing strategies for insurance offerings, managing financial reports, and analyzing statistical data to support the organization’s objectives. This role involves preparing comprehensive presentations and reports to communicate insights to senior management. The actuary conducts thorough pricing assessments for both new and existing insurance products and performs valuations to ensure adequate cash reserves. Additionally, they are responsible for overseeing risk management practices and reinsurance processes. The Insurance Pricing and Reserving Actuary leads a diverse team, demonstrating exceptional problem-solving abilities, analytical skills, and effective people management. Attention to detail and a diligent work ethic are essential, along with strong mathematical proficiency to accurately assess risks and estimate probabilities.	The Risk Management and Compliance Officer is responsible for assessing the compliance of insurance operations, preparing regulatory reports, and analyzing risk data to ensure adherence to industry standards. This position involves creating detailed documentation and presentations to inform the compliance team of findings. The officer conducts regular reviews of compliance protocols for both new and existing insurance policies and evaluates the effectiveness of risk mitigation strategies. Furthermore, they oversee compliance training and administration processes. The Risk Management and Compliance Officer leads a varied team, showcasing strong problem-solving capabilities, analytical thinking, and team leadership. They must be detail-oriented and disciplined, with robust analytical skills to evaluate compliance risks effectively and ensure regulatory adherence.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• num_train_epochs: 5
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss
1.0	73	0.5359	0.0290
2.0	146	0.0115	0.0129
3.0	219	0.0028	0.0122
4.0	292	0.006	0.0099
5.0	365	0.0046	0.0100

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.11
• Sentence Transformers: 5.1.0
• Transformers: 4.56.1
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.0
• Datasets: 4.0.0
• Tokenizers: 0.22.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}