SentenceTransformer based on nomic-ai/modernbert-embed-base

This is a sentence-transformers model finetuned from nomic-ai/modernbert-embed-base on the ssf-train-valid-v3 dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: nomic-ai/modernbert-embed-base
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- ssf-train-valid-v3

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Fatini/ssf-retriever-modernbert-embed-base-attempt1-2")
# Run inference
sentences = [
    'The Architectural Executive is responsible for reviewing data during the feasibility and design phases. He/She is involved with developing computational models. He provides advice on procurement methods to meet objectives. He collaborates with others and communicates with relevant teams as necessary. He continually looks for improvement opportunities. He also looks for opportunities to incorporate the latest industry trends into his work. He is analytical, technologically savvy and is able to use the latest digital tools in the course of his work. He is able to communicate with different stakeholders effectively. He is usually based in the office and is required to be on-site occasionally.',
    'The Architectural Executive is tasked with evaluating data throughout the feasibility and design stages. He/She participates in creating computational models and offers guidance on procurement strategies to achieve goals. Collaboration with colleagues and communication with pertinent teams is essential. He consistently seeks opportunities for enhancement and aims to integrate the most recent industry trends into his projects. He possesses strong analytical skills, is technologically proficient, and effectively utilizes advanced digital tools in his work. His ability to engage with various stakeholders is crucial. He typically operates from the office but is required to be present on-site from time to time.',
    'The Architectural Executive is focused on assessing financial reports during the audit and compliance phases. He/She is responsible for developing risk management strategies. He provides consultation on investment opportunities to align with corporate objectives. He often works independently and interacts with external partners as necessary. He frequently seeks ways to minimize costs and reduce operational inefficiencies. He is detail-oriented, financially savvy, and adept at utilizing traditional accounting tools in his tasks. He regularly communicates with upper management and is primarily based in a remote setting.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9407, 0.4348],
#         [0.9407, 1.0000, 0.4434],
#         [0.4348, 0.4434, 1.0000]])

Training Details

Training Dataset

ssf-train-valid-v3

Dataset: ssf-train-valid-v3 at 6bb80aa
Size: 3,016 training samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 1000 samples:

	anchor	positive	negative
type	string	string	string
details	min: 57 tokens mean: 171.02 tokens max: 380 tokens	min: 54 tokens mean: 160.13 tokens max: 325 tokens	min: 51 tokens mean: 132.96 tokens max: 265 tokens

Samples:

anchor	positive	negative
The Business Development Executive/Distribution Executive/Channel Executive/Partnerships and Affinity Management Executive assumes the responsibility of executing the various sales and marketing activities, monitoring the various channels for prospective partners and evaluating market research findings, data analysis and presenting the findings. He/She provides market feedback to the management. In addition, he supports relationship building with existing and new buyers and vendors, and organises events and roadshows. The Business Development Executive/Distribution Executive/Channel Executive/Partnerships and Affinity Management Executive is an individual who is able to multi-task and consistently meet deadlines in a fast-paced environment.	The Business Development Executive is tasked with carrying out a range of sales and marketing initiatives, overseeing channels for potential partners, and assessing market research results. This role involves data analysis and presenting insights to management. Furthermore, the executive fosters relationships with both current and prospective buyers and vendors while organizing events and promotional activities. The ideal candidate for the Business Development Executive position is adept at multi-tasking and excels in meeting deadlines within a dynamic work environment.	The Business Development Executive is responsible for managing administrative tasks related to customer service, maintaining relationships with clients, and coordinating internal team meetings. He/She provides support to the management by gathering reports and conducting employee training sessions. Additionally, the executive engages in routine clerical duties and assists with office supplies management. The Business Development Executive must be proficient in juggling multiple projects while adhering to strict office protocols in a structured environment.
The Landscape Technician executes technical processes involved in landscape maintenance. He/She performs maintenance tasks using chemicals, tools and equipment, for the purpose of hardscape and softscape maintenance. He identifies challenges that he may face during the performance of his tasks and reports them to supervisors. He also performs routine maintenance of tools and equipment. He executes allocated landscape maintenance tasks independently under prescribed guidelines and workplace safety and health regulations. The work involves high amount of manual work as well as the use of significant amount of mechanisation and automation. The work is primarily expected to be performed outdoors on landscape site. He should be task oriented and comfortable in strict adherence to processes and procedures. He should also have an eye for analysing tasks and looking for ways to improving processes.	The Landscape Technician carries out essential technical operations related to the upkeep of landscapes. This role involves performing maintenance activities using various chemicals, tools, and equipment to ensure the proper care of both hardscape and softscape elements. The technician identifies potential challenges encountered during tasks and communicates them to supervisors. Additionally, regular maintenance of tools and equipment is part of the role. The technician executes assigned landscape maintenance tasks independently while adhering to established guidelines and workplace safety regulations. The position requires a considerable amount of manual labor, alongside the use of mechanization and automation, and is primarily conducted outdoors on landscape sites. A focus on task completion and strict compliance with processes and procedures is essential, as well as a keen ability to analyze tasks and seek improvements in workflows.	The Landscape Designer creates innovative designs for various outdoor spaces. He/She develops plans using computer software, sketches, and models, aimed at enhancing aesthetic appeal and functionality. He identifies design challenges that may arise during project execution and discusses them with clients. He also oversees the installation of design elements and ensures compliance with local zoning laws and regulations. The work involves a high level of creative thinking and artistic vision, as well as collaboration with other professionals in the construction industry. He should be detail-oriented and comfortable with flexible project guidelines. He should also have a talent for visualizing spaces and proposing unique concepts to clients.
The Customer Success Director is responsible for establishing strategies to drive customer satisfaction to increase retention and lifetime value for the organisation. He/She defines critical success factors for the team and provides advice on the development of client onboarding, engagement initiatives and programs to ensure successful adoption of solutions and realisation of optimal value for the client. He oversees the development of educational resources and case studies, as well as recommendations and action plans to address challenges faced by the client. He leverages relationships with clients to drive opportunities for new business developments and up-selling and cross-selling. He works in a fast-paced and dynamic environment, and visits clients' premises as and when required. He is familiar with client relationship management and sales tools, as well as customer service frameworks and practices. He is knowledgeable of best practices pertaining to the use of the organisation's p...	The Customer Success Director is tasked with creating strategies to enhance customer satisfaction, thereby boosting retention and lifetime value for the organization. He/She identifies key success metrics for the team and offers guidance on the formulation of client onboarding processes, engagement strategies, and initiatives to ensure effective solution adoption and maximization of client value. He supervises the creation of instructional materials and case studies, along with strategic recommendations and action plans to resolve client challenges. He utilizes relationships with clients to uncover opportunities for new business initiatives and promote up-selling and cross-selling. He operates in a dynamic and fast-paced setting, visiting client locations as necessary. He is well-versed in client relationship management and sales tools, as well as customer service methodologies. He understands best practices regarding the organization's products and services, as well as the specific in...	The Customer Success Analyst is responsible for implementing strategies to drive customer engagement to enhance support and satisfaction for the company. He/She determines the essential metrics for the department and provides recommendations on the execution of client retention, feedback surveys, and initiatives to ensure efficient product delivery and service satisfaction. He oversees the creation of marketing collateral and client testimonials, as well as insights and plans to tackle opportunities presented by the client. He builds connections with customers to encourage prospects for business growth and retention. He operates in a structured and static environment, and conducts meetings at the company's headquarters regularly. He is familiar with project management and marketing tools, as well as product development frameworks and protocols. He is knowledgeable of compliance standards regarding the use of the organization’s services and the client’s operational needs. The Customer S...

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Evaluation Dataset

ssf-train-valid-v3

Dataset: ssf-train-valid-v3 at 6bb80aa
Size: 754 evaluation samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 754 samples:

	anchor	positive	negative
type	string	string	string
details	min: 58 tokens mean: 170.22 tokens max: 403 tokens	min: 62 tokens mean: 159.3 tokens max: 308 tokens	min: 51 tokens mean: 130.84 tokens max: 307 tokens

Samples:

anchor	positive	negative
The Project Supervisor is responsible for supervising the set-up, maintenance and safe and efficient operations of equipment used for moving cargo onto the transportation. He/She is also responsible for overseeing checks on the rigging or mechanised equipment. Analytical and systematic, he is required to supervise rigging and lifting operations, be observant and perform quality checks for the operation. He is also expected to work in rotating shifts and supervise a few teams. He is also expected to work with internal and external stakeholders to accomplish his work.	The Project Supervisor oversees the installation, upkeep, and safe operation of machinery utilized for transporting cargo. He/She is tasked with ensuring that checks are conducted on rigging and mechanical equipment. With a focus on analytical thinking and systematic processes, he/she supervises rigging and lifting activities, maintains vigilance, and conducts quality assessments during operations. The role requires working in rotating shifts and managing multiple teams while collaborating with both internal and external stakeholders to achieve objectives.	The Project Manager is responsible for developing marketing strategies and overseeing promotional campaigns for new products. He/She is tasked with coordinating with creative teams and analyzing market trends to drive sales growth. With a focus on innovative thinking and strategic planning, he/she leads project meetings and evaluates marketing performance metrics. The role requires a standard 9-to-5 schedule and managing a single team while engaging with customers to gather feedback on product features.
The Executive - On-Demand Media Technology and Operations supports the delivery of the organisation's content to the viewers over on-demand media platforms. He/She executes processes such as ingesting, encoding, transcoding and quality checks on incoming content to maintain compliance with the organisation's technical standards. He designs, develops and deploys media applications to meet media workflows and customer content delivery requirements. He also designs, deploys and operationalises content delivery networks (CDNs) to deliver the organisation's content to customers. The work involves end-to-end involvement in on-demand media operations, including design, development, deployment and maintenance. He should be able to think creatively and innovatively. He ought to be process-oriented. He is also expected to be effective at collaboration and stakeholder management to be able to work with teams across the organisation.	The Executive - On-Demand Media Technology and Operations is responsible for facilitating the distribution of the organization's content through various on-demand media platforms. He/She oversees processes such as ingesting, encoding, transcoding, and conducting quality assurance on incoming content to ensure adherence to the organization's technical guidelines. He designs, develops, and implements media applications tailored to media workflows and customer content delivery needs. Additionally, he creates, deploys, and manages content delivery networks (CDNs) to effectively transmit the organization's content to customers. This role requires comprehensive involvement in on-demand media operations, encompassing design, development, deployment, and maintenance. The ideal candidate should possess a creative and innovative mindset, be process-oriented, and excel in collaboration and stakeholder engagement to work effectively with cross-functional teams.	The Executive - On-Demand Media Technology and Operations focuses on the management of live streaming events and broadcasting. He/She is responsible for orchestrating technical setups, including sound engineering, camera operations, and live editing to ensure high-quality live broadcasts. He collaborates with external vendors and partners to enhance live production quality. The work entails extensive logistical planning and real-time problem-solving during events. The ideal candidate should be detail-oriented, possess strong leadership skills, and have experience in managing large teams under pressure.
The Brand Associate/Assistant supports the implementation of plans and initiatives relating to the brand, customer experience and loyalty in order to drive brand equity. He/she also provides support for market research activities to generate relevant market insights for the organisation. He works in a fast-paced and dynamic environment where he provides coordination and administrative support for various initiatives. He is a meticulous, energetic, organised and collaborative individual. He is a fast and independent learner who is able to multi-task effectively.	The Brand Associate/Assistant assists in executing strategies and initiatives focused on enhancing brand presence, customer engagement, and loyalty to boost brand value. They also aid in conducting market research to derive valuable insights for the company. Operating in a vibrant and fast-moving setting, they deliver coordination and administrative assistance for multiple projects. This role requires a detail-oriented, enthusiastic, organized, and team-oriented person who is a quick and self-sufficient learner capable of managing several tasks simultaneously.	`The Brand Manager oversees the development of promotional campaigns and strategies aimed solely at increasing sales revenue. They are responsible for managing a large team and ensuring compliance with corporate regulations. This position requires a strategic thinker who can also perform financial analysis and budgeting, which involves a high level of decision-making authority and years of experience in financial management.`

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
per_device_train_batch_size: 32
per_device_eval_batch_size: 16
gradient_accumulation_steps: 16
learning_rate: 2e-05
num_train_epochs: 5
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: True
tf32: False
load_best_model_at_end: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: epoch
prediction_loss_only: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 16
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 16
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 5
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: False
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss
1.0	6	0.1857	0.0190
2.0	12	0.012	0.0055
3.0	18	0.0067	0.0040
4.0	24	0.0046	0.0034
5.0	30	0.0038	0.0033

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.12.11
Sentence Transformers: 5.1.0
Transformers: 4.55.0
PyTorch: 2.8.0+cu128
Accelerate: 1.10.0
Datasets: 4.0.0
Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}