SentenceTransformer based on BAAI/bge-base-en-v1.5
This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5 on the parquet dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
Model Details
Model Description
- Model Type: Sentence Transformer
- Base model: BAAI/bge-base-en-v1.5
- Maximum Sequence Length: 512 tokens
- Output Dimensionality: 768 dimensions
- Similarity Function: Cosine Similarity
- Training Dataset:
Model Sources
Full Model Architecture
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': True, 'architecture': 'BertModel'})
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("sentence_transformers_model_id")
queries = [
"data integrity governance PowerBI development Juno Beach",
]
documents = [
'skills: 2-5 y of exp with data analysis/ data integrity/ data governance; PowerBI development; Python; SQL, SOQL\n\nLocation: Juno Beach, FL\nPLEASE SEND LOCAL CANDIDATES ONLY\n\nSeniority on the skill/s required on this requirement: Mid.\n\nEarliest Start Date: ASAP\n\nType: Temporary Project\n\nEstimated Duration: 12 months with possible extension(s)\n\nAdditional information: The candidate should be able to provide an ID if the interview is requested. The candidate interviewing must be the same individual who will be assigned to work with our client. \nRequirements:• Availability to work 100% at the Client’s site in Juno Beach, FL (required);• Experience in data analysis/ data integrity/ data governance;• Experience in analytical tools including PowerBI development, Python, coding, Excel, SQL, SOQL, Jira, and others.\n\nResponsibilities include but are not limited to the following:• Analyze data quickly using multiple tools and strategies including creating advanced algorithms;• Serve as a critical member of data integrity team within digital solutions group and supplies detailed analysis on key data elements that flow between systems to help design governance and master data management strategies and ensure data cleanliness.',
"QualificationsAdvanced degree (MS with 5+ years of industry experience, or Ph.D.) in Computer Science, Data Science, Statistics, or a related field, with an emphasis on AI and machine learning.Proficiency in Python and deep learning libraries, notably PyTorch and Hugging Face, Lightning AI, evidenced by a history of deploying AI models.In-depth knowledge of the latest trends and techniques in AI, particularly in multivariate time-series prediction for financial applications.Exceptional communication skills, capable of effectively conveying complex technical ideas to diverse audiences.Self-motivated, with a collaborative and solution-oriented approach to problem-solving, comfortable working both independently and as part of a collaborative team.\n\nCompensationThis role is compensated with equity until the product expansion and securing of Series A investment. Cash-based compensation will be determined after the revenue generation has been started. As we grow, we'll introduce additional benefits, including performance bonuses, comprehensive health insurance, and professional development opportunities. \nWhy Join BoldPine?\nInfluence the direction of financial market forecasting, contributing to groundbreaking predictive models.Thrive in an innovative culture that values continuous improvement and professional growth, keeping you at the cutting edge of technology.Collaborate with a dedicated team, including another technical expert, setting new benchmarks in AI-driven financial forecasting in a diverse and inclusive environment.\nHow to Apply\nTo join a team that's redefining financial forecasting, submit your application, including a resume and a cover letter. At BoldPine, we're committed to creating a diverse and inclusive work environment and encouraging applications from all backgrounds. Join us, and play a part in our mission to transform financial predictions.",
'skills and expertise, experience and other relevant factors (salary may be adjusted based on geographic location)\n\n What does it mean to work at Armstrong?\n\nIt means being immersed in a supportive culture that recognizes you as a key player in Armstrong\'s future. We are a large company with a local feel, where you will get to know and collaborate with leadership and your colleagues across the company.\n\nBy joining us, you\'ll have the opportunity to make the most of your potential. Alongside a competitive remuneration package, you will receive:\n\nA benefits package including: medical, dental, prescription drug, life insurance, 401k match, long-term disability coverage, vacation and sick time, product discount programs and many more.Personal development to grow your career with us based on your strengths and interests.A working culture that balances individual achievement with teamwork and collaboration. We draw on each other\'s strengths and allow for different work styles to build engagement and satisfaction to deliver results. \n\n\nAs a Data Scientist, you will leverage cutting-edge generative AI techniques to extract structured data from diverse document types. From there, you will build models that understand context, domain-specific jargon and generate documents. The output of your work will enable long-term strategic advantages for the company.\n\nEssential Duties and Responsibilities include the following. Other duties may be assigned.\n\nBuilding AI/ML features to evaluate document quality, account loyalty, market trends, etc.Constructing supervised learning datasetsWriting robust and testable codeDefining and overseeing regular updates to improve precision as the company’s challenges and data evolveCultivating strong collaborations with teammates and stakeholdersSharing technical solutions and product ideas with the team through design/code reviews and weekly meetings\n\n\nQualifications\n\nTo perform this job successfully, an individual must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skill, and/or ability required. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.\n\nExperience transforming natural language data into useful features using NLP techniques to feed classification algorithmsAbility to work with dashboarding and visualization software such as Tableau or Power BIKnowledge of software versioning control repositories such as GitHubAbility to translate data insights into actionable items and communicate findings in a simplistic wayExperience with generative AI would be a plus Enthusiasm for learning new things and going deep into detailed data analysisWorkflow flexibility, team player, and strong collaboration skills\n\n\nEducation And/or Experience\n\nBS in Computer Science, Statistics or Applied Mathematics or equivalent years of experience2+ years in software development, statistical modeling, and machine learning2+ years of experience in an analytical field using tools such as Python, R, SAS, MatlabFamiliarity with SQL or other querying languages is preferred\n\n\nWhy should you join Armstrong World Industries?\n\nArmstrong World Industries (AWI) is a leader in the design and manufacture of innovative commercial and residential ceiling, wall and suspension system solutions in the Americas. With approximately $1B in revenue, AWI has about 2,800 employees and a manufacturing network of fifteen facilities in North America.\n\nAt home, at work, in healthcare facilities, classrooms, stores, or restaurants, we offer interior solutions that help to enhance comfort, save time, improve building efficiency and overall performance, and create beautiful spaces.\n\nFor more than 150 years, we have built our business on trust and integrity. It set us apart then, and it sets us apart now, along with our ability to collaborate with and innovate for the people we\'re here to serve - our customers, our shareholders, our communities and our employees.\n\nWe are committed to developing new and sustainable ceiling solutions, with design and performance possibilities that make a positive difference in spaces where we live, work, learn, heal and play. It\'s an exciting, rewarding business to be in, and we\'re committed to continue to grow and prosper for the benefit of all of our stakeholders. We hope you join us.\n\nOur Sustainability Ambition\n\n"Bringing our Purpose to Life" - lead a transformation in the design and building of spaces fit for today and tomorrow.\n\nWe are committed to:\n\nEngaging a diverse, purpose-driven workforce;Transforming buildings from structures that shelter into structures that serve and preserve the health and well-being of people and planet;Pursuing sustainable, innovative solutions for spaces where we live, work, learn heal and play;Being a catalyst for change with all of our stakeholders; andMaking a positive difference in the environments and communities we impact.\n\n\nArmstrong is committed to engaging a diverse, purpose-driven workforce. As part of our dedication to diversity, AWI is committed to \n\nCome and build your future with us and apply today!',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
Evaluation
Metrics
Triplet
| Metric |
Value |
| cosine_accuracy |
1.0 |
Training Details
Training Dataset
parquet
Evaluation Dataset
parquet
Training Hyperparameters
Non-Default Hyperparameters
eval_strategy: steps
per_device_train_batch_size: 4
per_device_eval_batch_size: 4
gradient_accumulation_steps: 2
learning_rate: 1e-05
num_train_epochs: 2
warmup_ratio: 0.1
fp16: True
dataloader_num_workers: 2
load_best_model_at_end: True
batch_sampler: no_duplicates
All Hyperparameters
Click to expand
overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 4
per_device_eval_batch_size: 4
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 2
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 1e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 2
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 2
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}
Training Logs
| Epoch |
Step |
Training Loss |
Validation Loss |
all-nli-dev_cosine_accuracy |
| -1 |
-1 |
- |
- |
0.8812 |
| 0.4926 |
50 |
0.7066 |
0.3768 |
1.0 |
| 0.9852 |
100 |
0.3342 |
0.3119 |
1.0 |
| 1.4729 |
150 |
0.2092 |
0.2917 |
1.0 |
| 1.9655 |
200 |
0.209 |
0.2908 |
1.0 |
| -1 |
-1 |
- |
- |
1.0 |
- The bold row denotes the saved checkpoint.
Framework Versions
- Python: 3.12.11
- Sentence Transformers: 5.1.1
- Transformers: 4.57.0
- PyTorch: 2.8.0+cu126
- Accelerate: 1.10.1
- Datasets: 4.0.0
- Tokenizers: 0.22.1
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
MultipleNegativesRankingLoss
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}