Anup77Jindal/embedding-gemma-300m-finetuned-job-data-v2

SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m on the semantic_search and reranking datasets. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: google/embeddinggemma-300m
• Maximum Sequence Length: 2048 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Datasets:
  • semantic_search
  • reranking

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 2048, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (4): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Anup77Jindal/embedding-gemma-300m-finetuned-job-data-v2")
# Run inference
queries = [
    "Mid-level backend engineer with 4+ years of experience in Java and Kafka, remote",
]
documents = [
    'Backend Engineer at StreamFlow with 5 years of experience. Proficient in building event-driven systems using Java, Spring Boot, and Apache Kafka.',
    'Mid-level backend engineer with 4+ years of experience in Java and Kafka, remote',
    'Backend Engineer at StreamFlow with 5 years of experience. Proficient in building event-driven systems using Java, Spring Boot, and Apache Kafka.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.7730, 0.7873, 0.7730]])

Evaluation

Metrics

Information Retrieval

• Dataset: sts_validation
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.5
cosine_accuracy@3	0.8108
cosine_accuracy@5	0.8514
cosine_accuracy@10	0.9189
cosine_precision@1	0.5
cosine_precision@3	0.2703
cosine_precision@5	0.1703
cosine_precision@10	0.0919
cosine_recall@1	0.5
cosine_recall@3	0.8108
cosine_recall@5	0.8514
cosine_recall@10	0.9189
cosine_ndcg@10	0.7318
cosine_mrr@10	0.6699
cosine_map@100	0.6735

Binary Classification

• Dataset: reranking_validation
• Evaluated with BinaryClassificationEvaluator

Metric	Value
cosine_accuracy	0.8333
cosine_accuracy_threshold	0.9694
cosine_f1	0.8235
cosine_f1_threshold	0.8091
cosine_precision	0.875
cosine_recall	0.7778
cosine_ap	0.8785
cosine_mcc	0.6708

Training Details

Training Datasets

semantic_search

• Dataset: semantic_search
• Size: 293 training samples
• Columns: query, positive, sentence_0, and sentence_1
• Approximate statistics based on the first 293 samples:

  	query	positive	sentence_0	sentence_1
  type	string	string	string	string
  details	min: 5 tokens mean: 13.19 tokens max: 22 tokens	min: 21 tokens mean: 30.5 tokens max: 41 tokens	min: 5 tokens mean: 13.19 tokens max: 22 tokens	min: 21 tokens mean: 30.5 tokens max: 41 tokens

• Samples:

  query	positive	sentence_0	sentence_1
  A machine learning engineer with experience in recommender systems	ML Engineer at Recommendo. Specializes in building and deploying various types of recommender systems, from collaborative filtering to deep learning-based models.	A machine learning engineer with experience in recommender systems	ML Engineer at Recommendo. Specializes in building and deploying various types of recommender systems, from collaborative filtering to deep learning-based models.
  Database reliability engineer (DBRE)	Database Reliability Engineer at DB-Uptime. Focuses on the reliability, performance, and automation of database infrastructure, applying SRE principles to data stores.	Database reliability engineer (DBRE)	Database Reliability Engineer at DB-Uptime. Focuses on the reliability, performance, and automation of database infrastructure, applying SRE principles to data stores.
  Senior Data Scientist in Bengaluru with 6+ years experience in NLP and Transformers	Lead Data Scientist at Text Analytics India with 7 years of experience. Expert in Natural Language Processing, building models with Transformers and BERT for text classification.	Senior Data Scientist in Bengaluru with 6+ years experience in NLP and Transformers	Lead Data Scientist at Text Analytics India with 7 years of experience. Expert in Natural Language Processing, building models with Transformers and BERT for text classification.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

reranking

• Dataset: reranking
• Size: 144 training samples
• Columns: query, passage, label, sentence_0, and sentence_1
• Approximate statistics based on the first 144 samples:

  	query	passage	label	sentence_0	sentence_1
  type	string	string	int	string	string
  details	min: 8 tokens mean: 12.68 tokens max: 21 tokens	min: 15 tokens mean: 25.78 tokens max: 37 tokens	0: ~50.00% 1: ~50.00%	min: 8 tokens mean: 12.68 tokens max: 21 tokens	min: 15 tokens mean: 25.78 tokens max: 37 tokens

• Samples:

  query	passage	label	sentence_0	sentence_1
  Senior Machine Learning Engineer with experience in MLOps	Senior ML Engineer. Proficient in building and training models for computer vision tasks.	0	Senior Machine Learning Engineer with experience in MLOps	Senior ML Engineer. Proficient in building and training models for computer vision tasks.
  Business Intelligence (BI) Developer proficient in Power BI	BI Developer at VizPro. Specializes in creating insightful reports and interactive dashboards using Tableau and SQL.	0	Business Intelligence (BI) Developer proficient in Power BI	BI Developer at VizPro. Specializes in creating insightful reports and interactive dashboards using Tableau and SQL.
  Senior mobile developer with experience in both iOS (Swift) and Android (Kotlin)	Senior Mobile Engineer with 8 years of experience. Proficient in developing cross-platform applications using Xamarin and C#.	0	Senior mobile developer with experience in both iOS (Swift) and Android (Kotlin)	Senior Mobile Engineer with 8 years of experience. Proficient in developing cross-platform applications using Xamarin and C#.

• Loss: CosineSimilarityLoss with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss"
  }
  

Evaluation Datasets

semantic_search

• Dataset: semantic_search
• Size: 74 evaluation samples
• Columns: query, positive, sentence_0, and sentence_1
• Approximate statistics based on the first 74 samples:

  	query	positive	sentence_0	sentence_1
  type	string	string	string	string
  details	min: 5 tokens mean: 13.81 tokens max: 24 tokens	min: 23 tokens mean: 30.05 tokens max: 37 tokens	min: 5 tokens mean: 13.81 tokens max: 24 tokens	min: 23 tokens mean: 30.05 tokens max: 37 tokens

• Samples:

  query	positive	sentence_0	sentence_1
  Mid-level product owner with experience in a Scrum team	Product Owner at AgileProducts. 4 years of experience working within a Scrum team, managing the product backlog, and defining acceptance criteria for user stories.	Mid-level product owner with experience in a Scrum team	Product Owner at AgileProducts. 4 years of experience working within a Scrum team, managing the product backlog, and defining acceptance criteria for user stories.
  Looking for a remote cybersecurity expert in penetration testing	Senior Security Consultant at RedTeam Experts. OSCP certified professional with a specialization in network and web application penetration testing.	Looking for a remote cybersecurity expert in penetration testing	Senior Security Consultant at RedTeam Experts. OSCP certified professional with a specialization in network and web application penetration testing.
  Looking for a remote C# developer with experience in .NET Core	Principal Software Engineer at Microsoft, a specialist in C# and .NET Core. Led the development of a major microservices platform.	Looking for a remote C# developer with experience in .NET Core	Principal Software Engineer at Microsoft, a specialist in C# and .NET Core. Led the development of a major microservices platform.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

reranking

• Dataset: reranking
• Size: 36 evaluation samples
• Columns: query, passage, label, sentence_0, and sentence_1
• Approximate statistics based on the first 36 samples:

  	query	passage	label	sentence_0	sentence_1
  type	string	string	int	string	string
  details	min: 9 tokens mean: 12.83 tokens max: 17 tokens	min: 15 tokens mean: 25.81 tokens max: 33 tokens	0: ~50.00% 1: ~50.00%	min: 9 tokens mean: 12.83 tokens max: 17 tokens	min: 15 tokens mean: 25.81 tokens max: 33 tokens

• Samples:

  query	passage	label	sentence_0	sentence_1
  Looking for a C++ developer for high-frequency trading systems	Software Engineer at TradeFast. Develops backend web services using high-level languages like Java and Python.	0	Looking for a C++ developer for high-frequency trading systems	Software Engineer at TradeFast. Develops backend web services using high-level languages like Java and Python.
  Azure DevOps Engineer with 5+ years experience	Senior DevOps Engineer at AzureFlow. Proficient in managing CI/CD pipelines, repos, and boards using the Azure DevOps suite.	1	Azure DevOps Engineer with 5+ years experience	Senior DevOps Engineer at AzureFlow. Proficient in managing CI/CD pipelines, repos, and boards using the Azure DevOps suite.
  Seeking a senior Python developer in Bengaluru with expertise in Django and REST APIs	Lead Python Developer at TechCorp with 8+ years of experience. Proficient in building scalable backend systems using Django REST Framework and PostgreSQL.	1	Seeking a senior Python developer in Bengaluru with expertise in Django and REST APIs	Lead Python Developer at TechCorp with 8+ years of experience. Proficient in building scalable backend systems using Django REST Framework and PostgreSQL.

• Loss: CosineSimilarityLoss with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• learning_rate: 2e-05
• num_train_epochs: 10
• warmup_ratio: 0.1
• remove_unused_columns: False
• load_best_model_at_end: True
• prompts: {'semantic_search': 'task: sentence similarity | query: ', 'reranking': 'task: search result | query: '}

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 8
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 10
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: False
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: {'semantic_search': 'task: sentence similarity | query: ', 'reranking': 'task: search result | query: '}
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	semantic search loss	reranking loss	sts_validation_cosine_ndcg@10	reranking_validation_cosine_ap
0.4545	25	1.7059	-	-	-	-
0.9091	50	1.0157	-	-	0.8183	0.9120
0.4545	25	0.9967	-	-	-	-
0.9091	50	0.8205	1.4993	0.1305	0.7571	0.8962
1.3636	75	0.9117	-	-	-	-
1.8182	100	0.8707	1.4316	0.1271	0.7977	0.9131
2.2727	125	0.8071	-	-	-	-
2.7273	150	0.8184	1.3847	0.1235	0.7600	0.9341
3.1818	175	0.8655	-	-	-	-
3.6364	200	0.9919	1.4504	0.1019	0.7323	0.953
4.0909	225	0.8367	-	-	-	-
4.5455	250	0.8866	1.6648	0.1411	0.7233	0.8965
5.0	275	0.6635	-	-	-	-
5.4545	300	0.7921	1.9412	0.1234	0.7104	0.8885
5.9091	325	0.8513	-	-	-	-
6.3636	350	0.7408	2.0895	0.1802	0.6730	0.8482
6.8182	375	0.9116	-	-	-	-
7.2727	400	0.6594	1.9148	0.1356	0.7318	0.8785

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.2
• Sentence Transformers: 5.1.1
• Transformers: 4.56.2
• PyTorch: 2.8.0+cpu
• Accelerate: 1.10.1
• Datasets: 4.1.1
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}