SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: google/embeddinggemma-300m
• Maximum Sequence Length: 2048 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 2048, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (4): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    "MRCCC ada dimana?",
]
documents = [
    'Lokasinya di Jl. Garnisun 1 No.2-3, RT.5/RW.4, Karet Semanggi, Kecamatan Setiabudi, Kota Jakarta Selatan, Daerah Khusus Ibukota Jakarta 12930',
    'Nama lain Siloam Medan Petisah adalah Siloam Hospitals Medan atau SHMD',
    'CEO (Chief Executive Officer) Siloam Hospitals adalah Caroline Riady',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.6492, 0.2818, 0.7452]])

Evaluation

Metrics

Triplet

• Dataset: ai-faq-validation
• Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.9768

Training Details

Training Dataset

Unnamed Dataset

• Size: 2,113 training samples
• Columns: query, answer_positive, and answer_negative
• Approximate statistics based on the first 1000 samples:

  	query	answer_positive	answer_negative
  type	string	string	string
  details	min: 5 tokens mean: 10.81 tokens max: 20 tokens	min: 9 tokens mean: 36.06 tokens max: 409 tokens	min: 9 tokens mean: 29.68 tokens max: 149 tokens

• Samples:

  query	answer_positive	answer_negative
  Asuransi apa aja yg kerja sama sama Siloam?	Siloam Hospitals Group memastikan untuk terus memfasilitasi jaringan kerjasama termasuk dengan rekanan asuransi dalam negeri atau pun luar negeri. Untuk lebih detail, pastikan Anda melakukan konfirmasi kepada pihak asuransi mengenai jaringan dan ketentuan yang berlaku pada polis penjaminan Anda. Anda juga dapat menghubungi Rumah Sakit Siloam yang Anda tuju untuk menanyakan detail terkait hal ini.	Lokasinya di Jl. Magelang - Kopeng km 08 Tegalrejo Kabupaten Magelang 56192
  Gawat darurat, perlu rujukan dari faskes pertama ga?	Tidak, kondisi gawat darurat tidak membutuhkan Surat Rujukan dari Puskesmas/Klinik. Pasien dapat langsung mengunjungi instalasi gawat darurat selama dalam kondisi gawat darurat.	Nama lain Siloam Paal Dua adalah Siloam Hospitals Paal Dua atau SHPD
  apakah boleh merokok di siloam	Untuk memastikan lingkungan yang sehat bagi pasien kami, kami melarang aksi merokok di area rumah sakit.	Lokasinya di Jl. Yos Sudarso No.38-40, Paal Dua, Kec. Paal Dua, Kota Manado, Sulawesi Utara

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 604 evaluation samples
• Columns: query, answer_positive, and answer_negative
• Approximate statistics based on the first 604 samples:

  	query	answer_positive	answer_negative
  type	string	string	string
  details	min: 5 tokens mean: 10.78 tokens max: 22 tokens	min: 9 tokens mean: 34.21 tokens max: 409 tokens	min: 13 tokens mean: 29.36 tokens max: 149 tokens

• Samples:

  query	answer_positive	answer_negative
  presiden direktur siloam hospitals siapa	Presiden Direktur Siloam Hospitals adalah David Utama	Lokasinya di Jl. Pahlawan No.60, RT.006/RW.004, Aren Jaya, Kec. Bekasi Tim., Kota Bks, Jawa Barat 17111
  mysiloam itu apa sih gunanya	Aplikasi MySiloam adalah aplikasi mobile untuk membantu pengguna menikmati kemudahan layanan kesehatan yang disediakan oleh Rumah Sakit Siloam. Beberapa fitur yang sudah membantu banyak orang adalah, pembuatan janji temu dokter, pemesanan tes dan layanan kesehatan, emergency, dan ambulance call.	Lokasinya di Jl. Kelapa Dua Raya No.1001, Klp. Dua, Kec. Klp. Dua, Kabupaten Tangerang, Banten 15810
  siapa nama presiden komisaris siloam	Presiden Komisaris Siloam Hospitals adalah Yasonna Laoly	Saat ini, metode pembayaran yang kami terima adalah kartu kredit atau debit (Visa dan Mastercard), OVO, Virtual Account, dan Bank Transfer.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 14
• per_device_eval_batch_size: 14
• learning_rate: 0.0002
• num_train_epochs: 1
• warmup_ratio: 0.1
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 14
• per_device_eval_batch_size: 14
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 0.0002
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	ai-faq-validation_cosine_accuracy
-1	-1	-	-	0.6440
0.6623	100	1.4291	1.1531	0.9768

Framework Versions

• Python: 3.11.6
• Sentence Transformers: 5.1.2
• Transformers: 4.55.0
• PyTorch: 2.9.0+cu128
• Accelerate: 1.11.0
• Datasets: 4.3.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}