afalaudn/gemma-embedding-ft2

SentenceTransformer

This model was finetuned with Unsloth.

based on unsloth/embeddinggemma-300m

This is a sentence-transformers model finetuned from unsloth/embeddinggemma-300m on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: unsloth/embeddinggemma-300m
• Maximum Sequence Length: 768 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • json

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 768, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (4): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'master data payer group',
    "Primary reference table containing the master list of all external organizations responsible for patient payment guarantees. This includes Insurance Companies, Corporate Clients/Employers, and Government Health Schemes (e.g., BPJS, Jamkesda). The table stores Payer details such as Legal Name, Address, Contact Information, and specific Payer Group classifications. Use this table to link patient visits to their financial guarantors, generate invoices for corporate clients, or analyze revenue contribution by payer. Note: This table defines the 'Who Pays' entity; specific policy terms or benefit limits are typically stored in separate configuration tables.",
    'Strategic reference table linking Payers to specific Hospital Organizations (Units/Branches). This table manages the contractual relationships between insurance providers/corporate clients and individual hospital sites. It stores Contract Numbers, Validity Periods (Start/End Dates), Contract Status, and site-specific contact details. Use this table to validate insurance acceptance at a specific hospital branch, track contract expiration dates, or manage site-specific payer agreements. Note: This table enables the many-to-many relationship between Payers (Global) and Organizations (Local Sites).',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.9426, -0.8527],
#         [ 0.9426,  1.0000, -0.8639],
#         [-0.8527, -0.8639,  1.0000]])

Evaluation

Metrics

Information Retrieval

• Dataset: his-retrieval-eval
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.0016
cosine_accuracy@3	0.0049
cosine_accuracy@5	0.0073
cosine_accuracy@10	0.0122
cosine_precision@1	0.0016
cosine_precision@3	0.0016
cosine_precision@5	0.0015
cosine_precision@10	0.0012
cosine_recall@1	0.0016
cosine_recall@3	0.0049
cosine_recall@5	0.0073
cosine_recall@10	0.0122
cosine_ndcg@10	0.006
cosine_mrr@10	0.0041
cosine_map@100	0.0058

Training Details

Training Dataset

json

• Dataset: json
• Size: 4,927 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 3 tokens mean: 12.32 tokens max: 76 tokens	min: 63 tokens mean: 128.13 tokens max: 171 tokens	min: 63 tokens mean: 108.19 tokens max: 171 tokens

• Samples:

  anchor	positive	negative
  master patient	Primary reference table defining the lifecycle stages of a patient admission event. It categorizes visits into states such as 'Active' (currently in hospital), 'Discharged' (left hospital), 'Invoiced' (bill generated), or 'Cancelled'. Use this table to interpret AdmissionStatusId in transaction tables to filter visits by their current operational state (e.g., calculating current census vs. historical discharges). Note: This is a static lookup table for status definitions, NOT a transaction log of patient movements.	Operational transaction table recording unstructured free-text medical notes and preliminary clinical remarks associated with a patient admission. It captures initial diagnosis impressions, symptoms, or observation notes (e.g., 'Asthma', 'Observation Febris') entered during the admission process. Use this table to retrieve qualitative clinical context for a visit or search for specific medical conditions mentioned in preliminary notes. Note: This table contains raw free-text descriptions, NOT structured ICD-10 diagnosis codes used for billing.
  transaction ar invoice	Primary reference table containing the master list of all external organizations responsible for patient payment guarantees. This includes Insurance Companies, Corporate Clients/Employers, and Government Health Schemes (e.g., BPJS, Jamkesda). The table stores Payer details such as Legal Name, Address, Contact Information, and specific Payer Group classifications. Use this table to link patient visits to their financial guarantors, generate invoices for corporate clients, or analyze revenue contribution by payer. Note: This table defines the 'Who Pays' entity; specific policy terms or benefit limits are typically stored in separate configuration tables.	Operational transaction table recording unstructured free-text medical notes and preliminary clinical remarks associated with a patient admission. It captures initial diagnosis impressions, symptoms, or observation notes (e.g., 'Asthma', 'Observation Febris') entered during the admission process. Use this table to retrieve qualitative clinical context for a visit or search for specific medical conditions mentioned in preliminary notes. Note: This table contains raw free-text descriptions, NOT structured ICD-10 diagnosis codes used for billing.
  admission date	Primary reference table defining the high-level classification of patient visits and hospital service lines. Contains standard categories including Inpatient (Hospitalization), Outpatient (Clinical visits), Emergency (ER), and Health Checkups (MCU). Use this table to group patient volume by service type, filter admission logs, or analyze revenue streams by visit category. Note: This is a static lookup list defining the 'Types' of visits; it does not contain actual patient visit transaction records.	Operational transaction table recording unstructured free-text medical notes and preliminary clinical remarks associated with a patient admission. It captures initial diagnosis impressions, symptoms, or observation notes (e.g., 'Asthma', 'Observation Febris') entered during the admission process. Use this table to retrieve qualitative clinical context for a visit or search for specific medical conditions mentioned in preliminary notes. Note: This table contains raw free-text descriptions, NOT structured ICD-10 diagnosis codes used for billing.

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.COSINE",
      "triplet_margin": 0.5
  }
  

Evaluation Dataset

json

• Dataset: json
• Size: 1,232 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 3 tokens mean: 13.51 tokens max: 76 tokens	min: 63 tokens mean: 127.79 tokens max: 171 tokens	min: 63 tokens mean: 107.3 tokens max: 171 tokens

• Samples:

  anchor	positive	negative
  transaction ar item	Operational transaction table recording every patient registration and visit event at the hospital. This table consolidates patient demographics, visit types (Inpatient, Outpatient, Emergency), primary and referral doctors, payer/insurance eligibility, and critical timelines (Admission and Discharge dates). Use this table to calculate patient census, Average Length of Stay (ALOS), track patient flow, or analyze admission volume by doctor or department. Note: This table focuses on administrative registration and billing initiation; it does not contain detailed clinical notes, specific lab results, or medication prescriptions. When analyzing patient administrative inflow and outflow data, this table is the primary and essential source for all patient visit metrics.	Operational transaction table recording individual line items within patient invoices (Accounts Receivable). It captures granular billing details including specific items sold (drugs, services), quantities, unit prices, discounts, tax calculations, and the financial split between Patient and Payer (Insurance/Guarantor). It also tracks revenue allocation (Hospital vs. Doctor portion). Use this table to generate detailed patient bills, audit revenue streams per item, calculate doctor performance fees, or analyze discount utilization. Note: This table contains financial billing data per item, NOT the clinical medical results or the master list of available services.
  patient demographic country	Operational transaction table (Financial Log) recording the header-level details of patient invoices and billing events. This table captures the financial breakdown of a visit, distinguishing between Patient responsibility (Out-of-pocket) and Payer responsibility (Insurance/Corporate Coverage), including Gross Amounts, Discounts, Taxes, and Net Payable values. Use this table to analyze hospital revenue streams, track Accounts Receivable (AR), monitor billing cancellations, or calculate the financial yield per admission. Note: This is the Invoice HEADER table containing total values; it does not typically list the specific individual line items (drugs, labs, services) charged within the bill. For any financial analysis related to hospital revenue, Payments, Accounts Receivable (AR), billing breakdowns, or insurance claims, this invoice header table is the definitive starting point.	Operational transaction table recording individual line items within patient invoices (Accounts Receivable). It captures granular billing details including specific items sold (drugs, services), quantities, unit prices, discounts, tax calculations, and the financial split between Patient and Payer (Insurance/Guarantor). It also tracks revenue allocation (Hospital vs. Doctor portion). Use this table to generate detailed patient bills, audit revenue streams per item, calculate doctor performance fees, or analyze discount utilization. Note: This table contains financial billing data per item, NOT the clinical medical results or the master list of available services.
  BPJS Kesehatan	Primary reference table that classifies financial guarantors into high-level categories such as 'Government' programs, 'Corporate' accounts, 'Insurance' companies, and Third Party Administrators ('TPA'). Use this table to group and analyze patient revenue streams by the type of financial coverage or to interpret the PayerTypeId in the main Payer master data table. Note: This table defines the broad categories of payers only, not the specific insurance companies or corporate entities themselves (which are listed in the Payer table).	Operational transaction table that records the movement of inventory items from one storage location (store) to another within the hospital network. It captures the header-level details of each transfer, including the transaction number, date, the originating store, and the receiving store. Use this table to track the flow of goods, monitor stock levels across different warehouses or departments, and audit inventory movements for logistics and supply chain management. Note: This table contains only the header information for the transfer event; it does NOT list the specific items or quantities transferred. Join with the Transfer Detail table for item-level information.

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.COSINE",
      "triplet_margin": 0.5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• gradient_accumulation_steps: 2
• learning_rate: 2e-05
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• prompts: {'anchor': ' ', 'positive': '', 'negative': ''}
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 2
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: {'anchor': ' ', 'positive': '', 'negative': ''}
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	his-retrieval-eval_cosine_ndcg@10
-1	-1	-	-	0.0034
0.1299	5	0.4772	-	-
0.2597	10	0.2436	0.0812	0.0021
0.3896	15	0.1075	-	-
0.5195	20	0.0794	0.0613	0.0036
0.6494	25	0.089	-	-
0.7792	30	0.1045	0.0248	0.0030
0.9091	35	0.0629	-	-
1.0260	40	0.0552	0.0298	0.0025
1.1558	45	0.0597	-	-
1.2857	50	0.0684	0.0236	0.0017
1.4156	55	0.0629	-	-
1.5455	60	0.0438	0.0213	0.0040
1.6753	65	0.0504	-	-
1.8052	70	0.0501	0.0237	0.0010
1.9351	75	0.0443	-	-
2.0519	80	0.0202	0.0232	0.0052
2.1818	85	0.0414	-	-
2.3117	90	0.0497	0.0233	0.0033
2.4416	95	0.0367	-	-
2.5714	100	0.0491	0.0232	0.0054
2.7013	105	0.0262	-	-
2.8312	110	0.0222	0.0232	0.0045
2.9610	115	0.0225	-	-
-1	-1	-	-	0.0060

Framework Versions

• Python: 3.11.6
• Sentence Transformers: 5.2.2
• Transformers: 4.56.2
• PyTorch: 2.10.0+cu128
• Accelerate: 1.12.0
• Datasets: 4.3.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}