afalaudn/gemma-embedding-ft3

SentenceTransformer based on unsloth/embeddinggemma-300m

This is a sentence-transformers model finetuned from unsloth/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: unsloth/embeddinggemma-300m
• Maximum Sequence Length: 768 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 768, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (4): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("afalaudn/gemma-embedding-ft3")
# Run inference
sentences = [
    "Retrieve the names of all patients who have a 'MedicalRecordStatusId' indicating 'Active' in their PatientOrganization record and have received an invoice from any organization in the current month.",
    "Primary reference table storing the comprehensive master profiles of all patients registered in the hospital system. It contains critical demographic data including Name, Medical Record Number (MrNo), Birth Date, Gender (SexId), Address, Contact Information (Phone, Email), and Identification details (National ID, Passport). It also tracks patient status (Active/Merged/Deceased) and links to external Payer/Insurance information. Use this table to identify unique patients, retrieve contact details for notifications, segment patient populations by demographics, or link clinical transactions to specific individuals. Note: This table represents the 'Golden Record' of a patient's identity; historical changes might be tracked in audit logs, but this table holds the current state.",
    'Primary reference table defining the standard set of religions and spiritual beliefs for patient demographic data. It contains codes and names for major religions such as Islam, Christian, Catholic, Hindu, and Buddhist. **Use this table to** interpret the `ReligionId` from patient master data to standardize demographic reporting or analyze patient spiritual care needs. **Note: This table lists the CATEGORIES of religion only, not the religious affiliation of individual patients.**',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.6662, -0.1269],
#         [ 0.6662,  1.0000,  0.1456],
#         [-0.1269,  0.1456,  1.0000]])

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.0328
cosine_accuracy@3	0.0627
cosine_accuracy@5	0.0836
cosine_accuracy@10	0.1313
cosine_precision@1	0.0328
cosine_precision@3	0.0209
cosine_precision@5	0.0167
cosine_precision@10	0.0131
cosine_recall@1	0.0328
cosine_recall@3	0.0627
cosine_recall@5	0.0836
cosine_recall@10	0.1313
cosine_ndcg@10	0.0735
cosine_mrr@10	0.056
cosine_map@100	0.063

Training Details

Training Dataset

Unnamed Dataset

• Size: 1,340 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 10 tokens mean: 35.94 tokens max: 82 tokens	min: 63 tokens mean: 129.79 tokens max: 171 tokens	min: 63 tokens mean: 112.02 tokens max: 175 tokens

• Samples:

  anchor	positive	negative
  What is the total number of patient admissions that resulted in a 'Recovered' discharge condition for each organization in 2023?	Operational transaction table recording every patient registration and visit event at the hospital. This table consolidates patient demographics, visit types (Inpatient, Outpatient, Emergency), primary and referral doctors, payer/insurance eligibility, and critical timelines (Admission and Discharge dates). Use this table to calculate patient census, Average Length of Stay (ALOS), track patient flow, or analyze admission volume by doctor or department. Note: This table focuses on administrative registration and billing initiation; it does not contain detailed clinical notes, specific lab results, or medication prescriptions. When analyzing patient administrative inflow and outflow data, this table is the primary and essential source for all patient visit metrics.	This reference table defines the high-level administrative outcomes of a patient's discharge process. It includes standard codes, English names, and Indonesian translations (LocalName) such as 'Discharged' (Pulang) and 'Cancelled' (Batal). Use this table to categorize the final resolution of a hospital visit, determining if a patient successfully left care or if the discharge process was voided. Note: This table tracks the administrative status of the discharge event itself, not the clinical medical condition (e.g., improved, cured) or the specific reason for discharge.
  Retrieve the AdmissionNo and InvoiceNo for all admissions where the patient was not previously registered (Admission.IsPatientRegistered = 0) and the invoice was issued by an organization with an IsActive status of 'true'.	Operational transaction table (Financial Log) recording the header-level details of patient invoices and billing events. This table captures the financial breakdown of a visit, distinguishing between Patient responsibility (Out-of-pocket) and Payer responsibility (Insurance/Corporate Coverage), including Gross Amounts, Discounts, Taxes, and Net Payable values. Use this table to analyze hospital revenue streams, track Accounts Receivable (AR), monitor billing cancellations, or calculate the financial yield per admission. Note: This is the Invoice HEADER table containing total values; it does not typically list the specific individual line items (drugs, labs, services) charged within the bill. For any financial analysis related to hospital revenue, Payments, Accounts Receivable (AR), billing breakdowns, or insurance claims, this invoice header table is the definitive starting point.	Operational transaction table recording formal requests to retrieve physical medical record folders from the Medical Records Department (MRD). It tracks the details of the request, including the specific patient file (PatientOrganizationId), the requesting doctor (DoctorUserId), the intended destination (FolderDestinationTypeId), and the urgency or type of request (FolderRequestTypeId). Use this table to measure MRD service levels, track the volume of physical file retrievals, or audit the reasons for accessing physical records. Note: This tracks the request for a physical object, NOT the digital access to the EMR or the actual content of the medical record.
  What is the average age of patients with 'Active' status who had an 'Inpatient' admission in 2024, compared to those with 'Inactive' status?	Operational transaction table recording every patient registration and visit event at the hospital. This table consolidates patient demographics, visit types (Inpatient, Outpatient, Emergency), primary and referral doctors, payer/insurance eligibility, and critical timelines (Admission and Discharge dates). Use this table to calculate patient census, Average Length of Stay (ALOS), track patient flow, or analyze admission volume by doctor or department. Note: This table focuses on administrative registration and billing initiation; it does not contain detailed clinical notes, specific lab results, or medication prescriptions. When analyzing patient administrative inflow and outflow data, this table is the primary and essential source for all patient visit metrics.	Operational transaction table recording the official event of a patient leaving the hospital (Discharge). It captures the precise discharge timestamp, the patient's condition upon exit (e.g., Recovered, Improved), and the type of discharge (e.g., Medical Consent, Transfer) linked to their Admission. Use this table to calculate Length of Stay (LOS), analyze clinical outcomes, or track bed turnover rates. Note: This table signifies the physical or administrative end of a visit; it does NOT contain the final invoice amount, though it triggers the billing closure process.

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.COSINE",
      "triplet_margin": 1
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 335 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 335 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 10 tokens mean: 35.25 tokens max: 77 tokens	min: 63 tokens mean: 128.23 tokens max: 171 tokens	min: 63 tokens mean: 111.21 tokens max: 171 tokens

• Samples:

  anchor	positive	negative
  Retrieve the admission number, invoice number, and admission type name for all admissions that occurred in January 2023 and had an invoice issued within 7 days of the admission date.	Primary reference table defining the high-level classification of patient visits and hospital service lines. Contains standard categories including Inpatient (Hospitalization), Outpatient (Clinical visits), Emergency (ER), and Health Checkups (MCU). Use this table to group patient volume by service type, filter admission logs, or analyze revenue streams by visit category. Note: This is a static lookup list defining the 'Types' of visits; it does not contain actual patient visit transaction records.	Operational transaction table that records the detailed line items of an inventory stock take event. It captures the specific ItemId being counted, the quantity recorded in the system (SystemStockQuantity), the actual PhysicalStockQuantity counted, and the resulting VarianceStockQuantity. For batch-tracked items, it also includes BatchNo and ExpiryDate. Use this table to analyze inventory discrepancies, identify expired or expiring stock during a count, or audit the accuracy of stock records for a specific item. Note: This table contains the detailed results of a specific stock count event; it does NOT provide the real-time inventory balance of an item.
  What is the total number of invoices and the total value of these invoices issued in Q1 2024 for female patients whose current status is 'Active'?	Primary reference table defining the validity and lifecycle status of a patient's master record. Contains status codes such as 'Active', 'Inactive', and 'Deceased' to classify patient accounts. Use this table to look up status meanings or filter patient queries based on their account state (e.g., excluding deceased patients from active lists). Note: This is a lookup dictionary for status codes, NOT a transaction log of status changes for specific patients.	Operational snapshot table that holds the single, current financial state of a patient at a specific hospital unit. Unlike a transaction log, this table does not track history; it contains only the final calculated totals: BalanceAmount (Total AR/Debt outstanding) and DepositBalanceAmount (Total Advance Payment available). Use this table for quick validation of whether a patient can be discharged, checking total debt before billing, or viewing available deposit funds. Concept: Think of this as the 'ATM Screen Balance' showing only the final amount available right now.
  Tampilkan data Master Data Hospital Rooms (Room)	Primary reference table defining individual patient rooms within a hospital ward. It contains specific room codes and names, linking each room to its parent Ward (WardId) and hospital branch (OrganizationId), representing the most granular level of inpatient location master data. Use this table to map RoomId from bed management or admission transactions to a specific room number, analyze room availability, or manage hospital accommodation resources. Note: This table lists the physical room definitions only, not the individual beds within a room or the real-time patient occupancy status.	Operational transaction table recording the historical log of bed movements and transfers within the hospital organization. This table captures the exact timestamp of transfer events and identifies the specific Bed asset involved. Use this table to track the history of bed locations, audit bed usage timelines, or monitor the frequency of bed transfers. Note: This table focuses on the physical Bed entity's movement timeline; based on the provided columns, it does not explicitly contain Patient or Admission IDs.

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.COSINE",
      "triplet_margin": 1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 4
• learning_rate: 2e-05
• num_train_epochs: 5
• warmup_ratio: 0.1
• fp16: True
• load_best_model_at_end: True
• prompts: {'anchor': ' ', 'positive': '', 'negative': ''}
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 4
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: {'anchor': ' ', 'positive': '', 'negative': ''}
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	cosine_ndcg@10
-1	-1	-	-	0.0698
0.2381	5	0.7215	0.3602	0.0706
0.4762	10	0.3287	0.2451	0.0765
0.7143	15	0.2145	0.1896	0.0652
0.9524	20	0.2181	0.2286	0.0535
1.1905	25	0.2774	0.1610	0.0694
1.4286	30	0.1948	0.1818	0.0733
-1	-1	-	-	0.0733
0.2381	5	0.1696	0.1475	0.0705
0.4762	10	0.1657	0.2067	0.0661
0.7143	15	0.1346	0.1991	0.0635
0.9524	20	0.1197	0.1466	0.0473
1.1905	25	0.2657	0.1380	0.0544
1.4286	30	0.1706	0.1906	0.0719
1.6667	35	0.1686	0.1746	0.0741
1.9048	40	0.1217	0.1577	0.0669
2.1429	45	0.1191	0.1355	0.0606
2.3810	50	0.108	0.1362	0.0615
2.6190	55	0.1065	0.1595	0.0647
2.8571	60	0.1314	0.1126	0.0667
3.0952	65	0.072	0.0934	0.0589
3.3333	70	0.0868	0.0977	0.0655
3.5714	75	0.0719	0.1310	0.0496
3.8095	80	0.1184	0.1388	0.0681
4.0476	85	0.0997	0.1132	0.0656
4.2857	90	0.0659	0.1029	0.0724
4.5238	95	0.0554	0.1018	0.0707
4.7619	100	0.0729	0.0989	0.0659
5.0	105	0.0422	0.0971	0.0735
-1	-1	-	-	0.0735

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.2.2
• Transformers: 4.57.6
• PyTorch: 2.10.0+cu128
• Accelerate: 1.12.0
• Datasets: 4.3.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}