SentenceTransformer based on google/embeddinggemma-300m

This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: google/embeddinggemma-300m
Maximum Sequence Length: 256 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ahmedHamdi/IR-pt-en-gemma")
# Run inference
sentences = [
    "Tom Brand (Kevin Spacey) is a business tycoon who has separated his daughter Rebecca (Malina Weissman) and his wife Lara (Jennifer Garner). In a rush to get a last-minute gift for his daughter's birthday, he goes to a mysterious pet shop run by Felix Grant (Christopher Walken) and buys a cat. On the way home, Tom is involved in an accident and becomes trapped inside the cat's body. Felix tells him that he must reconcile with his family within a week or he will remain a cat forever.",
    "Tom Brand is a major business tycoon in New York City whose workaholic attitude ruined his first marriage with Madison, and his adult son David now works for him and strives for his approval. Tom and David now live with their second wife and stepmother Lara, who is a little more tolerant of the fact that Tom is never at home, and their daughter and half-sister Rebecca. Tom’s eponymous company FireBrand is nearing completion on its greatest achievement to date: the tallest skyscraper in the northern hemisphere that will be the new headquarters. In the opening scene, he skydives out of a plane and lands on the top of the new skyscraper, while David remains in the plane, refusing to jump. Rebecca's 11th birthday comes up and she has always wanted a cat, but Tom has always refused as he hates cats. Not wanting to disappoint her, he rushes to get a last-minute present for his daughter's birthday, but as he is distracted, the GPS redirects him to a mysterious pet store called Purrkins Pet Shop, brimming with odd and exotic cats. The store's eccentric owner Felix Perkins tells him he does not pick the cat, the cat picks him. The tomcat that picks him is called Mr. Fuzzypants, who, Perkins says, has used up seven of his nine lives. Tom buys the cat. On his way home for the party, Tom meets with Ian Cox, one of the top managers of the company in charge of the new building, and learns that another building in Chicago will be taller. They have an argument while on the roof that ends with Tom firing Ian, but lightning strikes the antenna. Tom is blown off the building along with the cat, and without any effort from Ian to save him, he plummets off the side of the skyscraper, but his leg gets snagged on some stray equipment and gets flung back inside through a window as he passes out. When he wakes up, he realizes that his human body is in the hospital in a state of coma, and his consciousness is trapped inside the cat's body. Tom is visited by Felix, who is able to talk to him and knows what happened. Felix warns Tom to reevaluate his priorities, connect with his family, and avoid past mistakes within one week or else remain as a cat forever. When Lara and Rebecca take Mr. Fuzzypants home, he acts in an odd and stubborn way to try to convince his wife and daughter that he is actually Tom. This only drives them mad and he slowly comes to see how much he has ignored his family. He suspects that Lara is cheating on him with model Josh Myers and they had been looking at houses together as a prelude to divorcing him. This motivates him to try to make her happy. He also learns that Ian is trying to make the company public with the help of the board of directors to take power from Tom, even though David is trying to stop him. Ian fires David from the company and plans to announce that the company will be sold at the party for the new tower opening. Eventually, Rebecca realizes that Mr. Fuzzypants is really her father. Meanwhile, Tom's body is in crisis at the hospital and Lara, David, and Rebecca all go there. Rebecca has the cat hidden in her backpack. In a moment alone, David apologizes to Tom for failing to save the company and takes Tom's ID badge, implying he will commit suicide at the tower. At the hospital, Lara and Dr. Cole plan for Lara to sign a DNR, which is a legal form, indicating that if Tom's heart stops beating, they won't save him. Rebecca calls to the cat to come prove he is really Tom, but Tom remembers what Felix said about love being sacrifice and decides to save David instead, sacrificing his only chance at regaining his humanity. At the tower, David jumps off the building and the cat jumps after him pulling a cable. It is then revealed that David is wearing a BASE jumping parachute. He lands in the middle of the party, presents the articles of incorporation of the company, and announces that he now controls his father's 51% of the stock. He says the company will remain a family company and fires Ian. Although the cat is not seen landing, Tom wakes up from his coma just in time to see David's announcement on the television. Meanwhile, Ian passes Felix who tells him to hang up his cell phone. Ian ignores him, but is then hit by a car, and his consciousness transfers into a cat that Felix takes back to his shop. Tom and Rebecca return to Felix where Tom asks if he has any dogs. Felix says he does not, but presents Mr. Fuzzypants, who has one life left.",
    "Aging Greek Theo Tomasis rose from his peasant roots to become a mogul who owns oil tankers, airlines, and Mediterranean islands and longs to be elected President of Greece. He eventually falls in love with the considerably younger Liz Cassidy. The two first meet when she is visiting his island estate with her husband James, the Senator from the state of Massachusetts. Theo immediately is attracted to her and, despite the fact she is happily married, begins to woo her aboard his yacht while James converses with the former Prime Minister of the United Kingdom.  Meanwhile, Theo's relationship with his son, Nico, is tenuous at best. He encourages Nico to join the business and make money in the same manner he did. Nico wants to manage his own affairs and convinces his father to jumpstart his ventures. The plan is to buy older freighters and convert them to oil tankers and sell them to the Saudi family. This would allow Saudi oil magnates to control the delivery of oil outside of American Corporate controls. Theo is impressed and helps bankroll the purchase of a fleet of boats.  Upon their return to the States, the Cassidys begin their run for President of the United States, eventually winning the primary and the general election. The First Lady later suffers a miscarriage and becomes depressed and tired of the public life. Tomasis, who has a good relationship with the First Family, offers a vacation on his yacht as a change of pace. The President does not care for Tomasis due to his shady dealings in the shipping business and Arab oil states, so Lizzie goes by herself, much to his anger. In Greece, the yacht is full of guests and celebrities. Tomasis seems to spend a lot of time with Lizzie, who is a bit restrained. During a dinner however, Lizzie joins in a traditional sensual dance with plates being thrown and the other guests clapping. Lizzie is attracted to Tomasis' free spirit and joie de vivre and leaves the vacation early to go back to the states-much to his protests. Back in the US, Lizzie and James walk on the family beach-front property, where the President is assassinated by a sharpshooter. After the funeral, Lizzie becomes depressed and stays outside of public eyes even though the former President's brother John (the attorney general) requests she help promote his campaign. She refuses, and Tomasis, re-invites her to Greece for a longer stay. In Greece, the relationship blooms and Lizzie cheers up. Tomasis' business dealings with Nico are becoming scrutinized by American politicians due to the OPEC embargoes and anger over oil prices. Freighters are getting confiscated and the investment of hundreds of millions at risk. Upon his return from the US, Theo asks Lizzie to marry him. Lizzie wants to but is worried about appearances in her country but he outlines the arrangement that is fair and financially attractive enough to abandon the Cassidy family fortune and control. After the wedding, there are growing pains for the relationship, as Lizzie is accustomed to being a voice in her husband's affairs in the past. After Tomasis berates her for interfering in his business, he apologizes for his Greek chauvinism, and they learn to live with each other's cultural differences. The marriage is loving until Niko dies in an accident, and Tomasis gets depressed. Lizzie manages to coax him out of his moods, and things appear to look up as his lawyer and representative cops him a cheap plea deal on the oil manipulation charges. However, his health declines and a doctor gives him a bad prognosis. He holds his medical information secret as his energy and health declines. When Liz wants to shop in Paris, he encourages her to go alone and enjoy herself.  She leaves the boat on her vacation, waving, and Tomasis visits his old peasant home on the coast for the last time visiting his old haunts and ending with traditional Greek dance.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.8029, 0.0491],
#         [0.8029, 1.0000, 0.0805],
#         [0.0491, 0.0805, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 5,804 training samples
Columns: sentence_0 and sentence_1
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1
type string string
details
min: 10 tokens
mean: 121.93 tokens
max: 256 tokens

min: 16 tokens
mean: 235.34 tokens
max: 256 tokens

	sentence_0	sentence_1
type	string	string
details	min: 10 tokens mean: 121.93 tokens max: 256 tokens	min: 16 tokens mean: 235.34 tokens max: 256 tokens

Samples:

sentence_0	sentence_1
`Steve Grayson (Elvis Presley) is a champion race car driver with a generous soul, sharing all his winnings with people in need. Because of his generosity and his boss's (Bill Bixby) passion for gambling and women, Steve finds himself drowning in debt.`	Steve Grayson (Presley) is a generous NASCAR race car driver with a heart of gold who feels compelled to bail friends and acquaintances out of financial hardship. However, Steve's manager and best friend Kenny Donford, a compulsive gambler, had been mismanaging Steve's winnings to support his gambling habits, landing Steve in deep trouble with the IRS for nonpayment of back taxes and causing many of Steve's valuable possessions to be repossessed. This proves to be a problem for Steve in his efforts to continue racing competitively and support those who depend on his intense generosity. Enter Susan Jacks (Sinatra), an IRS agent assigned to keep tabs on Steve and apply his future prize money toward his $150,000 debt, but she ends up taking a romantic interest in him as well.
`The lives of three New York couples take incredible turns after two of them, Rebecca (Marley Shelton) and Dag (Ron Eldard), share a forbidden night of love. More betrayal, passion, revenge, and death follow the couple's unmasking, but what if nothing more than a kiss had happened that night?`	Dag is a successful director of television commercials who lives with his girlfriend, Halley. Dag, however, has a serious case of wandering eye and is given to frequent flings with other women. Halley tries to turn a blind eye to Dag's infidelity, but when she discovers Dag had a one-night stand with Rebecca, a beautiful but troubled ballet dancer who is dating Dag's close friend Peter, she decides things have gone too far. To add to her feeling of betrayal, she hears a message on the answering machine for Dag from another woman looking to meet up. Halley breaks up with Dag; soon after, she makes the acquaintance of Andre, a handsome and well-mannered classical musician. Andre, however, is married to Colleen, a woman with exotic sexual tastes who meets up with Peter, now suddenly without a girlfriend, on an airline flight. Peter and Colleen have sex in the plane’s bathroom. Just before the plane lands, Peter makes a cell phone call, which causes interference with the signals from the p...
A horror and suspense thriller following the formula of Steven Spielberg's 1971 film Duel. The peaceful village of Santa Ynez, in the Southwestern United States, is shaken when a mysterious, all-black, extremely fast car runs over and kills people without any apparent reason. The episode unsettles Sheriff Wade Parent (Brolin), Sheriff Everett Peck (Marley), and Deputy Luke Johnson (Cox). The car reappears and kills Everett. An eyewitness, an old Native American woman, claims that the vehicle has no driver, and the assumption that it is a ghost car seems to be confirmed when the car threatens a school parade and Wade's girlfriend, Lauren (Lloyd), manages to lead students and teachers to a cemetery: in confrontation with Christian symbols, the car retreats and flees. The road suspense continues with a thrilling chase, culminating in a direct confrontation between Wade and the Devil's machine. And the plot unfolds with other climaxes until an epilogue." Apocalyptic, with special effects b...	Two cyclists riding through a canyon are followed down the road by a mysterious matte black car. At a bridge, the car proceeds to crush one cyclist against the wall and ram the other from behind, catapulting him off the bridge. A hitchhiker, hoping to get a ride, encounters the car and insults it after it purposely tries to run him down. It halts to a stop, reverses backwards, and runs him over several times before driving off. The local sheriff's office, called to a series of hit and runs, gets a lead on the car. It appears to be heavily customized and has no license plates, as pointed out by Amos Clemens after he saw it run over the hitchhiker. That night, after intentionally swerving around Amos, the car instead runs over Sheriff Everett Peck, leaving Chief Deputy Wade Parent in charge. During the resulting investigation, an eyewitness to the accident states there was no driver, furthering Wade's confusion. Wade asks his girlfriend, Lauren, a teacher at the local school, to cancel t...

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 4
per_device_eval_batch_size: 4
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 4
per_device_eval_batch_size: 4
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.3446	500	0.0194
0.6892	1000	0.0846
1.0338	1500	0.0424
1.3784	2000	0.0198
1.7229	2500	0.0171
2.0675	3000	0.0169
2.4121	3500	0.0039
2.7567	4000	0.003

Framework Versions

Python: 3.9.18
Sentence Transformers: 5.1.2
Transformers: 4.57.6
PyTorch: 2.8.0+cu128
Accelerate: 1.10.1
Datasets: 4.5.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}