SentenceTransformer based on Alibaba-NLP/gte-modernbert-base

This is a sentence-transformers model finetuned from Alibaba-NLP/gte-modernbert-base. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: Alibaba-NLP/gte-modernbert-base
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Instruct: Identify solution for the email\nQuery: How to grant Placer.ai access via Entra group App_PlacerAI',
    '::: table-wrap\n+----------------------+-----------------------------------------------------+\n| **Application Name** | Placer.ai                                           |\n+----------------------+-----------------------------------------------------+\n| **Description**      | Media tool                                          |\n+----------------------+-----------------------------------------------------+\n| **Owner**            | Lou Roberts                                         |\n+----------------------+-----------------------------------------------------+\n| **Requester**        |                                                     |\n+----------------------+-----------------------------------------------------+\n| **Service**          |                                                     |\n+----------------------+-----------------------------------------------------+\n| **On this page**     | ::: {.toc-macro .rbtoc1758184816272}                |\n|                      | -   [🚪 Access](#Placer.AI-Access)                  |\n|                      | -   [🧑\u200d🤝\u200d🧑 User Management](#Placer.AI-UserManagement) |\n|                      | -   [Known Issues](#Placer.AI-KnownIssues)          |\n|                      | :::                                                 |\n+----------------------+-----------------------------------------------------+\n:::\n\n## ![(blue star)](https://sitesandservices.atlassian.net/wiki/s/981112411/6452/348eaed8ba2477d472357ccbcf1a0ef6e8bafe14/_/images/icons/emoticons/72/1f6aa.png){.emoticon .emoticon-blue-star emoji-id="1f6aa" emoji-shortname=":door:" emoji-fallback="🚪" width="16" height="16" emoticon-name="blue-star"} Access {#Placer.AI-Access}\n\nAdd user to App_PlacerAI group in Entra.\n\nThere are no licensing limits, so you can add anyone from Media or related departments who requests access. If questions, ask the app owner above.\n\n## ![(blue star)](https://sitesandservices.atlassian.net/wiki/s/981112411/6452/348eaed8ba2477d472357ccbcf1a0ef6e8bafe14/_/images/icons/emoticons/72/1f9d1-200d-1f91d-200d-1f9d1.png){.emoticon .emoticon-blue-star emoji-id="1f9d1-200d-1f91d-200d-1f9d1" emoji-shortname=":people_holding_hands:" emoji-fallback="🧑\u200d🤝\u200d🧑" width="16" height="16" emoticon-name="blue-star"} User Management {#Placer.AI-UserManagement}\n\n::: table-wrap\n  -------------- -------------------------------------------------------------------\n  **Action**     **Function**\n  Add_User       Add user to group and have them sign in. It has JIT provisioning.\n  Remove_User    Manual via portal\n  Disable_User   Remove from App_PlacerAI group\n  -------------- -------------------------------------------------------------------\n:::\n\n## ![(broken heart)](https://sitesandservices.atlassian.net/wiki/s/981112411/6452/348eaed8ba2477d472357ccbcf1a0ef6e8bafe14/_/images/icons/emoticons/broken_heart.png){.emoticon .emoticon-broken-heart emoji-id="1f494" emoji-shortname=":broken_heart:" emoji-fallback="💔" width="16" height="16" emoticon-name="broken-heart"} Known Issues {#Placer.AI-KnownIssues}\n\n::: table-wrap\n  ----------- -------------\n  **Issue**   **Details**\n              \n              \n  ----------- -------------\n:::',
    '## **Computer Setup** {#CreatingaUSBdrivewithOSDCloud-ComputerSetup}\n\n> NOTE: Your machine needs to be able to write unencrypted files to a USB drive. If your computer is not already in Device_USB_Encryption_Exclusion, add it.\n\nOn the machine that will create the USB drive:\n\nInstall Windows 11 24H2 ADK: [https://go.microsoft.com/fwlink/?linkid=2289980](https://go.microsoft.com/fwlink/?linkid=2289980){.external-link rel="nofollow"}\n\n-   Install locally when prompted - we want to install on this machine\n\n-   You can accept all the defaults when installing\n\nInstall Windows PE add-on: [https://go.microsoft.com/fwlink/?linkid=2289981](https://go.microsoft.com/fwlink/?linkid=2289981){.external-link rel="nofollow"}\n\n-   There are no settings, so you can click Next to install.\n\n## **Configuring a USB drive with OSD Cloud** {#CreatingaUSBdrivewithOSDCloud-ConfiguringaUSBdrivewithOSDCloud}\n\n1.  Copy this script on a PC:\n\n[https://glondon.sharepoint.com/:u:/r/sites/Group_IT/Shared%20Documents/General/OSDCloud/OSDCloud%20Automated%20USB%20with%20User%20Prompt.ps1?csf=1&web=1&e=6FRniH](https://glondon.sharepoint.com/:u:/r/sites/Group_IT/Shared%20Documents/General/OSDCloud/OSDCloud%20Automated%20USB%20with%20User%20Prompt.ps1?csf=1&web=1&e=6FRniH){.external-link card-appearance="inline" rel="nofollow"}\n\n::: {.confluence-information-macro .confluence-information-macro-information}\n[]{.aui-icon .aui-icon-small .aui-iconfont-info .confluence-information-macro-icon}\n\n::: confluence-information-macro-body\nIt\'s stored in the team "Gravity -- Group IT" so use your Gravity Global account to access the link\n:::\n:::\n\n2.  Open PowerShell ISE as an administrator\n\n3.  Paste the script in PowerShell ISE and run (or you can download the file and run it). This script will remind you to have a 32GB flash drive connected to the PC.\n\n1.  NOTE: The initial popup may appear behind the ISE window. If it seems stuck, there is probably a popup hiding.\n\n2.  If you use Windows PowerShell instead (as admin), the first popup message will fail. You can ignore the error and it will proceed automatically.\n\n::: {.confluence-information-macro .confluence-information-macro-information}\n[]{.aui-icon .aui-icon-small .aui-iconfont-info .confluence-information-macro-icon}\n\n::: confluence-information-macro-body\nIt\'s probably best to disconnect other USB flash drives to not accidentally format the wrong flash drive\n:::\n:::\n\n4.  The script will also prompt for a name for the OSDCloud Template. Follow the naming scheme Win\\_(version of Windows)\\_(Build version)\\_(Current month and year).\n\n1.  Example: Win_11_24H2_Feb25.\n\n5.  In the console of PowerShell ISE, eventually you\'ll be asked to confirm which USB Drive you want to format.\n\n1.  If you only have one USB drive connected, it\'s the only USB drive listed. If you have more than one USB drive connected, be careful and choose the correct drive.\n\n## **Dell BIOS Configuration Files** {#CreatingaUSBdrivewithOSDCloud-DellBIOSConfigurationFiles}\n\nAdd these two files to the root of the OSDCloud drive. (Note there are two partitions on the USB stick - you want OSDCloud)\n\n-   CCTK Folder: [https://glondon.sharepoint.com/:f:/r/sites/Group_IT/Shared%20Documents/General/OSDCloud/CCTK?csf=1&web=1&e=q5sltN](https://glondon.sharepoint.com/:f:/r/sites/Group_IT/Shared%20Documents/General/OSDCloud/CCTK?csf=1&web=1&e=q5sltN){.external-link card-appearance="inline" rel="nofollow"}\n\n-   BIOSChanges.ini: [https://glondon.sharepoint.com/:u:/r/sites/Group_IT/Shared%20Documents/General/OSDCloud/BIOSChanges.ini?csf=1&web=1&e=P8ufTf](https://glondon.sharepoint.com/:u:/r/sites/Group_IT/Shared%20Documents/General/OSDCloud/BIOSChanges.ini?csf=1&web=1&e=P8ufTf){.external-link card-appearance="inline" rel="nofollow"}\n\nBoth the folder and file should be placed in the root of the USB drive.\n\n## Dell Command Update and other drivers {#CreatingaUSBdrivewithOSDCloud-DellCommandUpdateandotherdrivers}\n\nOn your OSDCloud USB stick:\n\n-   There are two partitions - choose the WINPE "drive" in Windows Explorer\n\n-   Create a folder called "Dell_Drivers" in the root\n\n-   Download and copy three files there:\n\n-   Installer for latest Dell Command Update (Windows Universal Application) [https://www.dell.com/support/kbdoc/en-us/000177325/dell-command-update](https://www.dell.com/support/kbdoc/en-us/000177325/dell-command-update){.external-link card-appearance="inline" rel="nofollow"}\n\n-   .NET Desktop Runtime (the version required by Dell Command Update; usually the LTS version): [https://dotnet.microsoft.com/en-us/download/dotnet/8.0](https://dotnet.microsoft.com/en-us/download/dotnet/8.0){.external-link card-appearance="inline" rel="nofollow"}\n\n-   Note that you want the DESKTOP RUNTIME, not the SDK, nor the CLI runtime. Annoying.\n\n-   Latest Intel NPU driver: [https://www.intel.com/content/www/us/en/download/794734/intel-npu-driver-windows.html](https://www.intel.com/content/www/us/en/download/794734/intel-npu-driver-windows.html){.external-link card-appearance="inline" rel="nofollow"}\n\n-   This is only necessary so long as Dell Command Update still installs a version with bugs. This shouldn\'t be necessary past 2025, we hope.\n\n## Rebuild the computer {#CreatingaUSBdrivewithOSDCloud-Rebuildthecomputer}\n\nIf rebuilding a PC already enrolled in Gravity Global, follow this article:\n\n[PC Rebuild Procedure](https://sitesandservices.atlassian.net/wiki/spaces/IT/pages/4294672385/PC+Rebuild+Procedure){linked-resource-id="4294672385" linked-resource-version="11" linked-resource-type="page"}\n\nIf needing to rebuild a PC and enroll it in Gravity Global, follow this article:\n\n[Enrolling a PC in Gravity from other organizations](https://sitesandservices.atlassian.net/wiki/spaces/IT/pages/4294311952/Enrolling+a+PC+in+Gravity+from+other+organizations){linked-resource-id="4294311952" linked-resource-version="10" linked-resource-type="page"}',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.7865, 0.1898],
#         [0.7865, 1.0000, 0.3530],
#         [0.1898, 0.3530, 1.0000]])

Evaluation

Metrics

Semantic Similarity

Dataset: validation
Evaluated with EmbeddingSimilarityEvaluator

Metric	Value
pearson_cosine	nan
spearman_cosine	nan

Training Details

Training Dataset

Unnamed Dataset

Size: 679 training samples
Columns: sentence_0, sentence_1, and sentence_2

Approximate statistics based on the first 679 samples:

	sentence_0	sentence_1	sentence_2
type	string	string	string
details	min: 15 tokens mean: 65.23 tokens max: 408 tokens	min: 6 tokens mean: 890.42 tokens max: 5617 tokens	min: 6 tokens mean: 861.59 tokens max: 5617 tokens

Samples:

sentence_0	sentence_1	sentence_2
`Instruct: Identify solution for the email Query: Account locked out MFA / password reset assistance`	`::: plugin_pagetree :::`	For systems running in AWS Elastic Container Service (ECS), often in Docker (and Fargate, for searching), you sometimes need to run console or command line commands in the Docker container while it is running. NOTE: It is possible to SSH into Docker containers, since they are just servers like anything else. But, you would need to use this process or build it into the Docker image to allow SSH, run SSH, and manage SSH keys. (And then you have SSH open, which may not be what you want long-term). This system allows you to do quick commands without messing with all that. Full description: Monitor Amazon ECS containers with ECS Exec - Amazon Elastic Container Service{.external-link rel="nofollow"} And there are plenty of external guides, like this: CLI documentation: [execute-command --- AWS CLI 2.24.20 Command Reference](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ecs/execute-command.ht...
`Instruct: Identify solution for the email Query: Help configuring VPC/subnet CIDRs, route tables, and network ACLs for a new AWS account per the internal guide`	AWS Organizations made it simpler for us to have many accounts, one for each purpose, instead of trying to set up byzantine permissions to segregate people within one account. For example, creating a limited admin in an account meant for one thing is much, much easier than trying to let people do things like create everything (RDS, EC2, S3) but not be able to touch the ones other people create. Here's how to set up a new account, since I always forget the steps. 1. In the Management account > AWS Organizations > Create a new AWS account 1. Use a plus address for the admin email 2. In AWS Organizations, move the new account to the proper folder in hierarchy 3. In IAM Identity Center, assign a group permissions to use the account 1. Typically you start with AWS_Console_Admins group having full Administrator permissions. 4. In Entra, set up 1-2 new groups: 1. AWS_Console_... if a subset of people need some level of access to this account, create a group for them and add ...	::: table-wrap +-------------------------------------+-----------------------------------------+ \| Location/Office \| London \| +-------------------------------------+-----------------------------------------+ \| Static or DHCP \| - [Static]{.placeholder-inline-tasks} \| \| \| - [DHCP]{.placeholder-inline-tasks} \| +-------------------------------------+-----------------------------------------+ \| Public IP \| 62.7.55.181 \| +-------------------------------------+-----------------------------------------+ \| Subnet Mask \| 255.255.255.254 \| +-------------------------------------+-----------------------------------------+ \| Gateway \| 62.7.55.180 \| +-------------------------------------+-----------------------------------------+ \|...
`Instruct: Identify solution for the email Query: Email 3: Trouble adding received hardware to Inventory and ensuring Automated Enrollment / AutoPilot enrollment for a Mac device (checking serial, asset creation, and AE/AutoPilot status).`	When creating a new Purchase Order, you may need to get a quote from a supplier before placing the order, other times the price and product are already known. ### Creating a New Purchase Order PO {#NewMachinePurchase-CreatingaNewPurchaseOrderPO} When purchasing a new Machine, a PO should be create in Fresh service. Go to Assets → Purchase Orders, then select New Purchase Order. Enter, Name, Vendor and Currency, Shipping and Billing Address. Leave the T&Cs blank unless you need to add notes, try to include the name of the user if known when purchasing, then Click Save. When adding a item, start typing the model of a machine, eg MacBook Air, and the click on Products select the correct product in the list. [{.confluence-embedded-image .image-center width="371"}]{.confluence-embedded-file-wrapper .image-center-wra...	::: table-wrap +----------------------+---------------------------------------------------+ \| Application Name \| Pigment \| +----------------------+---------------------------------------------------+ \| Description \| Finance team tool for some kind of forecasting \| +----------------------+---------------------------------------------------+ \| Owner \| Avi Gal \| +----------------------+---------------------------------------------------+ \| Requester \| \| +----------------------+---------------------------------------------------+ \| Service \| \| +----------------------+---------------------------------------------------+ \| On this page \| ::: {.toc-macro .rbtoc1758184811742} \| \| \| - 🚪 Access ...

Loss: MultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 1
per_device_eval_batch_size: 1
num_train_epochs: 5
fp16: True
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 1
per_device_eval_batch_size: 1
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 5
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	validation_spearman_cosine
0.0736	50	nan

Framework Versions

Python: 3.12.11
Sentence Transformers: 5.1.1
Transformers: 4.56.1
PyTorch: 2.8.0+cu128
Accelerate: 1.10.1
Datasets: 4.1.0
Tokenizers: 0.22.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}