Instructions to use OpenMed/GTE-ModernBERT-Biomedical-Embedding-Base-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OpenMed/GTE-ModernBERT-Biomedical-Embedding-Base-v1 with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("OpenMed/GTE-ModernBERT-Biomedical-Embedding-Base-v1") model = AutoModel.from_pretrained("OpenMed/GTE-ModernBERT-Biomedical-Embedding-Base-v1", device_map="auto") - Notebooks
- Google Colab
- Kaggle
OpenMed GTE-ModernBERT Biomedical Embedding Base v1
OpenMed's open model for finding evidence in the biomedical literature. It turns English research questions and PubMed-style passages into 768-dimensional vectors, so each question lands closest to the passages that answer it. Trained and evaluated by OpenMed. Apache-2.0, 149.0M parameters, runs on CPU.
| Developed by | OpenMed |
|---|---|
| Task | Biomedical question to literature passage retrieval (dense embeddings) |
| Output | 768-dimensional vector of unit length; the dot product of two vectors is their cosine similarity |
| Parameters | 149.0M |
| Max input | 384 tokens (the length it was trained and evaluated at) |
| Pooling | first token (CLS), then L2 normalization |
| Prefixes | none |
| Language | English |
| License | Apache-2.0 |
About OpenMed
OpenMed builds open models and open-source tools for clinical and biomedical text that run on your own hardware.
- 2,300+ open models on Hugging Face for clinical NER, de-identification, retrieval and more.
- The OpenMed Python SDK, Apache-2.0:
pip install openmedfor de-identification, clinical NER, and FHIR and HL7 tooling, all local. - New models every week: follow OpenMed on Hugging Face and @OpenMed_AI on X.
Quick start
pip install -U transformers torch
import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer
repo = "OpenMed/GTE-ModernBERT-Biomedical-Embedding-Base-v1"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModel.from_pretrained(repo, dtype=torch.float32).eval()
def embed(texts):
# The evaluated settings: no prefix, 384 tokens, first-token (CLS) pooling, unit length
batch = tokenizer(
texts, padding=True, truncation=True, max_length=384, return_tensors="pt"
)
with torch.no_grad():
cls = model(**batch).last_hidden_state[:, 0]
return F.normalize(cls, p=2, dim=1)
questions = [
"Which drugs are CFTR modulators?",
"What type of mutation causes Huntington disease?",
]
passages = [
(
"Cystic fibrosis is caused by mutations in the CFTR gene, which "
"encodes a chloride channel; the F508del variant is found on at "
"least one allele in most patients."
),
(
"Ivacaftor, a CFTR potentiator, increases channel open probability "
"and improved lung function in patients carrying the G551D "
"mutation."
),
(
"The CFTR correctors lumacaftor, tezacaftor and elexacaftor improve "
"the folding and trafficking of F508del-CFTR to the cell surface."
),
(
"Huntington disease is caused by an expanded CAG trinucleotide "
"repeat in exon 1 of the HTT gene, and longer repeats are "
"associated with earlier onset."
),
(
"Fragile X syndrome results from expansion of a CGG repeat in the "
"5' untranslated region of FMR1, leading to gene silencing."
),
]
scores = embed(questions) @ embed(passages).T # cosine similarity
for question, row in zip(questions, scores):
print(question)
for j in row.argsort(descending=True)[:3]:
print(f" {row[j]:.3f} {passages[j][:60]}...")
Which drugs are CFTR modulators?
0.675 The CFTR correctors lumacaftor, tezacaftor and elexacaftor i...
0.573 Ivacaftor, a CFTR potentiator, increases channel open probab...
0.370 Cystic fibrosis is caused by mutations in the CFTR gene, whi...
What type of mutation causes Huntington disease?
0.705 Huntington disease is caused by an expanded CAG trinucleotid...
0.482 Cystic fibrosis is caused by mutations in the CFTR gene, whi...
0.407 Fragile X syndrome results from expansion of a CGG repeat in...
The settings
- No prefixes or instructions: questions and passages are embedded as they are, as in training and evaluation.
- First-token (CLS) pooling, then L2 normalization, the base model's own pooling. This repository has no sentence-transformers configuration, so use the code above rather than
SentenceTransformer(...), which would fall back to mean pooling (see Good to know). max_length=384is the length used in training and evaluation (2 of the evaluation's 17,170 inputs were longer). The base model accepts up to 8,192 tokens; longer inputs were not evaluated after fine-tuning.dtype=torch.float32loads the stored float32 weights. On CPU the repository's own helper also runs in float32, as here; the evaluation ran on a GPU with bfloat16 autocast.- Cosine scores rank passages for one question; they are not probabilities.
Large collections, GPU
- Embed the passage collection once, in batches (
embed(passages[i:i + 64])), and store the vectors; at search time, embed only the question. - On a GPU:
model = model.to("cuda"), move each batch withbatch = batch.to("cuda")insideembed, and wrap the forward pass intorch.autocast("cuda", dtype=torch.bfloat16), the setting the evaluation used.
Evaluation
BioASQ, 200 held-out development questions: English biomedical questions written by experts for the BioASQ challenge, each with the PubMed abstract snippets the experts selected as evidence. OpenMed split the BioASQ training release into 1,467 training questions, 160 calibration questions and these 200 development questions (160 groups), keeping questions that share a paper or a snippet inside one split and leaving out papers used in OpenMed's earlier training and evaluation runs. Each question searches a catalog of 16,970 snippets: the 2,082 snippets selected for these 200 questions plus the 14,888 training snippets. The checkpoint was chosen on the calibration questions, then the model and scoring code were frozen and these questions were scored once. The official BioASQ test sets were not used, so these numbers are not comparable to published BioASQ results.
Known-positive scoring: only the snippets the BioASQ experts selected count as relevant. Every other snippet counts as not relevant, although some may also answer the question; they were never judged. Recall@10 is the share of a question's selected snippets that appear in the top ten; 73 of the 200 questions have more than ten selected snippets, so it cannot reach 100%.
| Model | NDCG@10 | Recall@10 | MRR@10 |
|---|---|---|---|
| OpenMed GTE-ModernBERT Biomedical Embedding Base v1 (this model) | 86.81 | 73.28 | 92.26 |
| Before OpenMed fine-tuning: Alibaba-NLP/gte-modernbert-base, same settings | 85.10 | 71.73 | 91.44 |
OpenMed's fine-tuning added +1.71 NDCG@10 points (95% CI +0.34 to +2.98, paired bootstrap over the 160 question groups, 2,000 resamples) and +1.55 Recall@10 points. The gain is small. The base model is already strong on this task, and the interval's lower end is near zero: expect this model to behave much like its base, slightly better on biomedical literature search.
By question type
| Question type | Questions | NDCG@10 before fine-tuning | NDCG@10 | Change |
|---|---|---|---|---|
| Factoid | 71 | 85.41 | 86.11 | +0.70 |
| List | 43 | 77.06 | 80.78 | +3.72 |
| Summary | 41 | 86.94 | 89.10 | +2.16 |
| Yes/no | 45 | 90.63 | 91.59 | +0.96 |
All four question types improved. These are point estimates from the same run, with no interval per type.
The OpenMed biomedical retrieval models
Two open OpenMed embedding models for biomedical search: one matches health questions to FAQ-style answers, the other matches research questions to literature passages. Each was scored on its own held-out questions, so their numbers are not comparable with each other.
| Model | Finds | Base model | Parameters | Dimensions | Max tokens | Prefixes | NDCG@10 before โ after |
|---|---|---|---|---|---|---|---|
| OpenMed EmbeddingGemma2 BiomedicalEvidence 270M v2 | the answer to a medical question (consumer health FAQ) | google/embeddinggemma-2 | 271.0M | 768 | 512 | question and answer prefixes | 81.64 โ 91.76 (MedQuAD, 1,966 questions) |
| OpenMed GTE-ModernBERT Biomedical Embedding Base v1 (this model) | evidence passages for a biomedical research question | Alibaba-NLP/gte-modernbert-base | 149.0M | 768 | 384 | none | 85.10 โ 86.81 (BioASQ, known-positive, 200 questions) |
Examples
Real outputs of this model with the quick-start code (same model call, prefixes and scoring), on CPU. The 13 snippets and 8 questions are synthetic, written for this card in the style of PubMed abstract sentences; none is BioASQ or PubMed text. Each question was written for one snippet (two for the list question) and searches all 13. The table gives the top three with cosine scores, the snippet written for the question in bold, and where it ranks for this model and for the base model with the same code.
| ID | Snippet |
|---|---|
pd1 |
Pembrolizumab is a humanized IgG4 monoclonal antibody that binds the programmed death-1 (PD-1) receptor on T cells and blocks its interaction with PD-L1 and PD-L2, restoring antitumor immune responses. |
ctla4 |
Ipilimumab blocks cytotoxic T-lymphocyte antigen 4 (CTLA-4), enhancing T-cell activation and proliferation. |
cftr-gene |
Cystic fibrosis is caused by mutations in the CFTR gene, which encodes a chloride channel; the F508del variant is found on at least one allele in most patients. |
ivacaftor |
Ivacaftor, a CFTR potentiator, increases channel open probability and improved lung function in patients carrying the G551D mutation. |
correctors |
The CFTR correctors lumacaftor, tezacaftor and elexacaftor improve the folding and trafficking of F508del-CFTR to the cell surface. |
sglt2 |
In randomized trials, dapagliflozin and empagliflozin reduced the risk of hospitalization for heart failure in patients with reduced ejection fraction, with or without type 2 diabetes. |
metformin |
Metformin lowers blood glucose mainly by reducing hepatic gluconeogenesis, an effect partly mediated by activation of AMP-activated protein kinase. |
statin |
Statins inhibit HMG-CoA reductase, the rate-limiting enzyme of cholesterol synthesis, and lower LDL cholesterol. |
htt |
Huntington disease is caused by an expanded CAG trinucleotide repeat in exon 1 of the HTT gene, and longer repeats are associated with earlier onset. |
fmr1 |
Fragile X syndrome results from expansion of a CGG repeat in the 5' untranslated region of FMR1, leading to gene silencing. |
vitd |
In a large randomized trial of community-dwelling adults, daily vitamin D3 supplementation did not lower the incidence of total, nonvertebral or hip fractures compared with placebo. |
brca |
Germline mutations in BRCA1 and BRCA2 substantially increase the lifetime risk of breast and ovarian cancer. |
olaparib |
Olaparib, an inhibitor of poly(ADP-ribose) polymerase (PARP), prolonged progression-free survival as maintenance therapy in women with BRCA-mutated advanced ovarian cancer. |
| # | Type | Question | Top three (cosine) | Rank of the written snippet: this model / base |
|---|---|---|---|---|
| 1 | Summary | What is the mechanism of action of pembrolizumab? | pd1 0.743; ctla4 0.497; olaparib 0.314 |
1 / 1 |
| 2 | Factoid | Which gene is mutated in cystic fibrosis? | cftr-gene 0.780; fmr1 0.466; correctors 0.440 |
1 / 1 |
| 3 | List | Which drugs are CFTR modulators? | correctors 0.675; ivacaftor 0.573; cftr-gene 0.370 |
1, 2 / 1, 2 |
| 4 | Yes/no | Do SGLT2 inhibitors reduce hospitalization for heart failure? | sglt2 0.666; correctors 0.432; ivacaftor 0.347 |
1 / 1 |
| 5 | Factoid | What type of mutation causes Huntington disease? | htt 0.705; cftr-gene 0.482; brca 0.414 |
1 / 1 |
| 6 | Summary | How does metformin lower blood glucose? | metformin 0.801; sglt2 0.387; statin 0.378 |
1 / 1 |
| 7 | Yes/no | Does vitamin D supplementation prevent fractures in older adults? | vitd 0.818; sglt2 0.326; correctors 0.263 |
1 / 1 |
| 8 | Factoid | Which PARP inhibitor is used for BRCA-mutated ovarian cancer? | olaparib 0.711; brca 0.493; pd1 0.391 |
1 / 1 |
Good to know
- All four BioASQ question types work here: factoid, list, yes/no and summary questions each rank their snippet first. The list question (example 3) returns both modulator snippets on top (
correctors0.675,ivacaftor0.573), above the CFTR gene snippet (0.370). - Snippets on the same subject that answer a different question rank lower: in example 8 the BRCA risk snippet comes second (0.493) after olaparib (0.711); in example 5 the fragile X snippet, also about a repeat expansion, ranks fourth.
- No misses on these 8 examples, and the base model also ranks the written snippets first on all 8. They are easy; the evaluation searches 16,970 snippets, and there the fine-tuning gain is small (+1.71 NDCG@10). Expect this model to behave much like its base.
- Use the code above, not
SentenceTransformer("OpenMed/GTE-ModernBERT-Biomedical-Embedding-Base-v1"): this repository has no sentence-transformers configuration, so sentence-transformers falls back to mean pooling. On the card's texts that gives different vectors (cosine as low as 0.75 with the evaluated CLS vectors). - Scores are not probabilities and there is no fixed cutoff: the written snippets here score from 0.573 to 0.818. Compare scores within one question.
Limitations
- Evaluated on BioASQ only: expert biomedical questions and PubMed abstract snippets in English. It has not been validated on clinical notes, patient questions, full-text articles or other languages; the examples above are synthetic.
- Known-positive relevance: snippets the experts did not select were never judged, so real precision may be higher or lower than measured.
- The gain over its base model is small: +1.71 NDCG@10 points on 200 questions, 95% CI +0.34 to +2.98.
- Trained and evaluated at 384 tokens on snippet-length passages; long passages were not evaluated.
- Cosine scores rank passages for one question; they are not calibrated probabilities.
- The base model's pretraining data is not fully public, so whether it saw these BioASQ questions or PubMed snippets is unknown.
- It ranks passages. It does not check that a finding is correct or current, does not give medical advice and is not a medical device.
Built on
OpenMed fine-tuned this model from Alibaba-NLP/gte-modernbert-base by Tongyi Lab, Alibaba Group (Apache-2.0), itself built on ModernBERT, with the open-source transformers library. Training and evaluation data: the BioASQ question answering training data (Zenodo record 7655130; Krithara, Nentidis, Bougiatiotis and Paliouras, 2023), questions and evidence snippets written and selected by biomedical experts for the BioASQ challenge (Tsatsaronis et al., 2015), released under CC BY 2.5. OpenMed used the training release only (training11b.json) and turned its questions and snippets into retrieval training pairs. Thank you to Tongyi Lab and to the BioASQ organizers and experts for releasing their work openly; this credit does not imply their endorsement.
This model is released under Apache-2.0. No BioASQ text is included in this repository.
Citation
@misc{panahi2025openmedner,
title={OpenMed NER: Open-Source, Domain-Adapted State-of-the-Art Transformers for Biomedical NER Across 12 Public Datasets},
author={Maziyar Panahi},
year={2025},
eprint={2508.01630},
archivePrefix={arXiv}
}
@article{tsatsaronis2015bioasq,
title={An overview of the {BIOASQ} large-scale biomedical semantic indexing and question answering competition},
author={Tsatsaronis, George and Balikas, Georgios and Malakasiotis, Prodromos and Partalas, Ioannis and Zschunke, Matthias and Alvers, Michael R. and Weissenborn, Dirk and Krithara, Anastasia and Petridis, Sergios and Polychronopoulos, Dimitris and Almirantis, Yannis and Pavlopoulos, John and Baskiotis, Nicolas and Gallinari, Patrick and Arti{\`e}res, Thierry and Ngonga Ngomo, Axel-Cyrille and Heino, Norman and Gaussier, Eric and Barrio-Alvers, Liliana and Schroeder, Michael and Androutsopoulos, Ion and Paliouras, Georgios},
journal={BMC Bioinformatics},
volume={16},
pages={138},
year={2015},
doi={10.1186/s12859-015-0564-6}
}
@article{krithara2023bioasqqa,
title={{BioASQ-QA}: A manually curated corpus for Biomedical Question Answering},
author={Krithara, Anastasia and Nentidis, Anastasios and Bougiatiotis, Konstantinos and Paliouras, Georgios},
journal={Scientific Data},
volume={10},
pages={170},
year={2023},
doi={10.1038/s41597-023-02068-4}
}
@inproceedings{zhang2024mgte,
title={mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval},
author={Zhang, Xin and Zhang, Yanzhao and Long, Dingkun and Xie, Wen and Dai, Ziqi and Tang, Jialong and Lin, Huan and Yang, Baosong and Xie, Pengjun and Huang, Fei and others},
booktitle={Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track},
pages={1393--1412},
year={2024}
}
OpenMed ยท Hugging Face ยท GitHub ยท PyPI ยท X
- Downloads last month
- 18
Model tree for OpenMed/GTE-ModernBERT-Biomedical-Embedding-Base-v1
Base model
answerdotai/ModernBERT-basePaper for OpenMed/GTE-ModernBERT-Biomedical-Embedding-Base-v1
Evaluation results
- NDCG@10 (known-positive) on BioASQ (200 held-out development questions, 16,970 snippets)self-reported86.810
- Recall@10 (known-positive) on BioASQ (200 held-out development questions, 16,970 snippets)self-reported73.280
- MRR@10 (known-positive) on BioASQ (200 held-out development questions, 16,970 snippets)self-reported92.260