Instructions to use OpenMed/mmBERT-ClinicalScope-Spanish-Base-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OpenMed/mmBERT-ClinicalScope-Spanish-Base-v1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="OpenMed/mmBERT-ClinicalScope-Spanish-Base-v1")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("OpenMed/mmBERT-ClinicalScope-Spanish-Base-v1") model = AutoModelForSequenceClassification.from_pretrained("OpenMed/mmBERT-ClinicalScope-Spanish-Base-v1", device_map="auto") - Notebooks
- Google Colab
- Kaggle
OpenMed mmBERT ClinicalScope Spanish Base v1
OpenMed's open model for negation and uncertainty in Spanish clinical text. Mark one word of a Spanish clinical sentence and it says whether the word is inside a negation scope (no se observa neumonía), inside an uncertainty scope (probable infección urinaria) or outside both. Trained and evaluated by OpenMed. Apache-2.0, 307.5M parameters, runs on CPU.
| Developed by | OpenMed |
|---|---|
| Task | Negation and uncertainty scope of one marked word, three classes: negation_scope, uncertainty_scope, outside_scope |
| Input | One Spanish sentence with one word marked: Texto: ...⟦word⟧... |
| Output | the class and its score |
| Parameters | 307.5M |
| Decoding | Highest-scoring class, no threshold |
| Language | Spanish |
| License | Apache-2.0 |
About OpenMed
OpenMed builds open models and open-source tools for clinical and biomedical text that run on your own hardware.
- 2,300+ open models on Hugging Face for clinical NER, de-identification and more.
- The OpenMed Python SDK, Apache-2.0:
pip install openmedfor de-identification, clinical NER, and FHIR and HL7 tooling, all local. - New models every week: follow OpenMed on Hugging Face and @OpenMed_AI on X.
Quick start
pip install -U "transformers>=5.17" torch
import re
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
repo = "OpenMed/mmBERT-ClinicalScope-Spanish-Base-v1"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo).eval()
def mark(sentence, start, end):
"""The evaluated input: "Texto: " and the sentence, with the word between ⟦ and ⟧."""
return "Texto: " + sentence[:start] + "⟦" + sentence[start:end] + "⟧" + sentence[end:]
def scope(sentence, words=None):
"""Label every word of the sentence (or only `words`): negation, uncertainty or outside scope."""
found = [m for m in re.finditer(r"\w+(?:-\w+)*", sentence) if words is None or m.group() in words]
texts = [mark(sentence, m.start(), m.end()) for m in found]
with torch.inference_mode():
probs = model(**tokenizer(texts, padding=True, return_tensors="pt")).logits.softmax(-1)
return [(m.group(), model.config.id2label[int(p.argmax())], float(p.max())) for m, p in zip(found, probs)]
sentence = "No se observa neumonía ni derrame pleural; probable infección urinaria."
for word, label, score in scope(sentence):
print(f"{word:<10} {label:<17} {score:.2f}")
No outside_scope 1.00
se negation_scope 0.92
observa negation_scope 1.00
neumonía negation_scope 1.00
ni negation_scope 0.99
derrame negation_scope 1.00
pleural negation_scope 1.00
probable uncertainty_scope 0.73
infección uncertainty_scope 1.00
urinaria uncertainty_scope 0.97
How it reads the input
mark()builds the input the model was trained and evaluated on:Texto:and the sentence, with the marked word between⟦and⟧. Keep it exactly as written and mark one word at a time.scope()marks every word of the sentence in turn, or only the words you pass, for examplescope(sentence, words={"neumonía", "infección"}). Pass one sentence at a time.- The label is the highest-scoring of the three classes, the decoding behind the evaluation; there is no threshold. Scores are not calibrated probabilities.
- Read the labels of nouns and adjectives: those are the words it was trained and scored on. Cue words (no, sin, probable) and verbs also get a label, but those were never scored.
- The training sentences had their punctuation split off by spaces (neumonía ;). Normal punctuation works: the quick-start sentence written that way gives the same labels.
Many sentences, GPU
- Many sentences:
scope()batches the words of one sentence; call it in a loop for many sentences. - On a GPU, move the model with
model.to("cuda")and the batch withtokenizer(...).to("cuda").
Evaluation
Held-out Spanish clinical sentences: 2,028 marked words (867 inside a negation scope, 165 inside an uncertainty scope, 996 outside both) in 1,277 sentences of an existing human-annotated corpus of Spanish clinical text with negation and uncertainty scopes. The held-out sentences come from two clinical specialties (cardiology and traumatology) that were kept out of training and tuning. The model was trained on 7,755 marked words from 14 other specialties for two fixed epochs, checked on 2,258 more from two further specialties (internal medicine and urology), and these 2,028 were scored once at the end, with the model, the references and the scoring code frozen first. Marked words are nouns and adjectives of three or more letters; the cue words themselves (no, sin, posible) are never marked. The source has no patient identifiers, so the split is by specialty and sentence, not by patient. Macro F1 averages the F1 of the three classes.
| Model | Macro F1 | Accuracy |
|---|---|---|
| OpenMed mmBERT ClinicalScope Spanish Base v1 (this model) | 93.42 | 95.86 |
| Supervised lexical reference: word and character TF-IDF with logistic regression, trained on the same marked words with the marked sentence and six words of context on each side | 80.40 | 86.29 |
| fastino/GLiNER2.5-Decide-1B zero-shot with the same task and prompts | 33.26 | 41.52 |
| Before OpenMed fine-tuning: jhu-clsp/mmBERT-base with a new, untrained three-class head (a control, not a baseline) | 23.95 | 24.46 |
| Lexical control that sees only the marked word, no context | 49.52 | 59.71 |
This model scores +13.02 macro F1 points above the supervised lexical reference (95% CI +10.51 to +15.49, paired bootstrap over 1,274 sentence groups, 2,000 resamples). Against the untrained head the change is +69.47 points, but that head is only a control: the real hurdles are the lexical reference and the Decide zero-shot row (+60.16 points, 95% CI +57.96 to +62.22). The lexical control that sees only the marked word reaches 49.52: scope depends on the context, not on the word.
By class
| Class | Marked words | Precision | Recall | F1 | F1, lexical reference |
|---|---|---|---|---|---|
| negation_scope | 867 | 96.12 | 97.23 | 96.67 | 87.81 |
| uncertainty_scope | 165 | 93.10 | 81.82 | 87.10 | 65.49 |
| outside_scope | 996 | 96.02 | 96.99 | 96.50 | 87.90 |
Uncertainty is the hardest class: 87.10 F1, recall 81.82. Of the 165 uncertain words it finds 135; it reads 18 as outside and 12 as negation. Uncertain words are also the rarest in the data.
Two harder slices: in sentences that hold both a negation and an uncertainty scope (107 marked words), accuracy is 91.59 (lexical reference 63.55); for words outside scope in sentences that do have a scope (705), recall is 96.17 (lexical reference 85.67), so the model does not stretch a scope over the whole sentence.
The OpenMed Spanish clinical scope models
Same training words, same 2,028 held-out words, same lexical reference (80.40 macro F1). The mmBERT model scores higher overall at about a quarter of the parameters; the Decide model finds 2 more of the 165 uncertain words.
| Model | Parameters | Code | Macro F1 | Uncertainty F1 | Change vs lexical reference (95% CI) |
|---|---|---|---|---|---|
| OpenMed mmBERT ClinicalScope Spanish Base v1 (this model) | 307.5M | Transformers | 93.42 | 87.10 | +13.02 (+10.51 to +15.49) |
| OpenMed GLiNER2.5 ClinicalScope Spanish Decide 1B v1 | 1.19B | OpenMed SDK | 91.83 | 84.83 | +11.43 (+9.06 to +13.72) |
Examples
Real outputs of this model with the quick-start scope() helper (Transformers 5.22.1, on CPU). The sentences are synthetic, written in Spanish for this card. Each word was marked in turn; runs of words inside a scope are in bold with their scope, and words outside both are plain.
Quick start: No se observa neumonía ni derrame pleuralnegation; probable infección urinariauncertainty.
- Negation list: Sin fiebre ni disneanegation; refiere dolor torácico opresivo desde ayer.
- Probable: Probable infección urinariauncertainty, pendiente de urocultivo.
- Ruled out: Se descarta trombosis venosa profunda mediantenegation ecografía doppler.
- Suspicion: Sospecha de fractura de escafoidesuncertainty, se solicita resonancia magnética.
- Not ruled out: No puede descartarsenegation una rotura parcial del ligamento cruzado anterioruncertainty.
- Negation then a finding: No se palpan adenopatías cervicalesnegation y el abdomen es blando y depresible.
- Both scopes: Niega dolor abdominalnegation, aunque podría tratarse de una apendicitis incipienteuncertainty.
- Double negation: No es imposiblenegation que exista una lesión meniscaluncertainty.
- Scope to the end: No hay evidencia de metástasis hepáticas ni óseas en el estudio actualnegation.
- Hedged impression: Impresiona de celulitis, sin datos de abscesonegation.
- Plan to rule out: Descartar tromboembolismo pulmonarnegation.
- Family history: Su padre falleció de un infarto de miocardio.
Good to know
- A scope stops at the end of its clause: in example 1, fiebre and disnea are negated and dolor torácico after the semicolon is outside; in the quick start the negation stops at the semicolon and probable opens an uncertainty scope.
- One sentence can hold both scopes: example 7 negates dolor abdominal and puts apendicitis incipiente in an uncertainty scope.
- A negated doubt reads as uncertainty, not negation: No puede descartarse una rotura (example 5) and the double negation No es imposible que exista una lesión (example 8) put rotura and lesión in an uncertainty scope.
- It misses some hedges: Impresiona de celulitis (example 10) leaves celulitis outside, although the phrase means the finding is only suspected. Sospecha de (example 4) is caught.
- A plan reads as a negation: Descartar tromboembolismo pulmonar (example 11) puts tromboembolismo in a negation scope, though nothing has been ruled out yet.
- A negation scope can run to the end of the sentence: estudio actual, where the findings were looked for, is inside the negation in example 9. In example 3 the scope takes in mediante and stops before ecografía doppler.
- Outside is not the same as present: infarto in example 12 is outside both scopes, but it is the father's history. The model reads negation and uncertainty only; it does not say who has a condition, or when.
- Most scores sit near 1.00 (77 of the 113 words above round to 1.00), so use them to rank words, not as probabilities.
- Cue words mostly come back outside (No in the quick start, Sin and Sospecha in examples 1 and 4). They were never scored, so read the findings.
Limitations
- It labels one marked word at a time for negation and uncertainty scope. It is not a full clinical assertion model: outside does not mean a condition is present, and it does not handle family history, time, hypotheticals or who the condition belongs to.
- Trained and tested on held-out sentences of one Spanish clinical corpus, split by specialty and sentence, not by patient; patient-level independence cannot be checked. It has not been validated on other institutions, other kinds of notes or other languages; the examples above are synthetic.
- Uncertainty is the weak class: 87.10 F1 and 81.82 recall; it missed 30 of the 165 uncertain words in the evaluation.
- Only nouns and adjectives were marked in training and evaluation. Labels on cue words, verbs and function words were not scored.
- Scores are not calibrated probabilities.
- It labels words. It does not give medical advice and is not a medical device.
Built on
OpenMed fine-tuned this model from jhu-clsp/mmBERT-base by Johns Hopkins University CLSP (MIT), built with the open-source Transformers library. Training and evaluation data: held-out and training sentences of an existing human-annotated corpus of Spanish clinical text, annotated for negation and uncertainty cues and their scopes. Thank you to Johns Hopkins University CLSP for releasing mmBERT-base openly.
This model is released under Apache-2.0. No training or evaluation text is included in this repository.
Citation
@misc{panahi2025openmedner,
title={OpenMed NER: Open-Source, Domain-Adapted State-of-the-Art Transformers for Biomedical NER Across 12 Public Datasets},
author={Maziyar Panahi},
year={2025},
eprint={2508.01630},
archivePrefix={arXiv}
}
@misc{marone2025mmbert,
title={mmBERT: A Modern Multilingual Encoder with Annealed Language Learning},
author={Marc Marone and Orion Weller and William Fleshman and Eugene Yang and Dawn Lawrie and Benjamin Van Durme},
year={2025},
eprint={2509.06888},
archivePrefix={arXiv}
}
OpenMed · Hugging Face · GitHub · PyPI · X
- Downloads last month
- 25
Model tree for OpenMed/mmBERT-ClinicalScope-Spanish-Base-v1
Base model
jhu-clsp/mmBERT-basePapers for OpenMed/mmBERT-ClinicalScope-Spanish-Base-v1
OpenMed NER: Open-Source, Domain-Adapted State-of-the-Art Transformers for Biomedical NER Across 12 Public Datasets
Article mentioning OpenMed/mmBERT-ClinicalScope-Spanish-Base-v1
Evaluation results
- Macro F1 (three classes) on Held-out Spanish clinical sentences (2,028 marked words)self-reported93.420
- Accuracy on Held-out Spanish clinical sentences (2,028 marked words)self-reported95.860
- F1 negation_scope on Held-out Spanish clinical sentences (2,028 marked words)self-reported96.670
- F1 uncertainty_scope on Held-out Spanish clinical sentences (2,028 marked words)self-reported87.100
- F1 outside_scope on Held-out Spanish clinical sentences (2,028 marked words)self-reported96.500