--- license: llama3.1 base_model: - meta-llama/Llama-3.1-8B-Instruct language: - es tags: - biomedical-entity-linking - entity-linking - entity-disambiguation - named-entity-linking - biomedical - healthcare - snomed - spaccc - medprocner - symptemist - distemist - text-generation - constrained-decoding - causal-lm - llm library_name: transformers pipeline_tag: text-generation datasets: - bigbio/spaccc finetuning_task: - entity-linking metrics: - recall model-index: - name: LongBEL-8B-SPACCC results: - task: type: entity-linking name: Biomedical Entity Linking dataset: type: AnonymousARR42/SPACCC name: SympTEMIST metrics: - type: recall name: Recall@1 value: 0.620 - task: type: entity-linking name: Biomedical Entity Linking dataset: type: AnonymousARR42/SPACCC name: DisTEMIST metrics: - type: recall name: Recall@1 value: 0.636 - task: type: entity-linking name: Biomedical Entity Linking dataset: type: AnonymousARR42/SPACCC name: MedProcNER metrics: - type: recall name: Recall@1 value: 0.690 --- # LongBEL: Long-Context and Document-Consistent Biomedical Entity Linking ## LongBEL **LongBEL** is a novel document-level framework for biomedical entity linking (BEL). Instead of normalizing each mention independently, LongBEL conditions each prediction on the document context and on previous normalizations produced in the same document. This design enforces document-level consistency and is enhanced by our **robust memory** mechanism. The method is introduced in our paper, currently under review. ## LongBEL (SPACCC Edition) This is a **finetuned version of LLaMA-3-8B** trained on **SPACCC**, applying the LongBEL framework to enable long context and robust memory predictions. | Field | Value | |---|---| | Base model | `meta-llama/Llama-3.2-8B-Instruct` | | Task | Biomedical Entity Linking | | Dataset | SPACCC | | Knowledge base | SNOMED CT Spanish Version (July 31, 2021 release) | | Input | BigBio-like documents with mention spans and semantic groups | | Output | Ranked SNOMED concept predictions | | Decoding | Semantic-guided constrained decoding | | Main metric | Recall@1 | ## Intended Use This model is intended for research on biomedical entity linking and document-level consistency. It assumes that mention spans and semantic groups are already provided. It does **not** perform named entity recognition. In a full pipeline, a NER model should first detect mentions and assign semantic groups, then LongBEL can normalize these mentions to SNOMED concepts. ## Usage ### Loading the model ```python import torch from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained( "AnonymousARR42/LongBEL_8B_SPACCC", trust_remote_code=True, torch_dtype=torch.bfloat16, device_map="auto", ) ``` ### Inference example The model expects BigBio-like documents. Each entity should include a mention text, character offsets, and a semantic group in the `type` field. ```python num_beams = 5 bigbio_pages = [ { "id": "001", "document_id": "doc_001", "passages": [ { "id": "0", "type": "paragraph", "text": [ "Una mujer embarazada de 29 años consultó por hipertensión grave, " "cefalea y dolor epigástrico. Las pruebas de laboratorio mostraron proteinuria. " "Fue ingresada durante la noche por sospecha de PET y se inició tratamiento urgente." ], "offsets": [[0, 227]], } ], "entities": [ { "id": "T1", "type": "ENFERMEDAD", "text": ["hipertensión grave"], "offsets": [[45, 63]], }, { "id": "T2", "type": "ENFERMEDAD", "text": ["proteinuria"], "offsets": [[131, 142]], }, { "id": "T3", "type": "ENFERMEDAD", "text": ["PET"], "offsets": [[191, 194]], }, ], "events": [], "coreferences": [], "relations": [], } ] predictions = model.sample( bigbio_pages=bigbio_pages, num_beams=num_beams, ) for i in range(0, len(predictions), num_beams): mention = predictions[i]["mention"] print(f"## Mention {(i // num_beams) + 1}: {mention}") for j in range(num_beams): pred = predictions[i + j] print( f" - Beam {j + 1}:\n" f" Predicted concept name: {pred['pred_concept_name']}\n" f" Predicted code: {pred['pred_concept_code']}\n" f" Beam score: {pred['beam_score']:.3f}\n" ) ``` **Example Output:** ```text ## Mention 1: hipertensión grave - Beam 1: Predicted concept name: hipertensión arterial Predicted code: 38341003 Beam score: 0.993 - Beam 2: Predicted concept name: degeneración vascular hipertensiva Predicted code: 38341003 Beam score: 0.249 - Beam 3: Predicted concept name: hipertensión arterial maligna Predicted code: 70272006 Beam score: 0.046 - Beam 4: Predicted concept name: degeneración macular senil Predicted code: 267718000 Beam score: 0.004 - Beam 5: Predicted concept name: hipertensión maligna secundaria, SAI Predicted code: 194784007 Beam score: 0.001 ## Mention 2: proteinuria - Beam 1: Predicted concept name: proteinuria de causa desconocida Predicted code: 231860006 Beam score: 0.000 - Beam 2: Predicted concept name: proteína de la membrana mitocondrial asociada con neurodegeneración Predicted code: 709415008 Beam score: 0.000 - Beam 3: Predicted concept name: proteinuria aislada concomitante con glomerulonefritis membranoproliferativa tipo III y debida a ella Predicted code: 368931000119104 Beam score: 0.000 - Beam 4: Predicted concept name: proteinosis alveolar pulmonar congénita Predicted code: 707442002 Beam score: 0.000 - Beam 5: Predicted concept name: proteinosis alveolar pulmonar Predicted code: 10501004 Beam score: 0.000 ## Mention 3: PET - Beam 1: Predicted concept name: preeclampsia Predicted code: 398254007 Beam score: 0.285 - Beam 2: Predicted concept name: preeclampsia en el puerperio Predicted code: 765182005 Beam score: 0.068 - Beam 3: Predicted concept name: púrpura trombocitopénica Predicted code: 302873008 Beam score: 0.000 - Beam 4: Predicted concept name: púrpura de la vulva Predicted code: 289487000 Beam score: 0.000 - Beam 5: Predicted concept name: pústula maligna Predicted code: 84980006 Beam score: 0.000 ``` ### Saliency map example The model can also return token-level saliency maps during inference. ```python predictions, saliency_maps = model.sample( bigbio_pages=bigbio_pages, num_beams=num_beams, with_saliency_maps=True, ) model.display_saliency_map(saliency_maps[2]) ```` Example saliency map for the mention `PET`:

Saliency map for PET prediction

## Evaluation Entity linking performance is reported using Recall@1 with bootstrap confidence intervals. The best result is shown in **bold**, and the second-best result is underlined and ⭐ marks the main LongBEL-8B model. | Model | MM-ST21PV
(English) | QUAERO-EMEA
(French) | SympTEMIST
(Spanish) | DisTEMIST
(Spanish) | MedProcNER
(Spanish) | | :--- | :---: | :---: | :---: | :---: | :---: | | **Context-Free BEL** ||||| | | SciSpacy | 53.8 ± 1.0 | 37.1 ± 4.3 | 9.8 ± 1.3 | 21.1 ± 1.9 | 10.3 ± 1.2 | | SapBERT | 65.6 ± 1.0 | 59.7 ± 3.8 | 34.2 ± 2.0 | 38.6 ± 2.6 | 30.4 ± 2.1 | | CODER-all | 62.9 ± 1.1 | 66.9 ± 4.0 | 42.2 ± 2.2 | 47.0 ± 2.6 | 42.7 ± 2.1 | | SapBERT-all | 64.6 ± 1.1 | 67.9 ± 3.9 | 49.8 ± 2.4 | 49.6 ± 2.6 | 45.1 ± 2.2 | | BERGAMOT | 60.9 ± 1.1 | 63.8 ± 4.9 | 48.0 ± 2.7 | 48.9 ± 2.4 | 42.3 ± 2.2 | | **Local-Context BEL** ||||| | | ArboEL | 76.9 ± 0.9 | 63.0 ± 3.9 | 55.4 ± 2.5 | 54.7 ± 2.6 | 59.7 ± 2.6 | | GENRE / mBART-large | 69.6 ± 1.0 | 69.3 ± 5.4 | 59.8 ± 2.7 | 58.7 ± 2.7 | 66.0 ± 2.3 | | GENRE / Llama-1B | 73.1 ± 1.0 | 75.1 ± 3.6 | 60.5 ± 2.4 | 62.5 ± 2.3 | 67.4 ± 2.1 | | GENRE / Llama-8B | 75.0 ± 0.9 | 73.8 ± 4.0 | 61.7 ± 2.5 | 63.2 ± 2.5 | 68.3 ± 2.2 | | **Global-Context BEL: LongBEL** ||||| | | LongBEL-1B| 77.6 ± 0.9 | 74.5 ± 3.7 | 59.8 ± 2.5 | 61.9 ± 2.4 | 66.6 ± 2.1 | | LongBEL-1B + Ensemble | 78.6 ± 0.8 | 77.2 ± 3.0 | 61.8 ± 2.5 | 64.3 ± 2.2 | 69.0 ± 2.0 | | **⭐ LongBEL-8B** | 79.3 ± 0.8 | 75.4 ± 3.4 | 62.0 ± 2.6 | 63.6 ± 2.1 | 69.0 ± 2.1 | | LongBEL-8B + Ensemble | **80.0 ± 0.8** | **77.6 ± 3.0** | **63.3 ± 2.5** | **65.8 ± 2.2** | **71.0 ± 2.0** | The score reported for this checkpoint is the **single LongBEL-8B model**. The ensemble result requires fusing several LongBEL input configurations and is not produced by this checkpoint alone. ## Speed and Memory Measured on a single NVIDIA H100 80GB GPU. | Model | Model memory | Candidate memory | Speed | | ----------------------- | -----------: | ---------------: | --------------: | | GENRE-Llama-8B baseline | 28.6 GB | 5.4 GB | 38.2 mentions/s | | LongBEL-8B | 28.6 GB | 5.4 GB | 15.2 mentions/s | LongBEL has the same model memory footprint as the sentence-level Llama-8B baseline, but it is slower because it processes longer contexts and updates document-level memory during inference. ## Limitations This model assumes that mention spans and semantic groups are given. It does not perform mention detection. LongBEL is most useful when concepts recur within a document. When most concepts appear only once, the memory mechanism has less information to exploit. Because LongBEL uses previous predictions as memory, early mistakes can still influence later predictions. Robust memory training reduces this risk but does not remove it completely. This model is intended for research use. It should not be used for clinical decision-making without additional validation and human oversight. ## Reproducibility Code and evaluation scripts are available in this [GitHub repository](https://anonymous.4open.science/r/LongBEL-31AD). Trained model checkpoints and processed datasets are available in the anonymous Hugging Face collection associated with LongBEL.