Vydiant/mesh-pipeline

This BiEncoder is designed with the specific purpose of normalizing annotations generated by the datummd/NCBI_BC5CDR_disease model to their corresponding Medical Subject Headings (MeSH) term, excluding branches C and F. Please download the mesh_embeddings file under Files and versions. These embeddings correspond to the [CLS] embeddings for MeSH terms. Load and set the file as the mesh_ordered_dictionary.

This code snippet demonstrates the processing of text data using various pipelines and transformers. It involves extracting disease-related information from a corpus of text and then normalizing the annotations.

Corpus

Here's the text corpus used in the code:

corpus = [
    'They support the hypothesis that short-term aerobic exercise interventions can act as a buffer against depression and perceived stress in university students after 6 weeks of aerobic exercise of low to moderate intensity.',
    'RESULTS: Vigorous physical activity, self-reported stress, a type A behavior pattern, and less frequent intakes of green-yellow vegetables were significantly associated with an increased risk of amyotrophic lateral sclerosis, whereas smoking and drinking habits were not.',
    'Benfotiamine decreased oxidative stress and inflammation, two major characteristics of neurodegenerative diseases, in a neuroblastoma cell line (Neuro2a) and an immortalized brain microglial cell line (BV2).',
    'Vitamin C Attenuates Oxidative Stress and Behavioral Abnormalities Triggered by Fipronil and Pyriproxyfen Insecticide Chronic Exposure on Zebrafish Juveniles.',
    'Synthesis suggests beneficial associations between green space exposure and reduced stress, positive mood, fewer depressive symptoms, better emotional well-being, improved mental health and behavior, and decreased psychological distress in adolescents.'
]

Annotation Pipeline

We use the BertForTokenClassification model by datummd/NCBI_BC5CDR_disease to create an annotation pipeline for disease-related information. Here's how it's done:

from transformers import pipeline

annotation_pipeline = pipeline(
    model="datummd/NCBI_BC5CDR_disease",
    tokenizer="datummd/NCBI_BC5CDR_disease",
    aggregation_strategy='average',
    device=0
)

annotations = annotation_pipeline(corpus)

Extracting Annotations

Annotations with a confidence score greater than 0.75 are extracted and stored in a list. The code snippet for this is as follows:

annotations = [
    {'text': text, 'start': span['start'], 'end': span['end']}
    for text, spans in zip(corpus, annotations)
    for span in spans if span['score'] > 0.75
]

Normalizing Annotations

We utilize this pipeline for mapping the annotations to their respective MeSH term. The embeddings for each MeSH term can be downloaded under "Files and versions" and then used for normalizing the annotations:

import torch

mesh_ordered_dict = torch.load('/path/to/mesh-pipeline/mesh_embeddings.pt')

mesh_pipeline = pipeline(
    model="Vydiant/mesh-pipeline",
    trust_remote_code=True,
    device=0,
    mesh_ordered_dictionary=mesh_ordered_dict
)

normalized_annotations = mesh_pipeline(annotations)

The outputs will be the similiarity scores of each MeSH term with respective to the given annotation and the MeSH record with the highest score.