|
|
--- |
|
|
language: |
|
|
- en |
|
|
library_name: transformers |
|
|
metrics: |
|
|
- f1 |
|
|
- accuracy |
|
|
- recall |
|
|
- precision |
|
|
pipeline_tag: text-classification |
|
|
widget: |
|
|
- text: The past 25 years have seen a strong increase in the number of publications |
|
|
related to criticality in different areas of neuroscience. The potential of criticality |
|
|
to explain various brain properties, including optimal information processing, |
|
|
has made it an increasingly exciting area of investigation for neuroscientists. |
|
|
Recent reviews on this topic, sometimes termed brain criticality, make brief mention |
|
|
of clinical applications of these findings to several neurological disorders such |
|
|
as epilepsy, neurodegenerative disease, and neonatal hypoxia. Other clinicallyrelevant |
|
|
domains - including anesthesia, sleep medicine, developmental-behavioral pediatrics, |
|
|
and psychiatry - are seldom discussed in review papers of brain criticality. Thorough |
|
|
assessments of these application areas and their relevance for clinicians have |
|
|
also yet to be published. In this scoping review, studies of brain criticality |
|
|
involving human data of all ages are evaluated for their current and future clinical |
|
|
relevance. To make the results of these studies understandable to a more clinical |
|
|
audience, a review of the key concepts behind criticality (e.g., phase transitions, |
|
|
long-range temporal correlation, self-organized criticality, power laws, branching |
|
|
processes) precedes the discussion of human clinical studies. Open questions and |
|
|
forthcoming areas of investigation are also considered. |
|
|
base_model: arazd/MIReAD |
|
|
--- |
|
|
|
|
|
# MIReAD Neuro |
|
|
|
|
|
This model is a fine-tuned version of [arazd/MIReAD](https://huggingface.co/arazd/MIReAD) on a dataset of Neuroscience papers from 200 journals collected from various sources for a journal classification task. |
|
|
It achieves the following results on the evaluation set: |
|
|
|
|
|
- Loss: 2.7117 |
|
|
- Accuracy: 0.4011 |
|
|
- F1: 0.3962 |
|
|
- Precision: 0.4066 |
|
|
- Recall: 0.3999 |
|
|
|
|
|
|
|
|
## Model description |
|
|
|
|
|
This model was trained on a journal classification task. |
|
|
|
|
|
## Intended uses & limitations |
|
|
|
|
|
The intended use of this model is to create abstract embeddings for semantic similarity search for neuroscience-related articles. |
|
|
|
|
|
## Model Usage |
|
|
|
|
|
To load the model: |
|
|
|
|
|
```py |
|
|
from transformers import BertForSequenceClassification, AutoTokenizer |
|
|
model_path = "biodatlab/MIReAD-Neuro" |
|
|
model = BertForSequenceClassification.from_pretrained(model_path) |
|
|
tokenizer = AutoTokenizer.from_pretrained(model_path) |
|
|
``` |
|
|
|
|
|
To create embeddings and for classification: |
|
|
|
|
|
```py |
|
|
# sample abstract & title text |
|
|
title = "Why Brain Criticality Is Clinically Relevant: A Scoping Review." |
|
|
abstract = "The past 25 years have seen a strong increase in the number of publications related to criticality in different areas of neuroscience..." |
|
|
text = title + tokenizer.sep_token + abstract |
|
|
tokens = tokenizer( |
|
|
text, |
|
|
max_length=512, |
|
|
padding=True, |
|
|
truncation=True, |
|
|
return_tensors="pt" |
|
|
) |
|
|
|
|
|
# to generate an embedding from a given title and abstract |
|
|
with torch.no_grad(): |
|
|
output = model.bert(**tokens) |
|
|
embedding = output.last_hidden_state[:, 0, :] |
|
|
|
|
|
# to classify (200 journals) a given title and abstract |
|
|
output = model(**tokens) |
|
|
class = output.logits |
|
|
``` |
|
|
|
|
|
## Training procedure |
|
|
|
|
|
### Training hyperparameters |
|
|
|
|
|
The following hyperparameters were used during training: |
|
|
- learning_rate: 3e-05 |
|
|
- train_batch_size: 16 |
|
|
- eval_batch_size: 16 |
|
|
- num_epochs: 6 |
