suitch/radbert-english-ctrate-classifier

RadBERT English CTRate Classifier

A RadBERT-based multi-label classifier for predicting 18 pathology labels from English-language radiology reports.
The training data consists of English radiology reports from the CTRate dataset.

Model Details

Property	Value
Base model	RadBERT (RoBERTa-base architecture, pre-trained on radiology text)
Task	Multi-label text classification (18 labels)
Language	English (en)
Framework	🤗 Transformers + PyTorch
Problem type	multi_label_classification

Labels (18 pathologies)

ID	Label
0	Medical material
1	Arterial wall calcification
2	Cardiomegaly
3	Pericardial effusion
4	Coronary artery wall calcification
5	Hiatal hernia
6	Lymphadenopathy
7	Emphysema
8	Atelectasis
9	Lung nodule
10	Lung opacity
11	Pulmonary fibrotic sequela
12	Pleural effusion
13	Mosaic attenuation pattern
14	Peribronchial thickening
15	Consolidation
16	Bronchiectasis
17	Interlobular septal thickening

Quick Start

Installation

pip install transformers torch

Loading the model

from transformers import AutoTokenizer, AutoConfig
from modeling_radbert import RadBertForSequenceClassification
import torch

repo_id = "suitch/radbert-english-ctrate-classifier"

# Download the custom model class (or copy modeling_radbert.py locally)
from huggingface_hub import hf_hub_download
import sys, os

modeling_path = hf_hub_download(repo_id=repo_id, filename="modeling_radbert.py")
sys.path.insert(0, os.path.dirname(modeling_path))

# Load config, model, and tokenizer
config = AutoConfig.from_pretrained(repo_id)
model = RadBertForSequenceClassification.from_pretrained(repo_id, config=config)
tokenizer = AutoTokenizer.from_pretrained(repo_id)

model.eval()

Inference example

text = "The heart is mildly enlarged. A small left pleural effusion is noted."

inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=512)

with torch.no_grad():
    logits = model(**inputs)

probabilities = torch.sigmoid(logits).squeeze()
threshold = 0.5
predicted_labels = [
    config.id2label[i] for i, p in enumerate(probabilities) if p >= threshold
]

print("Predicted labels:", predicted_labels)
print("Probabilities:")
for i, p in enumerate(probabilities):
    print(f"  {config.id2label[i]}: {p:.4f}")

Training Details

• Base checkpoint: RadBERT (RoBERTa-base weights pre-trained on radiology corpora)
• Training data: English radiology reports from the CTRate dataset
• Classification head: Linear layer on top of the [CLS] / pooler output
• Loss: Binary Cross-Entropy with Logits (per-label sigmoid)

Limitations

• This model is trained for label inference from report text only — it does not process images.
• It should not be treated as a clinical decision support system.

Citation

If you use this model, please cite the CTRate dataset and RadBERT:

@article{hamamci2024ctrate,
  title={CT-RATE: A Large-Scale Computed Tomography Report-Image Dataset for AI in Radiology},
  author={Hamamci, Ibrahim Ethem and others},
  journal={arXiv preprint},
  year={2024}
}

@article{yan2022radbert,
  title={RadBERT: Adapting Transformer-based Language Models to Radiology},
  author={Yan, Di and others},
  journal={Radiology: Artificial Intelligence},
  year={2022}
}

License

MIT