README.md

---
license: mit
library_name: transformers
pipeline_tag: text-classification
tags:
- radiology
- roberta
- text-classification
---

# RadBERT-CT

Custom RadBERT sequence-classification model converted from a training checkpoint with:
- backbone initialized from `zzxslp/RadBERT-RoBERTa-4m`
- Finetuned on CT-RATE reports in the paper **"Generalist foundation models from a multimodal dataset for 3D computed tomography"**
- Number of labels: `18`

## Load Model and Tokenizer

```python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

repo_id = "IAMJB/RadBERT-CT"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForSequenceClassification.from_pretrained(repo_id)
model.eval()
```

## Get Logits + Predicted Positive Class

```python
import torch

texts = [
    "No acute cardiopulmonary abnormality.",
    "Right lower lobe opacity, suspicious for pneumonia."
]

inputs = tokenizer(
    texts,
    padding=True,
    truncation=True,
    max_length=512,
    return_tensors="pt",
)

with torch.no_grad():
    outputs = model(**inputs)
    logits = outputs.logits
    probs = torch.sigmoid(logits)
    pred_mask = probs > 0.5

print("logits:", logits)  
print("logits shape:", logits.shape)  
print("probs over 0.5:", probs > 0.5)      # [batch_size, num_labels]
print("pred label mask:", pred_mask.tolist())
print(
    "pred label indices:",
    [[i for i, on in enumerate(row) if on] for row in pred_mask.tolist()],
)
```


## Citation

```bibtex
@article{Hamamci2026Generalist,
  author    = {Hamamci, Ibrahim Ethem and Er, Selim and Wang, Chen and others},
  title     = {Generalist foundation models from a multimodal dataset for 3D computed tomography},
  journal   = {Nature Biomedical Engineering},
  year      = {2026},
  month     = feb,
  day       = {12},
  doi       = {10.1038/s41551-025-01599-y},
  url       = {https://doi.org/10.1038/s41551-025-01599-y},
  publisher = {Springer Nature}
}
```

Metric available in **RadEval**
```bibtex
@inproceedings{xu-etal-2025-radeval,
    title = "{R}ad{E}val: A framework for radiology text evaluation",
    author = "Xu, Justin  and
      Zhang, Xi  and
      Abderezaei, Javid  and
      Bauml, Julie  and
      Boodoo, Roger  and
      Haghighi, Fatemeh  and
      Ganjizadeh, Ali  and
      Brattain, Eric  and
      Van Veen, Dave  and
      Meng, Zaiqiao  and
      Eyre, David W  and
      Delbrouck, Jean-Benoit",
    editor = {Habernal, Ivan  and
      Schulam, Peter  and
      Tiedemann, J{\"o}rg},
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.emnlp-demos.40/",
    doi = "10.18653/v1/2025.emnlp-demos.40",
    pages = "546--557",
    ISBN = "979-8-89176-334-0",
}
```