metadata
language:
- en
license: apache-2.0
library_name: transformers
tags:
- chest X-ray report generation
- radiology report generation
- image captioning
- chest X-ray
- X-ray
- radiology
- cxrmate
- cxrmate-2
- cxrmate-ed
- cxrmate-rrg24
- report
- radiology report
- multimodal
- patient data
- mimic-cxr
CXRMate-2 — Chest X-ray Radiology Report Generation
From: Toward Clinically Acceptable Chest X-ray Report Generation: A Retrospective Pilot Study
The pre-print for CXRMate-2 is available at:
Coming soon...
Abstract from the paper:
Coming soon...
Load CXRMate-2:
alias = 'aehrc/cxrmate-2'
model = transformers.AutoModelForCausalLM.from_pretrained(alias, trust_remote_code=True).to(device='cuda')
model.eval()
generation_config = transformers.GenerationConfig.from_pretrained(alias, trust_remote_code=True)
processor = transformers.AutoProcessor.from_pretrained(alias, trust_remote_code=True)
Generate from URL:
url = 'https://prod-images-static.radiopaedia.org/images/220869/76052f7902246ff862f52f5d3cd9cd_big_gallery.jpg'
display(Image(url=url))
processed = processor(images=url)
processed = processed.to(device='cuda')
generated_ids = model.generate(**processed, generation_config=generation_config)
findings, impression = processor.split_and_decode_sections(generated_ids)
print(f'Findings:\t{findings[0]}\nImpression:\t{impression[0]}')
Generate from study DICOMs:
dcm_path = [
'./physionet.org/files/mimic-cxr/2.0.0/files/p12/p12000264/s55271473/522f9570-7cb12ecb-6327c8b8-b248b4be-58bb3dfd.dcm',
'.//datasets/work/hb-mlaifsp-mm/work/repositories/25_cxrmate2/work/data/physionet.org/files/mimic-cxr/2.0.0/files/p12/p12000264/s55271473/d0b61aff-f64c4ecf-85fae310-43668cf1-0c1d4c2d.dcm',
]
processed = processor(images=dcm_path)
processed = processed.to(device='cuda')
generated_ids = model.generate(**processed, generation_config=generation_config)
findings, impression = processor.split_and_decode_sections(generated_ids)
print(f'Findings:\t{findings[0]}\nImpression:\t{impression[0]}')
Generate from ReXgradient HuggingFace dataset:
Scripts to create the ReXgradient, CheXpert Plus, and MIMC-CXR Hugging Face datasets will be released soon
# Run prepare_chexpert_plus.py or prepare_mimic_cxr_jpg.py or prepare_rexgradient.py to create dataset:
test_set = datasets.load_from_disk('/scratch3/nic261/database/cxrmate2/rexgradient_160k_dataset')['test']
# Wrap dataset to get priors:
test_set = processor.wrap_dataset(test_set)
random_idx = random.randint(0, len(test_set) - 1)
example = test_set[random_idx]
processed = processor(
images=example['images'], # This includes both the current and prior images.
image_datetime=example['image_datetime'], # This includes the datetimes for both the current and prior images.
views=example['views'], # This includes both the current and prior views.
indication=example.get('indication', None),
history=example.get('history', None),
comparison=example.get('comparison', None),
technique=example.get('technique', None),
study_datetime=example.get('study_datetime', None),
prior_findings=example.get('prior_findings', None),
prior_impression=example.get('prior_impression', None),
prior_study_datetime=example.get('prior_study_datetime', None),
)
processed = processed.to(device='cuda')
generated_ids = model.generate(**processed, generation_config=generation_config)
findings, impression = processor.split_and_decode_sections(generated_ids)
print(f'Findings:\t{findings[0]}\nImpression:\t{impression[0]}')
Environment requirements:
Coming soon...
Training:
Coming soon...