mamounyosef/chest-xray-bench

Chest X-ray Bench: 43 CheXpert classifiers

Code, configs and the technical report on GitHub

43 chest X-ray classifiers, all trained on CheXpert under one fixed pipeline so their scores are directly comparable. Released alongside a technical report that compares the design choices behind them.

Most models predict five findings: Atelectasis, Cardiomegaly, Consolidation, Edema, Pleural Effusion.

---

The ensemble members

Blended together these six score 0.9130 mean AUROC on test500, the best result in the study.

Model	valid200	test500	Input size	Params
medmae_vitb_nih_B_448_s1_seed1337	0.8979	0.8935	448x384	86.2M
convnext_base_22k_1600x1312	0.8964	0.8879	1600x1312	87.6M
medmae_vitb_nih_B_768_s2_seed1337	0.8964	0.9002	768x640	87.1M
rad_dino_vitB_1064x896	0.8940	0.8993	1064x896	86.6M
medmae_vitb_nih_B_768_s2	0.8934	0.8960	768x640	87.1M
rad_dino_vitB_768	0.8929	0.9027	784x644	86.6M
All six blended		0.9130

---

Quick start

pip install torch timm transformers safetensors huggingface_hub opencv-python

Download modeling.py from this repo, then:

import cv2, torch
from modeling import load_model, preprocess

model, cfg = load_model("rad_dino_vitB_768")     # any folder name below

img = cv2.imread("frontal.jpg", cv2.IMREAD_GRAYSCALE)
x = preprocess(img, cfg)

with torch.no_grad():
    probs = model(x).sigmoid()[0]

for task, p in zip(cfg["tasks"], probs.tolist()):
    print(f"{task:18} {p:.3f}")

load_model picks the right builder for each backbone, so the same two lines work for every model here. preprocess reproduces the training pipeline: resize to fit the target box keeping the aspect ratio, zero pad the short side, normalize. Never mirror a chest X-ray at inference, it moves the heart to the wrong side.

---

What is in each folder

File	Description
model.safetensors	Weights only, fp32. No optimizer state.
config.json	Backbone, input size, normalization, label policy, head layout, scores.
thresholds.json	Per finding decision thresholds, tuned for F1 on the large validation split then frozen. Only needed for hard yes/no predictions.

A note on outputs

Most models emit one logit per finding, so sigmoid gives five probabilities. A few trained with the three way uncertainty head emit 9 logits, and the single pathology models emit 1. config.json always states which, under head:

"head": {
  "n_logits": 5,
  "layout": "one sigmoid logit per task",
  "task_slices": {"Atelectasis": [0, 1], "Cardiomegaly": [1, 2], "...": []}
}

---

How the ensemble was combined

The report's headline, 0.9130 mean AUROC on the official test set, averages the six models above in logit space, with a separate set of member weights fitted per finding on the 234 image validation split.

That is +0.0103 over the best single member, with a 95% bootstrap interval of [+0.0022, +0.0186] over 10,000 resamples. The lesson from the report: diversity beats count. Seven runs of the same backbone averaged to 0.8972, below the best single model, while three genuinely different backbones reached 0.9115.

---

All models

valid200 and test500 are the official radiologist labelled splits, 234 and 668 frontal images. Neither was trained on.

Chest X-ray pretrained

Model	Backbone	Input size	Params	valid200	test500
rad_dino_vitB_768	vit_base_patch14_dinov2 (microsoft/rad-dino)	784x644	86.6M	0.8929	0.9027
medmae_vitb_nih_B_768_s2_seed1337	vit_base_patch16_224	768x640	87.1M	0.8964	0.9002
rad_dino_vitB_1064x896	vit_base_patch14_dinov2 (microsoft/rad-dino)	1064x896	86.6M	0.8940	0.8993
medmae_vitb_nih_B_768_s2_seed7	vit_base_patch16_224	768x640	87.1M	--	0.8989
medmae_vitb_nih_B_448_s1_seed7	vit_base_patch16_224	448x384	86.2M	--	0.8969
medmae_vitb_nih	vit_base_patch16_224	384x320	86.0M	0.8891	0.8968
medmae_vitb_nih_B_768_s2	vit_base_patch16_224	768x640	87.1M	0.8934	0.8960
medmae_vitb_nih_B_448_s1_seed1337	vit_base_patch16_224	448x384	86.2M	0.8979	0.8935
medmae_vitb_raw	vit_base_patch16_224	384x320	86.0M	0.8805	0.8901
medmae_vitb_chexpert	vit_base_patch16_224	384x320	86.0M	0.8789	0.8899
medmae_vitb_nih_B_448_s1	vit_base_patch16_224	448x384	86.2M	--	--

High resolution ConvNeXt

Model	Backbone	Input size	Params	valid200	test500
convnext_base_22k_768x640	convnext_base.fb_in22k_ft_in1k_384	768x640	87.6M	0.8854	0.8929
convnext_base_22k_1600x1312	convnext_base.fb_in22k_ft_in1k_384	1600x1312	87.6M	0.8964	0.8879
convnext_large_22k_768x640	convnext_large.fb_in22k_ft_in1k_384	768x640	196.2M	0.8774	0.8767

Uncertainty and objective

Model	Backbone	Input size	Params	valid200	test500
convnext_base_22k_final_stage1	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8808	0.8835
convnext_base_22k_aucm	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8750	0.8698
densenet121_cons_zeros	densenet121	384x320	7.0M	0.8451	0.8634
densenet121_u_mixed	densenet121	384x320	7.0M	0.8411	0.8576
resnet50_u_zeros	resnet50	384x320	23.6M	0.8307	0.8521

Single pathology

Model	Backbone	Input size	Params	valid200	test500
convnext_base_22k_final_stage2_pleural_effusion	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.9012	0.9374
convnext_base_22k_final_stage2_edema	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.9248	0.9236
convnext_base_22k_final_stage2_cardiomegaly	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8192	0.8722
convnext_base_22k_consolidation_only	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8877	0.8721
convnext_base_22k_cardiomegaly_only	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8055	0.8571
convnext_base_22k_final_stage2_consolidation	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.9056	0.8543
convnext_base_22k_final_stage2_atelectasis	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8451	0.8295
convnext_base_22k_atelectasis_only	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8365	0.8279

Backbone comparison

Model	Backbone	Input size	Params	valid200	test500
convnext_base_22k_seed1337	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8780	0.8849
convnext_base_22k_cxr14_pretrain_lowlr	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8747	0.8813
convnext_base_22k_seed7	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8809	0.8797
convnext_tiny	convnext_tiny	384x320	27.8M	0.8671	0.8769
convnext_base_22k_cxr14_pretrain_lowlr_all	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8736	0.8768
convnext_large_22k_cxr14_pretrain	convnext_large.fb_in22k_ft_in1k_384	384x320	196.2M	0.8695	0.8699
convnext_small	convnext_small	384x320	49.5M	0.8535	0.8685
convnext_base_22k_cxr14_pretrain	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8628	0.8659
convnext_large_22k	convnext_large.fb_in22k_ft_in1k_384	384x320	196.2M	0.8519	0.8650
convnext_base_22k	convnext_base.fb_in22k_ft_in1k_384	384x320	87.6M	0.8743	0.8643
densenet121_seed7	densenet121	384x320	7.0M	0.8533	0.8619
densenet121	densenet121	384x320	7.0M	0.8566	0.8590
densenet201	densenet201	384x320	18.3M	0.8540	0.8546
densenet121_seed123	densenet121	384x320	7.0M	0.8461	0.8501
resnet50_without_clahe	resnet50	384x320	23.6M	0.8381	0.8453
resnet50_with_clahe	resnet50	384x320	23.6M	0.8390	0.8448

---

Training setup

Data	CheXpert train split, frontal views only, split 90/10 by patient
Loss	Binary cross entropy over the five findings, masked where a target is undefined
Optimizer	AdamW, batch 64, cosine schedule with a one epoch warmup
Augmentation	Rotation, translation, scale, brightness, contrast. No horizontal flip
Metric	Mean AUROC over the five findings, scored per image

Full configurations, training code and the report are on GitHub.

---

Intended use and limits

These are research artifacts, released to support a technical report.

• Not a medical device. Do not use them to make clinical decisions.
• No external validation. Every number here comes from CheXpert's own splits. Performance on images from other hospitals, scanners or populations is unknown.
• The test sets are small. 234 and 668 images, so per model rankings are unstable and confidence intervals are wide.
• Labels come from an automatic labeler applied to radiology reports, so the models learn that labeler's conventions along with the findings.

License and data

CC BY-NC 4.0: free to use, share and build on with attribution, non-commercial only. This matches CheXpert's Stanford University Dataset Research Use Agreement, which permits research use and forbids commercial use.

The CheXpert data is not redistributed here, in this repo or on GitHub. Request it from Stanford AIMI directly.

Citation

@techreport{yosef2026chexpert,
  title       = {A Systematic Study of Design Choices for Multi-Label Chest X-ray Classification on CheXpert},
  author      = {Yosef, Ma'moun},
  year        = {2026},
  institution = {University of Jordan}
}