hitheshkaranth/skanda-grounding

Skanda

Reads chest X-ray, CT and MRI, outlines every finding, and says what is inside each outline

This branch (main): run 3, the best CT and MRI checkpoint

Results · By disease · Files · Memory

Research use only. Skanda drafts findings for a radiologist to review. It is not a medical device and has not been cleared or approved for clinical use by any regulator.

✨ At a glance

72% CT finding named right MedGemma 27B: 49%

89% MRI finding named right MedGemma 27B: 42%

91% · 94% lesion-free CT · MRI cleared MedGemma 27B: 6% · 2%

4–12× box precision vs MedGemma on every modality

Branch	Checkpoint	Best at
🟦 main	Run 3 · 1 Nov 2026	CT and MRI: finding type, normals cleared, outline recall · X-ray finding type
⬜ run2	Run 2 · Oct 2026	Chest X-ray normals cleared · X-ray box precision and recall

---

🏁 Results

All numbers are percentages on held-out test cases. No test patient was seen in training.

	🩻 Chest X-ray · 111 films
Metric	MedGemma 27B	MedGemma + MAIRA-2	Skanda run 2	Skanda run 3
Abnormal flagged	90	91	85	84
Normals called normal	72	72	83	69
Right finding type	41	47	37	40.5
Box precision	6	–	28	25.5
Box recall	4	–	15.5	10
	🫁 CT · 340 cases
Abnormal flagged	97	–	80	82
Normals called normal	6	–	88	91
Right finding type	49	–	69	72
Box precision	1	–	20	22
Box recall	3	–	14	16
	🧠 MRI · 240 cases
Abnormal flagged	99	–	89	89
Normals called normal	2	–	97	94
Right finding type	42	–	87	89
Box precision	6	–	64	70
Box recall	8	–	59	65

_{Percent, higher is better; bold = best per row. MAIRA-2 reads chest X-rays only, so it has no CT or MRI numbers.}

💡 What the numbers say

• 🫁🧠 On CT and MRI, Skanda is well ahead of MedGemma.
  • It names the right finding 72% / 89% of the time, against 49% / 42% for MedGemma 27B.
  • It clears 91% / 94% of lesion-free cases, where MedGemma clears 6% / 2%: it calls nearly every case abnormal.
• 🩻 On chest X-ray, run 3 is a trade-off.
  • Finding type rose from 37% to 40.5%, but it clears fewer normals than run 2 (69% vs 83%) and finds fewer boxes (recall 10% vs 15.5%).
  • Each X-ray number moves about ±10 points by chance across 111 films. On the 106 dev films the same checkpoint cleared 74% of normals at 15.5% recall.
• 🎯 Skanda outlines; the baselines don't. MedGemma and MAIRA-2 give boxes only as free-text coordinates, and Skanda's box precision is 4–12× MedGemma's.

📏 How it is measured

Metric	Meaning
Abnormal flagged	abnormal cases the model calls abnormal
Normals called normal	normal or lesion-free cases the model clears
Right finding type	abnormal cases where the model names a ground-truth finding class
Box precision / recall	X-ray: a predicted box counts when its centre lies inside a radiologist box of the same class. CT/MRI: box IoU ≥ 0.3 with a radiologist outline, matched one-to-one

• Chest X-ray: ChestX-Det test films, scored by the Skanda Radiology app's own benchmark. The same runner is used for every model. Skanda's numbers are the mean of two passes.
• CT and MRI: a fixed sample of held-out slices and volumes, with and without lesions. One script scores every model.

🦠 Detection by disease

How often each model names each kind of finding, on the same held-out cases as the table above.

	Skanda leads	MedGemma leads
🩻 X-ray	consolidation 78% vs 69% · effusion 64% vs 30% (53% fused) · cardiomegaly 75% vs 25% · diffuse nodule 50% vs 0%	pneumothorax 100% vs 15% · atelectasis · nodule · mass · fibrosis · emphysema
🫁 CT	lung 92% vs 76% · liver 74% vs 57% · pancreas 46% vs 0% · mediastinum 69% vs 38%	colon 20% vs 13%
🧠 MRI	brain tumour 96% vs 74% · acute stroke 47 of 60 found vs 0	–

_{Groups under about 15 cases (kidney, mediastinum, colon, most rare X-ray labels) are directional only.}

🔍 Side by side

📦 Files and loading

File	What it is
vlm/	Merged Qwen3-VL-30B-A3B (bf16 safetensors, 17 shards, LoRA merged) with tokenizer, processor and chat template
skanda_heads.pt	SAM 2.1 (image encoder and trained mask decoder), the [SEG] projector and the score head
skanda_config.json	[SEG] token id 151669, training arguments, calibrated gate thresholds per modality, hint thresholds
hint_scores.jsonl	Score-head outputs the hint thresholds were chosen on (validation and dev only, never test)

About 59 GB in total; see memory and quantization. The inference code lives in the private GitHub repo Hitheshkaranth/skanda-grounding.

huggingface-cli download hitheshkaranth/skanda-grounding --local-dir skanda-r3                  # main = run 3
huggingface-cli download hitheshkaranth/skanda-grounding --revision run2 --local-dir skanda-r2  # run 2

# in a checkout of skanda-grounding
from PIL import Image
from skanda.serve import Skanda

sk = Skanda("skanda-r3")                                       # vlm/ + skanda_heads.pt + skanda_config.json
report, raw_text, finished, n_tokens = sk.read(Image.open("chest.png").convert("RGB"))

python -m skanda.serve --model skanda-r3 --port 1234   # the HTTP server the Skanda Radiology app calls

💾 Memory and quantization

Memory (bf16, measured)

	Size	Notes
Download / disk	59 GB	vlm/ 58.5 GB + skanda_heads.pt 0.46 GB
Inference, steady	~68 GB	chest X-ray, CT/MRI slices and 32-slice volumes alike; one read at a time
Inference, peak	~93 GB	while loading the checkpoint and on the largest volumes

_{Measured on an NVIDIA DGX Spark (GB10, 128 GB unified memory) from node memory during run 3's test evaluation and app benchmark, minus the idle baseline. The server runs in a container capped at 112 GB.}

Hardware	Fits?
DGX Spark / GB10, 128 GB unified	✅ tested: everything here was trained and benchmarked on one
1 × 80 GB GPU (A100 / H100 80 GB)	⚠️ expected to fit: 62 GB of weights + activations; not tested
2 × 48 GB GPUs (device_map="auto")	⚠️ expected to fit; not tested
24–48 GB GPUs	❌ not at bf16; needs a quantized build (below)

Quantization

Quantized builds of the newer checkpoint, Skanda V1, are in hitheshkaranth/Skanda-V1-int8 and hitheshkaranth/Skanda-V1-int4, with measured memory and benchmarks.

---

_{Skanda · AI for healthier tomorrows}