README.md

metadata

title: CXR Draft Auditor
emoji: 🩻
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 6.18.0
app_file: app.py
python_version: '3.12'
license: apache-2.0
short_description: Research QA for chest X-ray draft impressions (not a device)
pinned: false
startup_duration_timeout: 1h
tags:
  - track:backyard
  - sponsor:nvidia
  - achievement:offgrid
  - achievement:welltuned
  - achievement:offbrand
  - achievement:sharing
  - achievement:fieldnotes
  - achievement:tinytitan
  - achievement:bestdemo

CXR Draft Auditor

RESEARCH / EDUCATIONAL QA ONLY. This project is NOT a medical device, NOT a diagnostic tool, and NOT a substitute for a qualified radiologist. It must NEVER be used for clinical decision-making, screening, or patient care. See DISCLAIMER.md.

The idea

I wanted an auditor, not a report generator. There are already plenty of demos that look at a chest X-ray and write a radiology report; I find that the wrong shape for a small model, because a model that confidently writes prose is hard to trust and hard to check. So I built the inverse: you give it the image AND the human draft impression, and it tells you where the two appear to disagree, with image evidence attached. The point is the audit loop, not a verdict. It surfaces apparent MISSING findings, UNSUPPORTED claims, and URGENT review flags, then shows the bounding-box evidence so a person can look again and decide for themselves.

This framing is what keeps the task small enough for small models. I never ask one model to do everything. I decompose the problem into three narrow jobs, two of which are perception and one of which is pure logic.

How it works

1. Image to grounded findings. A fine-tuned MedGemma vision-language model runs on the GPU and emits a constrained JSON list of findings over a fixed six-finding label space, each with a normalized bounding box. This is the only model that looks at pixels.
2. Draft to labels. NVIDIA Nemotron-3 Nano 4B, run on the GPU through transformers, parses the draft impression into the same six-finding label space, marking each label asserted or denied and keeping the verbatim draft span. It reasons briefly before answering (reasoning materially improves its extraction accuracy) and that reasoning trace is stripped before the labels are read. For example, "Cardiomegaly is present. No pneumothorax." parses to cardiomegaly present and pneumothorax explicitly denied, each tied to the phrase it came from. If the draft cannot be parsed, the audit degrades to an image-only pass with a visible note.
3. Deterministic comparison. A pure-logic comparator (no model) compares the two label sets and flags:
   • MISSING: present in the image findings, absent or denied in the draft.
   • UNSUPPORTED: asserted in the draft, absent from the image findings.
   • URGENT: any flagged finding on the urgent whitelist (pneumothorax and nodule/mass) is surfaced for radiologist review.

A full audit takes well under a minute, roughly 15 to 30 seconds: MedGemma grounding and the Nemotron draft parse both run on the GPU. The only judgment layer is the deterministic comparator, so every flag is auditable: each one ties an image finding (with a box) and a draft phrase (with a verbatim span) to a single comparator decision.

Why two perception models

My first instinct was to make the fine-tuned MedGemma do both jobs, image grounding and draft parsing. It grounds the image well, but once it has been fine-tuned for grounding it reads free-text drafts unreliably, especially denials like "no pneumothorax." Rather than fight that with prompt tricks, I gave the text job to a model built for it. Nemotron-3 Nano 4B is a small instruction-following model that parses the draft into the canonical labels cleanly, including denials with their verbatim spans. Two small specialized models plus a deterministic comparator beat one model stretched across two jobs, and the whole thing still fits on free hosted hardware.

Canonical finding set

pleural_effusion, pneumothorax, lung_opacity_consolidation, nodule_mass, cardiomegaly, no_finding.

Evaluation

On a held-out set of 273 images, scored with the production draft parser, presence reaches a macro-F1 of 0.735; localization reaches an IoU@0.3 hit rate of 0.63 at precision 0.79, with a mean IoU of 0.70 over matched boxes. Urgent recall is small-sample and weak: nodule/mass 4 of 9, pneumothorax 1 of 1. Those urgent counts are tiny, and I report them as-is rather than rounding them into a story. This tool is research and educational only. It is frequently wrong, it is not a medical device, and the absence of an URGENT flag never means an image is safe.

How each track and badge is earned

• Backyard AI (track:backyard): I built this for a real backyard problem, radiology draft quality assurance. A radiologist I know needed a reliable second read on chest X-ray draft impressions; he tried the live demo and shared his view (see below).
• NVIDIA (sponsor:nvidia): NVIDIA Nemotron-3 Nano 4B is core to the experience; it is the model that parses the draft impression into labels and denials.
• Off the Grid (achievement:offgrid): the audit calls no external cloud inference APIs. Both models, the fine-tuned MedGemma grounding model and NVIDIA Nemotron-3 Nano 4B, run inside the Space itself on hosted ZeroGPU through transformers, so the whole pipeline runs on the models in front of you rather than calling out to a third-party API.
• Well-Tuned (achievement:welltuned): the grounding model is a MedGemma I fine-tuned for CXR findings, published on the Hugging Face Hub (see Links).
• Off-Brand (achievement:offbrand): the app wears a custom "Reading Room / Clinical Light" interface with a light and dark toggle, not the stock Gradio look.
• Sharing is Caring (achievement:sharing): I published a small open dataset of real audit traces from this Space (see Links).
• Field Notes (achievement:fieldnotes): I wrote up what I learned, including an evaluation-integrity lesson, as a blog post (see Links).
• Tiny Titan (achievement:tinytitan): every model the app runs is genuinely tiny; the fine-tuned MedGemma grounding model and NVIDIA Nemotron-3 Nano 4B are each 4B parameters, within the badge's 4B limit, with no larger model anywhere in the stack.
• Best Demo (achievement:bestdemo): the submission is the full package; a custom Reading Room app, a demo video that walks through an audit end to end, and a social post (see Links).

Models

Grounding model (image to findings): a fine-tuned google/medgemma-1.5-4b-it (Health AI Developer Foundations license). MedGemma natively emits CXR bounding boxes as a JSON list of {label, box_2d: [y0, x0, y1, x1]} normalized to [0, 1] with (y0, x0) top-left and (y1, x1) bottom-right; the canonical box-format constant is normalized_y0x0y1x1. It runs on the GPU in bf16. The fine-tuned weights are a MedGemma Model Derivative governed by the HAI-DEF Terms of Use, not Apache-2.0; the license: apache-2.0 field above covers only the Space's own application code and does not relicense the model weights.

Draft parser (draft to labels): nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 (NVIDIA Nemotron Open Model License), run on the GPU through transformers in bf16. Its native nemotron_h (Mamba2-Transformer hybrid) architecture is supported by transformers directly, so the Space needs no extra runtime and no CUDA build. The model reasons before emitting the label JSON, which materially improves extraction accuracy on multi-clause drafts; the reasoning trace is stripped before the labels are parsed.

Data and licensing

The example images shipped with the Space are open NIH ChestX-ray14 images (via natealberti/ChestX-Det). The fine-tuning corpus is built on VinDr-CXR, whose pixels are licensed for non-commercial research; that corpus is private and the VinDr images are not redistributed here. The Nemotron Open Model License is commercial-use ready. Read and accept every applicable model and dataset license before using any of them.

Links

• Served grounding model: https://huggingface.co/alex-feeel/medgemma-cxr-auditor-v2
• Draft parser (text model): https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16
• Open audit-trace dataset (Sharing is Caring): https://huggingface.co/datasets/build-small-hackathon/cxr-draft-auditor-traces
• Demo video: https://youtu.be/g52GIp4yBnQ
• Field Notes blog post: https://huggingface.co/blog/build-small-hackathon/chest-x-ray-draft-auditor
• Social post (X): https://x.com/alexfeeel/status/2066005144567771376?s=20
• Social post (LinkedIn): https://www.linkedin.com/posts/activity-7471769538703835137-a_qq

Where this started: a radiologist's need

This grew from a real need described by a pediatric radiologist I know, Alexey Amelin (https://vk.ru/xraydiag): under real reading volume, fatigue, and time pressure, a calm second read is genuinely valuable, with the radiologist always in the loop and always the one who decides. He has tried the tool himself and sees it as exactly that kind of helper, a research and educational quality-assurance aid, never a diagnosis. As a mostly-pediatric radiologist he also cautions that AI is far less validated for children than for adults (models trained on adult data can show clinically meaningful age-related bias in children), so it must be independently validated and recalibrated on pediatric data before any pediatric use, and the final word always rests with a qualified radiologist. His full reflection, in his own words, is in the Field Notes blog post.

Built by Aleksandr Filippov for the Build Small Hackathon (June 2026).