README.md

Rubrics-RL: Rubric-Guided Reinforcement Learning for Medical Report Generation

This repository contains the code, training configurations, evaluation outputs, and processed data for the Rubrics-RL project (ICLR 2027 submission). The project investigates rubric-guided reinforcement learning for medical report generation, focusing on the "scissors gap" between LLM-judge rewards and verifiable faithfulness metrics.

Repository Structure

├── code/                          # All training and evaluation scripts
│   ├── audit/                     # M1 judge vulnerability audit
│   ├── m2/                        # M2-M4 closed-form VQA experiments
│   ├── m4/                        # M4 counterfactual faithfulness
│   └── m5/                        # M5-M19 open-ended report generation
│       ├── report_reward.py       # M5 reward (5 conditions)
│       ├── report_reward_m6.py    # M6 routed gate reward
│       ├── report_reward_m9.py    # M9 RadGraph dual-gate reward
│       ├── chexbert_server.py     # CheXbert lie-detector service
│       ├── radgraph_server.py     # RadGraph lie-detector service
│       ├── rubric_gen.py          # RubricHub rubric generation
│       ├── atom_router.py         # Atom routing for gated reward
│       ├── train_m5.sh            # M5 training script
│       ├── train_m6.sh            # M6 training script
│       ├── train_m9.sh            # M9 training script
│       ├── plot_m5.py             # M5 scissors gap plot
│       ├── plot_m6.py             # M6 routed gate plot
│       ├── plot_m9.py             # M9 RadGraph gate plot
│       ├── m11_calibration.py     # M11 clinical calibration
│       ├── m11_adversarial_probe.py # M11 adversarial CheXbert probe
│       ├── m13a_overcredit.py     # M13 over-credit analysis
│       └── ...
├── data/                          # Project-specific processed data
│   ├── m2_vqarad/                 # Processed VQA-RAD for M2 training
│   ├── m5_iuxray/                 # Processed IU-Xray for M5 training
│   └── new_datasets/
│       ├── CRIMSON/               # RadJudge/RadPref calibration data
│       └── TAIX-VQA/              # ICU CXR faithfulness eval
├── outputs/                       # Evaluation results, plots, caches
│   ├── *.json                     # Evaluation result JSONs
│   ├── *.png                      # Visualization plots
│   ├── *.csv                      # Per-sample results
│   ├── *_cache/                   # Judge inference caches
│   └── ...
├── logs/                          # Training logs
├── env_backup/                    # Conda environment specifications
└── README.md                      # This file

Publicly Available Datasets (not included)

The following datasets used in this project are publicly available. Download them from their respective sources:

Dataset	Access	Source	Size	Usage
VQA-RAD	OPEN	HuggingFace: flaviagiammarino/vqa-rad	34 MB	M1-M4 VQA warm-up
SLAKE	OPEN	HuggingFace: mdwiratathya/SLAKE-vqa-english (or BoKelvin/SLAKE)	<1 GB	VQA warm-up + spatial
IU-Xray	OPEN	Open-i / HuggingFace: dz-osamu/IU-Xray	~14 GB	M5-M19 main dataset
TAIX-VQA	OPEN	HuggingFace: jomoll/TAIX-VQA (images: TLAIM/TAIX-Ray)	small	ICU CXR faithfulness
CRIMSON	OPEN	GitHub: rajpurkarlab/CRIMSON	small	RadJudge/RadPref calibration
ReXGradient-160K	HF-GATED	HuggingFace: rajpurkarlab/ReXGradient-160K (accept license first)	~160 GB	Scale-up training
MIMIC-CXR	PHYSIONET	PhysioNet: mimic-cxr / mimic-cxr-jpg	~570 GB	(Future scale-up)

Download example:

# Open datasets
pip install huggingface_hub
huggingface-cli download flaviagiammarino/vqa-rad --local-dir data/vqa-rad
huggingface-cli download mdwiratathya/SLAKE-vqa-english --local-dir data/SLAKE

# IU-Xray (or use the HF mirror)
huggingface-cli download dz-osamu/IU-Xray --local-dir data/IU-Xray-RRG

# Gated dataset (accept license on HF page first)
huggingface-cli download rajpurkarlab/ReXGradient-160K --local-dir data/ReXGradient-160K --token YOUR_HF_TOKEN

Base Models (not included)

All base models are publicly available on HuggingFace:

Model	HuggingFace ID	Size
Qwen2.5-VL-3B-Instruct	Qwen/Qwen2.5-VL-3B-Instruct	7.1 GB
Qwen2.5-VL-7B-Instruct	Qwen/Qwen2.5-VL-7B-Instruct	16 GB
Qwen3-VL-4B-Instruct	Qwen/Qwen3-VL-4B-Instruct	8.3 GB
HuatuoGPT-Vision-7B	FreedomIntelligence/HuatuoGPT-Vision-7B	17 GB

Training Framework

This project uses EasyR1 (commit dd71bbd) with one compatibility patch:

# verl/workers/fsdp_workers.py — transformers>=5 compatibility
-from transformers.modeling_utils import no_init_weights
+try:
+    from transformers.modeling_utils import no_init_weights
+except ImportError:
+    from transformers.initialization import no_init_weights

External Tools

• GREEN (Grounded Radiology Report Evaluation): https://github.com/Stanford-AIMI/GREEN
• CheXbert: Used as a verifiable lie-detector (14-label chest finding classifier)
• RadGraph: Entity-relation extraction for fine-grained faithfulness verification

Experiment Overview

Milestone	Description	Key Finding
M1	Judge vulnerability audit	LLM judges exploitable via master-key attacks
M2-M4	Closed-form VQA RL	Strong judge resists short-horizon hacking; gate prototype works
M5	Open-ended report RL (5 conditions)	Scissors gap confirmed (judge ~0.5 vs CheXbert-F1 ~0.12)
M6	Routed per-atom gate	Gate preserves faithfulness; high verif_rate makes gate invisible
M7	Master-key attack (report-level)	Does not reproduce (honest negative result)
M8	2nd-seed replication	M6 conclusions seed-stable
M9	RadGraph dual-gate	Gate becomes visible — finer signal exposes cross-stream gate effect
M11	Clinical calibration	CheXbert-F1 vs radiologist ρ=0.63; dualgate fixes single-stream blindspots
M12-M19	Extended experiments	Confound controls, long training, emergent downstream harm

Environment Setup

Three conda environments were used:

• easyr1: Training (GRPO + EasyR1)
• r1: Evaluation, lie-detector services (requires nvJitLink fix)
• chx: CheXbert CPU inference

Environment specs are in env_backup/.

Gated Access

This repository requires manual approval for access. Please provide your affiliation and intended use when requesting access.

License

Research use only. Please contact the authors for commercial licensing.