README.md

metadata

title: GazeCorrect
emoji: 👁️
colorFrom: blue
colorTo: red
sdk: gradio
sdk_version: 4.44.1
app_file: app.py
pinned: false
license: mit
short_description: Gaze-guided Gaussian-noise image regeneration
hf_oauth: true
hf_oauth_scopes:
  - gated-repos

GazeCorrect — Gaze-Guided Regeneration

Interactive demo that converts clinician gaze into a duration-weighted Gaussian attention map. It previews Gaussian noise in the attended region, then uses a text description to generate a chest X-ray with RoentGen-v2 and composites its corresponding region into the uploaded image. No DINOv3 or segmentation pipeline is used.

How to use

1. Upload a chest X-ray.
2. Click gaze points or upload a CSV containing x,y,duration.
3. Enter a disease or radiology description.
4. Press Generate to receive the gaze attention map, attention-weighted Gaussian-noise image, and corrected regenerated image.

Synthetic chest X-ray correction

Install the additional inference dependencies:

pip install -U diffusers transformers accelerate

RoentGen-v2 is a gated text-to-image model. Before using the correction button, sign in to Hugging Face and accept the model's access conditions. The app uses its supported DiffusionPipeline inference path, then applies the gaze mask when compositing the generated CXR region. The model and its outputs are for research/education only, never for clinical diagnosis.

Deploying on Hugging Face Spaces

The Space must have GPU hardware selected. In Settings → Secrets, add an HF_TOKEN secret created by the Hugging Face account that accepted access to stanfordmimi/RoentGen-v2. The token is read server-side and is never shown in the web interface. The first correction request downloads the model, so it can take several minutes; the interface displays the attention-weighted Gaussian noise preview immediately if RoentGen cannot start.

For a ZeroGPU Space, this application already decorates the segmentation and correction callbacks with @spaces.GPU. Keep app.py unchanged at the top level; removing those decorators causes the ZeroGPU startup error “No @spaces.GPU function detected”.

Notes

• RoentGen-v2 is trained for chest X-rays; do not use this workflow for other medical modalities.
• The first generation downloads the model. A GPU or ZeroGPU Space is recommended.
• This Space is for research/demonstration only — it is not a clinical diagnostic tool.

Full code, configs, and the unified evaluation pipeline: GitHub repository.