README.md

---

title: GazeCorrect
emoji: 👁️
colorFrom: blue
colorTo: red
sdk: gradio
sdk_version: 4.44.1
app_file: app.py
pinned: false
license: mit
short_description: Gaze-guided Gaussian-noise image regeneration
hf_oauth: true
hf_oauth_scopes:
  - gated-repos
---


# GazeCorrect — Gaze-Guided Regeneration

Interactive demo that converts clinician gaze into a duration-weighted Gaussian
attention map. It previews Gaussian noise in the attended region, then uses a
text description to generate a chest X-ray with RoentGen-v2 and composites its
corresponding region into the uploaded image. No DINOv3 or segmentation
pipeline is used.

## How to use
1. Upload a chest X-ray.
2. Click gaze points or upload a CSV containing `x,y,duration`.
3. Enter a disease or radiology description.
4. Press **Generate** to receive the gaze attention map, attention-weighted
   Gaussian-noise image, and corrected regenerated image.

## Synthetic chest X-ray correction

Install the additional inference dependencies:

```bash
pip install -U diffusers transformers accelerate
```

RoentGen-v2 is a gated text-to-image model. Before using the correction
button, sign in to Hugging Face and accept the model's access conditions. The
app uses its supported `DiffusionPipeline` inference path, then applies the
gaze mask when compositing the generated CXR region. The model and its outputs
are for research/education only, never for clinical diagnosis.

### Deploying on Hugging Face Spaces

The Space must have GPU hardware selected. In **Settings → Secrets**, add an
`HF_TOKEN` secret created by the Hugging Face account that accepted access to
`stanfordmimi/RoentGen-v2`. The token is read server-side and is never shown in
the web interface. The first correction request downloads the model, so it can
take several minutes; the interface displays the attention-weighted Gaussian
noise preview immediately if RoentGen cannot start.

For a **ZeroGPU** Space, this application already decorates the segmentation
and correction callbacks with `@spaces.GPU`. Keep `app.py` unchanged at the
top level; removing those decorators causes the ZeroGPU startup error
“No @spaces.GPU function detected”.

## Notes
- RoentGen-v2 is trained for chest X-rays; do not use this workflow for other
  medical modalities.
- The first generation downloads the model. A GPU or ZeroGPU Space is
  recommended.
- This Space is for research/demonstration only — it is **not** a clinical
  diagnostic tool.

Full code, configs, and the unified evaluation pipeline:
[GitHub repository](https://github.com/<your-org>/gazerefine).