---
title: GRACE Reader Study
emoji: 🩺
colorFrom: indigo
colorTo: blue
sdk: gradio
sdk_version: 4.44.1
python_version: "3.11"
app_file: app.py
pinned: false
---

# GRACE Reader Study annotation app

Blinded, resume-safe radiologist reader study for the GRACE project. One case per screen,
no scrolling: reference CXR on the left, each anonymized system's output on the right with its
rating controls directly beneath the image it refers to. Scale legends and subjective-term
definitions are printed inline; per-image zoom / brightness / contrast are display-only.

## What a reader does per case
For each anonymized item: **answer correctness** (Correct / Incorrect / Indeterminate),
**decision appropriateness** (1-5), **grounding relevance** (Relevant / Partial / Not relevant),
optional note. Plus one case-level question: is the reference (ground-truth) annotation acceptable.
No best/worst ranking is asked; rankings are derived in the backend from per-item scores.

## Run locally
```bash
pip install -r requirements.txt
python build_cases_example.py      # generates data/cases.json + placeholder images (demo)
python app.py                      # http://127.0.0.1:7860
```
Demo credentials (when READER_CREDENTIALS is unset): `reader1 / changeme`, `reader2 / changeme`.

## Deploy as a Hugging Face Space
1. Create a **Gradio** Space.
2. Upload `app.py`, `requirements.txt`, `README.md`, and (for a demo) the `data/` folder.
   For the real study do NOT commit patient images: put `cases.json` + images in a **private
   dataset** and set `CASES_DATASET` instead (see below).
3. In **Settings -> Variables and secrets**, add these **secrets** (never commit them):
   - `HF_TOKEN` = your HF token with **write** permission (the value in `Keys.txt`; do not paste it into any file).
   - `READER_CREDENTIALS` = JSON, e.g. `{"Dr Kotak":"<pw1>","Dr B":"<pw2>","Resident C":"<pw3>"}`.
   - `RESPONSE_DATASET` = e.g. `DrSyedFaizan/grace-reader-responses` (created automatically, private).
   - `CASES_DATASET` (optional) = e.g. `DrSyedFaizan/grace-reader-cases` (private; pulled at boot).
   - `APP_SECRET` (optional) = any random string that signs resume tokens.

## Data flow and storage
- Responses stream to the private dataset `RESPONSE_DATASET` on every **Save & Next**, as
  `responses/<reader>.jsonl`, and to a local backup under `local_responses/` (so an HF write
  hiccup never loses data).
- **Robust schema:** one append-only record per `(annotator, case_id, item_id)` with a
  `dims` dict of value-per-dimension, plus a `__case__` record for case-level answers. Later
  changes to layout or wording can never overwrite or invalidate saved annotations; reads take
  the latest record per key.

## Auth / session / resume
- First visit always lands on the login page; the app is gated by per-reader credentials.
- On login the reader resumes at their first unfinished case and sees a live `X / N` counter
  (their own completed count, not a reset to 1). Item order is shuffled deterministically per
  `(reader, case)` so a resumed case looks identical.
- Closing the tab does NOT log out (a signed token is kept in `localStorage`); use the explicit
  **Logout** button to end the session.

## Preparing the real cases.json
Run `build_cases_example.py` and read its docstring for the exact schema. Each item's `item_id`
is the TRUE system id (used only in the backend for the blinded GRACE-vs-baseline comparison and
derived rankings); the reader never sees it. Keep `items` per case small (2-3) so the no-scroll
goal holds. Enrich the case set per `../reader_protocol.md` (near-boundary, deferrals, errors,
rare pathology, easy anchors).