add README.md

README.md

@@ -1,13 +1,106 @@
 ---
-title: Deferrl Reader Study
-emoji: 🌖
-colorFrom: green
-colorTo: red
+title: DEFER-RL Reader Study
+emoji: 🩻
+colorFrom: indigo
+colorTo: gray
 sdk: gradio
-sdk_version: 6.19.0
-python_version: '3.13'
+sdk_version: 6.17.3
 app_file: app.py
 pinned: false
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# DEFER-RL Radiologist Reader Study
+
+A blinded, multi-reader appropriateness study for the DEFER-RL project. Each case shows one
+reference imaging panel beside several anonymized, order-randomized **deferral-system decision
+panels** (DEFER-RL plus baselines). Readers rate each panel's decision; the backend derives any
+best/worst rankings. No model names are shown.
+
+## What the reader rates (per case)
+
+* **Per panel (one row each):** decision appropriateness (1-5), evidence-gathering adequacy (1-5),
+  reading soundness (1-5, only when the panel chose *Trust*), and a yes/no *misleading* judgement
+  with an inline definition and worked example.
+* **Per case:** "should this case **not** be auto-read?" (yes/partial/no) and "is the reference
+  imaging adequate to judge?" (yes/partial/no).
+
+Every scale legend is printed inline, every rating value has a hover tooltip, and each image has its
+own display-only zoom / brightness / contrast strip directly beneath it.
+
+## Deploy as a Hugging Face Space
+
+1. Create a **Gradio** Space (SDK version `6.17.3`, set by this README) and upload `app.py`,
+   `requirements.txt`, `README.md`. (Gradio is installed from `sdk_version`; do not pin it in
+   `requirements.txt`.)
+2. Add a **private dataset** for responses (e.g. `your-org/deferrl-reader-responses`).
+3. In the Space **Settings -> Variables and secrets**, set:
+   * `ANNOTATORS` (secret) - JSON of per-user credentials, e.g.
+     `{"dr_smith":"s3cret-a","dr_lee":"s3cret-b"}`. The username each reader types is their
+     annotator name and keys their own response file.
+   * `DATASET_REPO` (variable) - the private dataset id above.
+   * `HF_TOKEN` (secret) - a token with **write** access to that dataset.
+   * optional: `COMMIT_EVERY_MIN` (default `1`), `MAX_ITEMS` (default `5`), `DATA_DIR`, `RESP_DIR`.
+4. (Recommended) Enable **persistent storage** on the Space so `responses_local/` survives restarts
+   between dataset syncs.
+
+If `DATASET_REPO`/`HF_TOKEN` are unset the app still runs and writes responses locally (good for a
+dry run). With them set, every **Save & Next** is streamed to the private dataset by a
+`CommitScheduler`.
+
+## Loading real cases
+
+Replace the auto-generated sample data by committing your own `data/cases.json` and `data/images/`.
+Schema for each case:
+
+```json
+{
+  "case_id": "C001",
+  "cohort": "LIDC-IDRI chest CT",
+  "reference_image": "images/C001_ref.png",
+  "show_trail": true,
+  "ground_truth": {"image": "images/C001_gt.png", "text": "Reference standard: ..."},
+  "items": [
+    {"item_id": "defer_rl", "action": "Defer", "reading": "(routed to radiologist)",
+     "image": "images/C001_defer_rl.png", "trail": ["images/C001_defer_rl_t0.png", "..."]},
+    {"item_id": "atcxr", "action": "Trust", "reading": "No suspicious finding. BI-RADS 1.",
+     "image": "images/C001_atcxr.png", "trail": ["..."]}
+  ]
+}
+```
+
+* `item_id` is the true system name (never shown to readers); the UI shows blinded "Panel A/B/...".
+* `action` is `"Trust"` or `"Defer"`; soundness is only asked for `Trust` panels.
+* `show_trail` is the per-case **evidence-trail ablation** condition (saved with every response).
+* Put a balanced mix of difficulty / cohort / routing in the manifest for stratified analysis.
+
+If `data/cases.json` is absent, the app generates six synthetic sample cases so the Space runs
+immediately; delete them once real data is in place.
+
+## Response schema (robust to UI changes)
+
+Responses are append-only JSONL, one line per **(annotator, case_id, item_id, dimension) -> value**:
+
+```json
+{"schema_version":"deferrl-reader-1","ts":"...Z","annotator":"dr_smith","case_id":"C001",
+ "item_id":"defer_rl","dimension":"appropriateness","value":"4","presented_pos":2,
+ "item_action":"Defer","case_condition_show_trail":true}
+```
+
+Case-level answers use `item_id":"__case__"`. Because each value is an atomic, self-describing row,
+later changes to layout, controls, or wording can never overwrite or invalidate prior annotations,
+and best/worst rankings are derived offline from the per-panel scores.
+
+## Analysis pointers (offline)
+
+* `P_app` = fraction of *Defer* decisions (per system) with median reader rating >= 4.
+* Inter-rater agreement: weighted Cohen's kappa and Gwet's AC1 over the ordinal ratings.
+* Evidence-trail effect: compare ratings on `show_trail=true` vs `false` cases.
+* Rankings: order systems within each case by appropriateness; readers are never asked to rank.
+
+## Notes / limits
+
+* "Zero scrolling" is best on a wide display; each panel is self-contained (its controls, legend, and
+  tooltips sit with its image), so you never scroll to learn what a control means. With `MAX_ITEMS`
+  large on a small screen, rows may extend below the fold - lower `MAX_ITEMS` or use a wide monitor.
+* Zoom/brightness/contrast are pure CSS on the displayed image and never alter stored data.
+* Closing the tab keeps you logged in (session cookie); use **Log out** to end the session.