Store each review as a separate file under reviews/<submission>/

- Submissions are now immutable; reviews live in their own files, so
concurrent reviews by different people never conflict (no read-modify-write).
- add_review() writes reviews/<base>/<stamp>__<reviewer>.json.
- list_submissions() derives current status from the latest review and bundles
the full review timeline + submission record per item (single repo listing).
- Admin page reads bundled data; shows timeline + 'Add a review' form.
- README documents the new submissions/ + reviews/ layout and Python loader.

README.md

@@ -23,7 +23,7 @@ A Streamlit intake form for trial statisticians. Submissions are saved to a **Hu
     - `extraction_only` → 1 rubric: `output.json`
     - `derivation_required` → 4 rubrics: `output.json` × {Inputs used, Calculated value, Method} + `output.R` × {Reproducibility}
   - Each rubric collects `points`, `tolerance`, `criterion`.
-- **Admin page (`pages/1_Admin.py`)** — password-gated review console. A submission can be reviewed many times by different people: each review (status + reviewer name + comment) is appended to the submission's `review_history`, and the page shows the full timeline. The top-level `status` always reflects the most recent review. Every review is committed back to the dataset.
+- **Admin page (`pages/1_Admin.py`)** — password-gated review console. A submission can be reviewed many times by different people: each review (status + reviewer name + comment) is written as its own file under `reviews/<submission>/`, and the page shows the full timeline. The current status is the most recent review's status. Submissions themselves are never modified.
 
 ## Run locally
 
@@ -85,9 +85,20 @@ The Space will restart automatically and pick up the new secrets.
 ### 6. Test
 
 - Open the Space URL → fill the form → **Submit**. A new file lands in `submissions/<trial_id>__<username>__<timestamp>.json` in the dataset repo.
-- Open the **Admin** page (left sidebar) → enter password → see the submission with status `pending` → change status, add reviewer/note → save.
+- Open the **Admin** page (left sidebar) → enter password → see the submission with status `pending` → add a review (your name + status + comment). It appears in the review timeline and a new file lands under `reviews/<submission>/`. Add more reviews to build up the history.
 
-## Submission record shape
+## Dataset layout
+
+Submissions are **immutable**. Each review is a **separate file** — so a
+submission can be reviewed many times by different people, and concurrent
+reviews never conflict (each is a brand-new file, never an overwrite).
+
+```text
+submissions/<trial>__<user>__<stamp>.json            # the submission (never rewritten)
+reviews/<trial>__<user>__<stamp>/<stamp>__<rev>.json # one file per review
+```
+
+### Submission file (`submissions/*.json`)
 
 ```json
 {
@@ -95,14 +106,6 @@ The Space will restart automatically and pick up the new secrets.
   "submittedAt": "2026-06-01T...",
   "trial_id": "NCT0001",
   "username": "jdoe",
-  "status": "needs_fix",
-  "reviewer": "Dr. Lee",
-  "reviewerNote": "still missing the power assumption",
-  "reviewedAt": "2026-06-01T16:00:00+00:00",
-  "review_history": [
-    {"at": "2026-06-01T15:30:00+00:00", "reviewer": "Dr. Smith", "status": "reviewed",  "note": "looks good"},
-    {"at": "2026-06-01T16:00:00+00:00", "reviewer": "Dr. Lee",   "status": "needs_fix", "note": "still missing the power assumption"}
-  ],
   "comparison": {
     "trial_id": "NCT0001",
     "username": "jdoe",
@@ -125,14 +128,40 @@ The Space will restart automatically and pick up the new secrets.
 }
 ```
 
-Load all submissions in Python:
+### Review file (`reviews/<submission>/*.json`)
+
+```json
+{
+  "submissionId": "submissions/NCT0001__jdoe__2026-06-01T...Z.json",
+  "at": "2026-06-01T16:00:00+00:00",
+  "reviewer": "Dr. Lee",
+  "status": "needs_fix",
+  "note": "still missing the power assumption"
+}
+```
+
+The **current status** of a submission is derived as the most recent review's
+status (or `pending` if it has no reviews yet).
+
+### Load everything in Python
 
 ```python
 from huggingface_hub import snapshot_download
-import json, glob
+import json, glob, os
 
 local = snapshot_download("ttt-77/tdb-intake-submissions", repo_type="dataset")
-records = [json.load(open(f)) for f in glob.glob(f"{local}/submissions/*.json")]
+
+submissions = {
+    os.path.basename(f)[:-5]: json.load(open(f))
+    for f in glob.glob(f"{local}/submissions/*.json")
+}
+# reviews grouped by submission base name
+reviews = {}
+for f in glob.glob(f"{local}/reviews/*/*.json"):
+    base = os.path.basename(os.path.dirname(f))
+    reviews.setdefault(base, []).append(json.load(open(f)))
+for base in reviews:
+    reviews[base].sort(key=lambda r: r["at"])  # oldest first
 ```
 
 ## Project structure

lib/storage.py

@@ -1,13 +1,22 @@
 """Storage backend: Hugging Face Dataset repo, with local filesystem fallback for dev.
 
+Layout inside the dataset repo:
+
+    submissions/<trial>__<user>__<stamp>.json          (immutable: the submission)
+    reviews/<trial>__<user>__<stamp>/<stamp>__<rev>.json (one file per review)
+
+Submissions are never rewritten. Each review is a brand-new file, so multiple
+reviewers can review the same submission concurrently with no write conflict.
+The "current status" of a submission is derived from its most recent review.
+
 Env vars (set in HF Space → Settings → Variables and secrets):
     HF_TOKEN            - HF user access token with Write permission
     HF_DATASET_REPO     - e.g. "ttt-77/tdb-intake-submissions"
     HF_DATASET_BRANCH   - optional, defaults to "main"
-    ADMIN_PASSWORD      - shared password for the /Admin page
+    ADMIN_PASSWORD      - shared password for the Admin page
 
-If HF_TOKEN or HF_DATASET_REPO is missing, all I/O goes to ./data/submissions/*.json
-so local dev works without HF credentials.
+If HF_TOKEN or HF_DATASET_REPO is missing, all I/O goes to ./data/... so local
+dev works without HF credentials.
 """
 
 from __future__ import annotations
@@ -35,6 +44,7 @@ _api = HfApi(token=HF_TOKEN) if hf_configured else None
 
 LOCAL_DATA_DIR = Path("data")
 SUBMISSIONS_PREFIX = "submissions"
+REVIEWS_PREFIX = "reviews"
 
 
 def _safe(s: str) -> str:
@@ -45,11 +55,17 @@ def _now_iso() -> str:
     return datetime.now(timezone.utc).isoformat()
 
 
-def _stamp_for_filename() -> str:
-    return _now_iso().replace(":", "-").replace(".", "-")
+def _stamp(iso: Optional[str] = None) -> str:
+    return (iso or _now_iso()).replace(":", "-").replace(".", "-")
+
 
+def _base_id(submission_id: str) -> str:
+    """'submissions/foo.json' -> 'foo'"""
+    name = submission_id.split("/")[-1]
+    return name[:-5] if name.endswith(".json") else name
 
-# ---- HF helpers ----------------------------------------------------------
+
+# ---- low-level read/write/list (HF or local) -----------------------------
 
 def _hf_upload_json(path_in_repo: str, payload: Dict[str, Any], commit_message: str) -> None:
     assert _api is not None
@@ -65,182 +81,153 @@ def _hf_upload_json(path_in_repo: str, payload: Dict[str, Any], commit_message:
 
 
 def _hf_read_json(path_in_repo: str) -> Optional[Dict[str, Any]]:
-    """Fetch via the resolve URL — no cache, always fresh."""
     url = (
         f"https://huggingface.co/datasets/{HF_DATASET_REPO}"
         f"/resolve/{HF_DATASET_BRANCH}/{path_in_repo}"
     )
-    headers = {"Authorization": f"Bearer {HF_TOKEN}"}
-    r = requests.get(url, headers=headers, timeout=20)
+    r = requests.get(url, headers={"Authorization": f"Bearer {HF_TOKEN}"}, timeout=20)
     if r.status_code == 404:
         return None
     r.raise_for_status()
     return r.json()
 
 
-def _hf_list_submissions() -> List[str]:
-    assert _api is not None
+def _write_json(path_in_repo: str, payload: Dict[str, Any], commit_message: str) -> None:
+    if hf_configured:
+        _hf_upload_json(path_in_repo, payload, commit_message)
+        return
+    p = LOCAL_DATA_DIR / path_in_repo
+    p.parent.mkdir(parents=True, exist_ok=True)
+    p.write_text(json.dumps(payload, indent=2, ensure_ascii=False), encoding="utf-8")
+
+
+def _read_json(path_in_repo: str) -> Optional[Dict[str, Any]]:
+    if hf_configured:
+        return _hf_read_json(path_in_repo)
+    p = LOCAL_DATA_DIR / path_in_repo
+    if not p.exists():
+        return None
     try:
-        files = _api.list_repo_files(
-            repo_id=HF_DATASET_REPO,
-            repo_type="dataset",
-            revision=HF_DATASET_BRANCH,
-        )
-    except HfHubHTTPError as e:
-        if e.response is not None and e.response.status_code == 404:
-            return []
-        raise
-    return [
-        f for f in files
-        if f.startswith(f"{SUBMISSIONS_PREFIX}/") and f.endswith(".json")
-    ]
+        return json.loads(p.read_text(encoding="utf-8"))
+    except Exception:
+        return None
 
 
-# ---- Public API ----------------------------------------------------------
+def _all_files() -> List[str]:
+    """List every file path in the repo (HF) or under ./data (local)."""
+    if hf_configured:
+        assert _api is not None
+        try:
+            return _api.list_repo_files(
+                repo_id=HF_DATASET_REPO,
+                repo_type="dataset",
+                revision=HF_DATASET_BRANCH,
+            )
+        except HfHubHTTPError as e:
+            if e.response is not None and e.response.status_code == 404:
+                return []
+            raise
+    if not LOCAL_DATA_DIR.exists():
+        return []
+    return [p.relative_to(LOCAL_DATA_DIR).as_posix() for p in LOCAL_DATA_DIR.rglob("*.json")]
+
+
+# ---- public API ----------------------------------------------------------
 
 def create_submission(trial_id: str, username: str, comparison: Dict[str, Any]) -> Dict[str, Any]:
-    """Write a new submission. Returns a result dict with submissionId and (optionally) url."""
-    file_name = f"{_safe(trial_id)}__{_safe(username)}__{_stamp_for_filename()}.json"
+    """Write a new (immutable) submission file. Returns submissionId + url."""
+    file_name = f"{_safe(trial_id)}__{_safe(username)}__{_stamp()}.json"
     submission_id = f"{SUBMISSIONS_PREFIX}/{file_name}"
     record = {
         "submissionId": submission_id,
         "submittedAt": _now_iso(),
         "trial_id": trial_id,
         "username": username,
-        # These top-level fields mirror the most recent review for easy
-        # filtering/sorting; they are updated by add_review().
-        "status": "pending",
-        "reviewer": "",
-        "reviewerNote": "",
-        "reviewedAt": "",
-        # Full append-only log: one entry per review by any reviewer.
-        "review_history": [],
         "comparison": comparison,
     }
+    _write_json(submission_id, record, f"Add submission: {trial_id} — {username}")
+    url = (
+        f"https://huggingface.co/datasets/{HF_DATASET_REPO}"
+        f"/blob/{HF_DATASET_BRANCH}/{submission_id}"
+        if hf_configured
+        else None
+    )
+    return {"submissionId": submission_id, "url": url, "record": record}
 
-    if hf_configured:
-        _hf_upload_json(
-            submission_id,
-            record,
-            commit_message=f"Add submission: {trial_id} — {username}",
-        )
-        return {
-            "submissionId": submission_id,
-            "url": (
-                f"https://huggingface.co/datasets/{HF_DATASET_REPO}"
-                f"/blob/{HF_DATASET_BRANCH}/{submission_id}"
-            ),
-            "record": record,
-        }
-
-    # local fs fallback
-    path = LOCAL_DATA_DIR / submission_id
-    path.parent.mkdir(parents=True, exist_ok=True)
-    path.write_text(json.dumps(record, indent=2, ensure_ascii=False), encoding="utf-8")
-    return {"submissionId": submission_id, "url": None, "record": record}
 
+def add_review(submission_id: str, status: str, reviewer: str, note: str = "") -> Dict[str, Any]:
+    """Append a review as its own file under reviews/<base>/.
 
-def list_submissions() -> List[Dict[str, Any]]:
-    """Return summaries (small fields only) of every submission."""
-    if hf_configured:
-        records = []
-        for path in _hf_list_submissions():
-            data = _hf_read_json(path)
-            if not data:
-                continue
-            records.append(_summarize(data))
-        records.sort(key=lambda r: r.get("submittedAt", ""), reverse=True)
-        return records
-
-    dir_ = LOCAL_DATA_DIR / SUBMISSIONS_PREFIX
-    if not dir_.exists():
-        return []
-    summaries = []
-    for f in sorted(dir_.glob("*.json"), reverse=True):
-        try:
-            data = json.loads(f.read_text(encoding="utf-8"))
-            summaries.append(_summarize(data))
-        except Exception:
-            continue
-    return summaries
+    Each review is a new file (never overwrites), so concurrent reviews by
+    different people cannot conflict.
+    """
+    base = _base_id(submission_id)
+    now = _now_iso()
+    review = {
+        "submissionId": submission_id,
+        "at": now,
+        "reviewer": reviewer,
+        "status": status,
+        "note": note,
+    }
+    review_path = f"{REVIEWS_PREFIX}/{base}/{_stamp(now)}__{_safe(reviewer) or 'anon'}.json"
+    _write_json(review_path, review, f"Review ({status}) by {reviewer or 'anon'} on {base}")
+    return review
+
+
+def list_reviews(submission_id: str, all_files: Optional[List[str]] = None) -> List[Dict[str, Any]]:
+    """All reviews for a submission, oldest first."""
+    base = _base_id(submission_id)
+    prefix = f"{REVIEWS_PREFIX}/{base}/"
+    files = all_files if all_files is not None else _all_files()
+    paths = sorted(f for f in files if f.startswith(prefix) and f.endswith(".json"))
+    reviews = [r for r in (_read_json(p) for p in paths) if r]
+    reviews.sort(key=lambda r: r.get("at", ""))
+    return reviews
 
 
 def get_submission(submission_id: str) -> Optional[Dict[str, Any]]:
     if not submission_id.startswith(f"{SUBMISSIONS_PREFIX}/"):
         return None
-    if hf_configured:
-        return _hf_read_json(submission_id)
-    path = LOCAL_DATA_DIR / submission_id
-    if not path.exists():
-        return None
-    return json.loads(path.read_text(encoding="utf-8"))
+    return _read_json(submission_id)
 
 
-def add_review(
-    submission_id: str,
-    status: str,
-    reviewer: str,
-    note: str = "",
-) -> Optional[Dict[str, Any]]:
-    """Append a review to the submission's history.
+def list_submissions() -> List[Dict[str, Any]]:
+    """Every submission with its derived status and full review timeline.
 
-    A submission can be reviewed many times by different people. Each call adds
-    one entry to ``review_history`` and mirrors it into the top-level
-    status/reviewer/reviewerNote/reviewedAt fields (which always reflect the
-    latest review).
+    Each item: submissionId, trial_id, username, submittedAt, status,
+    reviewedAt, reviewer, review_count, reviews (list), submission (full record).
     """
-    record = get_submission(submission_id)
-    if record is None:
-        return None
-
-    now = _now_iso()
-    entry = {
-        "at": now,
-        "reviewer": reviewer,
-        "status": status,
-        "note": note,
-    }
-    history = record.get("review_history")
-    if not isinstance(history, list):
-        history = []
-    history.append(entry)
-    record["review_history"] = history
-
-    # Mirror the latest review into the top-level fields.
-    record["status"] = status
-    record["reviewer"] = reviewer
-    record["reviewerNote"] = note
-    record["reviewedAt"] = now
-
-    if hf_configured:
-        _hf_upload_json(
-            submission_id,
-            record,
-            commit_message=f"Review ({status}) by {reviewer or 'anon'}: "
-            f"{submission_id.rsplit('/', 1)[-1]}",
+    files = _all_files()
+    sub_paths = sorted(
+        f for f in files if f.startswith(f"{SUBMISSIONS_PREFIX}/") and f.endswith(".json")
+    )
+    result: List[Dict[str, Any]] = []
+    for sp in sub_paths:
+        sub = _read_json(sp)
+        if not sub:
+            continue
+        reviews = list_reviews(sp, all_files=files)
+        latest = reviews[-1] if reviews else None
+        result.append(
+            {
+                "submissionId": sp,
+                "trial_id": sub.get("trial_id", ""),
+                "username": sub.get("username", ""),
+                "submittedAt": sub.get("submittedAt", ""),
+                "status": latest["status"] if latest else "pending",
+                "reviewedAt": latest["at"] if latest else "",
+                "reviewer": latest["reviewer"] if latest else "",
+                "review_count": len(reviews),
+                "reviews": reviews,
+                "submission": sub,
+            }
         )
-    else:
-        path = LOCAL_DATA_DIR / submission_id
-        path.write_text(json.dumps(record, indent=2, ensure_ascii=False), encoding="utf-8")
-    return record
-
-
-def _summarize(record: Dict[str, Any]) -> Dict[str, Any]:
-    history = record.get("review_history")
-    review_count = len(history) if isinstance(history, list) else 0
-    return {
-        "submissionId": record.get("submissionId", ""),
-        "trial_id": record.get("trial_id", ""),
-        "username": record.get("username", ""),
-        "submittedAt": record.get("submittedAt", ""),
-        "status": record.get("status", "pending"),
-        "reviewedAt": record.get("reviewedAt", ""),
-        "reviewer": record.get("reviewer", ""),
-        "review_count": review_count,
-    }
+    result.sort(key=lambda r: r.get("submittedAt", ""), reverse=True)
+    return result
 
 
-# ---- Admin gate ----------------------------------------------------------
+# ---- admin gate ----------------------------------------------------------
 
 def check_admin_password(supplied: str) -> bool:
     if not ADMIN_PASSWORD:

pages/1_Admin.py

@@ -1,8 +1,8 @@
-"""Admin review console — list submissions, view review history, add reviews.
+"""Admin review console — list submissions, view review timeline, add reviews.
 
-A submission can be reviewed many times by different people. Each review is
-appended to the submission's ``review_history``; the most recent one is mirrored
-into the top-level status fields for filtering/sorting.
+Submissions are immutable. Each review is stored as its own file under
+``reviews/<submission>/``; a submission can be reviewed many times by different
+people. The "current status" shown here is the most recent review's status.
 """
 
 from __future__ import annotations
@@ -16,7 +16,6 @@ from lib.storage import (
     ADMIN_PASSWORD,
     add_review,
     check_admin_password,
-    get_submission,
     hf_configured,
     list_submissions,
 )
@@ -59,7 +58,7 @@ if not st.session_state.admin_authed:
 # ------------- load list -------------------------------------------------
 
 if not hf_configured:
-    st.info("ℹ️ Reading from `./data/submissions/` (local dev mode).")
+    st.info("ℹ️ Reading from `./data/` (local dev mode).")
 
 cols = st.columns([3, 1])
 with cols[0]:
@@ -74,23 +73,23 @@ with cols[1]:
         st.rerun()
 
 try:
-    summaries = list_submissions()
+    items = list_submissions()
 except Exception as e:
     st.error(f"Failed to list submissions: {e}")
     st.stop()
 
 if status_filter:
-    summaries = [s for s in summaries if s["status"] in status_filter]
+    items = [s for s in items if s["status"] in status_filter]
 
-st.caption(f"{len(summaries)} submission(s)")
+st.caption(f"{len(items)} submission(s)")
 
-if not summaries:
+if not items:
     st.info("No submissions match this filter.")
     st.stop()
 
 # ------------- list display ----------------------------------------------
 
-for s in summaries:
+for s in items:
     n_reviews = s.get("review_count", 0)
     review_tag = f"  ·  💬 {n_reviews}" if n_reviews else "  ·  no reviews yet"
     label = (
@@ -99,32 +98,26 @@ for s in summaries:
         f"_{s['submittedAt']}_{review_tag}"
     )
     with st.expander(label):
-        record = get_submission(s["submissionId"])
-        if record is None:
-            st.error("Could not fetch this record.")
-            continue
-
         meta_c1, meta_c2 = st.columns(2)
         with meta_c1:
             st.markdown(
-                f"**Current status:** {status_badge(record.get('status', 'pending'))}  \n"
-                f"**Submitted:** {record.get('submittedAt', '')}  \n"
-                f"**Last reviewed:** {record.get('reviewedAt', '') or '—'}"
+                f"**Current status:** {status_badge(s.get('status', 'pending'))}  \n"
+                f"**Submitted:** {s.get('submittedAt', '')}  \n"
+                f"**Last reviewed:** {s.get('reviewedAt', '') or '—'}"
             )
         with meta_c2:
             st.markdown(
-                f"**File:** `{record.get('submissionId', '')}`  \n"
-                f"**Last reviewer:** {record.get('reviewer', '') or '—'}"
+                f"**File:** `{s.get('submissionId', '')}`  \n"
+                f"**Last reviewer:** {s.get('reviewer', '') or '—'}"
             )
 
-        # ---- Review history ------------------------------------------
-        history = record.get("review_history") or []
-        st.markdown(f"#### Review history ({len(history)})")
-        if not history:
+        # ---- Review timeline -----------------------------------------
+        reviews = s.get("reviews") or []
+        st.markdown(f"#### Review history ({len(reviews)})")
+        if not reviews:
             st.caption("No reviews yet.")
         else:
-            # Newest first.
-            for rev in reversed(history):
+            for rev in reversed(reviews):  # newest first
                 st.markdown(
                     f"- {status_badge(rev.get('status', ''))} — "
                     f"**{rev.get('reviewer') or 'anon'}** "
@@ -132,14 +125,14 @@ for s in summaries:
                     + (f"  \n  {rev.get('note')}" if rev.get("note") else "")
                 )
 
-        # ---- Add a new review ----------------------------------------
+        # ---- Add a review --------------------------------------------
         st.markdown("#### Add a review")
         with st.form(f"review_{s['submissionId']}"):
             new_status = st.radio(
                 "Status",
                 options=VALID_STATUSES,
-                index=VALID_STATUSES.index(record.get("status", "pending"))
-                if record.get("status") in VALID_STATUSES
+                index=VALID_STATUSES.index(s.get("status", "pending"))
+                if s.get("status") in VALID_STATUSES
                 else 0,
                 horizontal=True,
             )
@@ -148,8 +141,7 @@ for s in summaries:
                 reviewer = st.text_input("Your name", placeholder="e.g., Dr. Smith")
             with rc2:
                 note = st.text_input("Comment", placeholder="optional")
-            submitted = st.form_submit_button("Add review", type="primary")
-            if submitted:
+            if st.form_submit_button("Add review", type="primary"):
                 if not reviewer.strip():
                     st.error("Please enter your name.")
                 else:
@@ -166,4 +158,4 @@ for s in summaries:
                         st.error(f"Failed to add review: {e}")
 
         with st.expander("Raw submission JSON"):
-            st.code(json.dumps(record, indent=2, ensure_ascii=False), language="json")
+            st.code(json.dumps(s.get("submission", {}), indent=2, ensure_ascii=False), language="json")