Spaces:

devaanand
/

ccr-platform

Sleeping

devaanand commited on Jul 16

Commit

9fce59c

1 Parent(s): ed98758

Refine documentation and fix upload handling

- Update MANUAL_TESTING.md to clarify parsing behavior for multi-column files.
- Correct title and formatting in guide.html and index.html for consistency.
- Enhance README.md to specify behavior for skipped duplicates and blank lines.
- Implement unique ID assignment for corpus uploads in main.py to prevent file clobbering.
- Add regression test in test_api.py to ensure distinct uploads do not overwrite each other.

Files changed (6) hide show

MANUAL_TESTING.md +2 -1
backend/app/guide.html +12 -12
backend/app/main.py +5 -0
backend/tests/test_api.py +38 -0
frontend/index.html +1 -1
sample_data/README.md +1 -1

MANUAL_TESTING.md CHANGED Viewed

@@ -63,7 +63,8 @@ with a "Sign in (top right)" message. Sign in and retry: accepted.
 2. Try a CSV with `item,reverse` columns (1/true/yes/R = reverse) or a
    single-column file with `(R)` markers.
 3. Expect: items fill the textarea ((R) appended where flagged), the filename
-   becomes the suggested name, and parse notes list skipped blanks/duplicates.
    Nothing is saved until you review and press Save. Item files are never
    retained on the server.

 2. Try a CSV with `item,reverse` columns (1/true/yes/R = reverse) or a
    single-column file with `(R)` markers.
 3. Expect: items fill the textarea ((R) appended where flagged), the filename
+   becomes the suggested name, and parse notes list skipped duplicates (and
+   blank cells in multi-column files; fully blank lines are dropped at read).
    Nothing is saved until you review and press Save. Item files are never
    retained on the server.

backend/app/guide.html CHANGED Viewed

@@ -3,7 +3,7 @@
 <head>
 <meta charset="utf-8">
 <meta name="viewport" content="width=device-width, initial-scale=1">
-<title>CCR Platform — Testing Guide</title>
 <style>
   :root {
     --bg: #ffffff; --fg: #1a1a1a; --muted: #666; --border: #ddd;
@@ -49,16 +49,16 @@
 <body>
 <main>
-<h1>CCR Platform — Testing Guide</h1>
 <p class="sub">Everything built so far, as click-through scenarios: what to do and exactly what
-you should see. Every dataset referenced is downloadable right here — all texts are
 synthetic, written for this kit; no real participant data anywhere.</p>
 <div class="note">
-  <b>⚠ This is the free dev instance — two caveats before you start:</b>
   <ul>
     <li><b>Storage is ephemeral.</b> Accounts, projects, and saved runs are wiped whenever
-        the instance rebuilds or restarts. If your account disappears, that's expected —
         just re-register. (Google sign-in recreates itself automatically.)</li>
     <li><b>It sleeps when idle.</b> After ~48&nbsp;h without visitors, the first page load
         takes about a minute to wake it up.</li>
@@ -138,12 +138,12 @@ are deleted after analysis. Upload
 <ol>
   <li>"+ Custom construct" &gt; "Upload items from CSV/XLSX".</li>
   <li>Try <a href="/samples/construct_items_demo.csv" download>construct_items_demo.csv</a>
-      (<code>item,reverse</code> columns — 1/true/yes/R = reverse),
       <a href="/samples/construct_items_marker_demo.csv" download>construct_items_marker_demo.csv</a>
       (single column with <code>(R)</code> markers), or
       <a href="/samples/construct_items_demo.xlsx" download>construct_items_demo.xlsx</a> (Excel).</li>
   <li>Expect: items fill the textarea ((R) appended where flagged), the filename becomes
-      the suggested name, and parse notes list skipped blanks/duplicates. Nothing is saved
       until you review and press Save. Item files are never retained on the server.</li>
 </ol>
@@ -159,7 +159,7 @@ are deleted after analysis. Upload
   <tr><td>french_demo.csv</td><td>fr + Multilingual E5</td><td>no language warnings</td></tr>
   <tr><td><a href="/samples/mixed_language_demo.csv" download>mixed_language_demo.csv</a></td><td>en + MiniLM</td><td>LANGUAGE_UNCERTAIN (majority 50%)</td></tr>
   <tr><td><a href="/samples/long_documents_demo.csv" download>long_documents_demo.csv</a></td><td>en + MiniLM</td>
-      <td>TEXTS_MAYBE_TRUNCATED (4) + LANGUAGE_UNCERTAIN (only 10 rows, below the 20-row minimum — by design)</td></tr>
 </table>
 </div>
 <p>Warnings are per-run snapshots: changing language/model requires a NEW run; old result
@@ -181,12 +181,12 @@ pages don't update.</p>
 <h2 id="accounts">6. Accounts</h2>
 <ol>
-  <li>Sign in (top right) &gt; "Create a free account" &gt; email + password (min 8 chars) —
       or use "Continue with Google".</li>
   <li>You're signed in immediately; the header shows your name.</li>
   <li>Sign out, sign back in; wrong password gives "Incorrect email or password";
       registering the same email again gives "already exists".</li>
-  <li>Email is case-insensitive. No self-service password reset yet — reset = admin action.
       (And remember: password accounts vanish when this dev instance rebuilds.)</li>
 </ol>
@@ -223,8 +223,8 @@ pages don't update.</p>
 </ol>
 <h2 id="feedback">10. Found something off?</h2>
-<p>Anything that doesn't match what this guide says it should do — or anything confusing,
-slow, or missing — please note the section number and what you saw, and send it to
 Deva (<a href="mailto:devaanand@umass.edu">devaanand@umass.edu</a>). Screenshots welcome.</p>
 </main>

 <head>
 <meta charset="utf-8">
 <meta name="viewport" content="width=device-width, initial-scale=1">
+<title>CCR Platform - Testing Guide</title>
 <style>
   :root {
     --bg: #ffffff; --fg: #1a1a1a; --muted: #666; --border: #ddd;
 <body>
 <main>
+<h1>CCR Platform - Testing Guide</h1>
 <p class="sub">Everything built so far, as click-through scenarios: what to do and exactly what
+you should see. Every dataset referenced is downloadable right here - all texts are
 synthetic, written for this kit; no real participant data anywhere.</p>
 <div class="note">
+  <b>⚠ This is the free dev instance - two caveats before you start:</b>
   <ul>
     <li><b>Storage is ephemeral.</b> Accounts, projects, and saved runs are wiped whenever
+        the instance rebuilds or restarts. If your account disappears, that's expected -
         just re-register. (Google sign-in recreates itself automatically.)</li>
     <li><b>It sleeps when idle.</b> After ~48&nbsp;h without visitors, the first page load
         takes about a minute to wake it up.</li>
 <ol>
   <li>"+ Custom construct" &gt; "Upload items from CSV/XLSX".</li>
   <li>Try <a href="/samples/construct_items_demo.csv" download>construct_items_demo.csv</a>
+      (<code>item,reverse</code> columns - 1/true/yes/R = reverse),
       <a href="/samples/construct_items_marker_demo.csv" download>construct_items_marker_demo.csv</a>
       (single column with <code>(R)</code> markers), or
       <a href="/samples/construct_items_demo.xlsx" download>construct_items_demo.xlsx</a> (Excel).</li>
   <li>Expect: items fill the textarea ((R) appended where flagged), the filename becomes
+      the suggested name, and parse notes list skipped duplicates. Nothing is saved
       until you review and press Save. Item files are never retained on the server.</li>
 </ol>
   <tr><td>french_demo.csv</td><td>fr + Multilingual E5</td><td>no language warnings</td></tr>
   <tr><td><a href="/samples/mixed_language_demo.csv" download>mixed_language_demo.csv</a></td><td>en + MiniLM</td><td>LANGUAGE_UNCERTAIN (majority 50%)</td></tr>
   <tr><td><a href="/samples/long_documents_demo.csv" download>long_documents_demo.csv</a></td><td>en + MiniLM</td>
+      <td>TEXTS_MAYBE_TRUNCATED (4) + LANGUAGE_UNCERTAIN (only 10 rows, below the 20-row minimum - by design)</td></tr>
 </table>
 </div>
 <p>Warnings are per-run snapshots: changing language/model requires a NEW run; old result
 <h2 id="accounts">6. Accounts</h2>
 <ol>
+  <li>Sign in (top right) &gt; "Create a free account" &gt; email + password (min 8 chars) -
       or use "Continue with Google".</li>
   <li>You're signed in immediately; the header shows your name.</li>
   <li>Sign out, sign back in; wrong password gives "Incorrect email or password";
       registering the same email again gives "already exists".</li>
+  <li>Email is case-insensitive. No self-service password reset yet - reset = admin action.
       (And remember: password accounts vanish when this dev instance rebuilds.)</li>
 </ol>
 </ol>
 <h2 id="feedback">10. Found something off?</h2>
+<p>Anything that doesn't match what this guide says it should do - or anything confusing,
+slow, or missing - please note the section number and what you saw, and send it to
 Deva (<a href="mailto:devaanand@umass.edu">devaanand@umass.edu</a>). Screenshots welcome.</p>
 </main>

backend/app/main.py CHANGED Viewed

@@ -11,6 +11,7 @@ from __future__ import annotations
 import json
 import os
 from contextlib import asynccontextmanager
 from pathlib import Path
@@ -483,7 +484,11 @@ async def upload_corpus(
             f"Anonymous uploads are limited to {mb} MB. Sign in (top right) to upload larger files.",
         )
     corpus = Corpus(
         project_id=project_id, filename=file.filename, path="", n_rows=0, columns_json="[]"
     )
     # Parse from a local temp file, then hand the bytes to the storage backend

 import json
 import os
+import uuid
 from contextlib import asynccontextmanager
 from pathlib import Path
             f"Anonymous uploads are limited to {mb} MB. Sign in (top right) to upload larger files.",
         )
+    # Assign the id NOW: the model's default fires at INSERT flush, so reading
+    # corpus.id before commit yields None - every upload would then share one
+    # "None.csv" on disk, each new upload silently clobbering the previous one.
     corpus = Corpus(
+        id=uuid.uuid4().hex,
         project_id=project_id, filename=file.filename, path="", n_rows=0, columns_json="[]"
     )
     # Parse from a local temp file, then hand the bytes to the storage backend

backend/tests/test_api.py CHANGED Viewed

@@ -269,3 +269,41 @@ def test_script_export_is_valid_offline_python(client, flow):
     reqs = client.get(f"/api/jobs/{job_id}/script-requirements")
     assert reqs.status_code == 200
     assert "==" in reqs.text  # pinned versions, not ranges

     reqs = client.get(f"/api/jobs/{job_id}/script-requirements")
     assert reqs.status_code == 200
     assert "==" in reqs.text  # pinned versions, not ranges
+# ------------------------------------------- uploads must not clobber each other
+def test_second_upload_does_not_clobber_first(client, flow):
+    """Regression: corpus ids used to be read before INSERT (still None), so
+    every upload stored to the same "None.csv" - a later upload silently
+    replaced an earlier corpus's file, and runs analyzed the wrong dataset."""
+    project = client.post("/api/projects", json={"name": "clobber-check"}).json()
+    first = upload(client, project["id"], "first.csv",
+                   b"text\nalpha one\nalpha two\nalpha three\n").json()
+    second = upload(client, project["id"], "second.csv",
+                    b"text\nbeta one\nbeta two\nbeta three\nbeta four\n").json()
+    # Distinct stored files, both still present after the second upload.
+    from pathlib import Path
+    assert first["id"] and second["id"] and first["id"] != second["id"]
+    paths = {
+        c["filename"]: c for c in
+        client.get(f"/api/projects/{project['id']}/corpora").json()
+    }
+    assert paths["first.csv"]["n_rows"] == 3 and paths["second.csv"]["n_rows"] == 4
+    # Running the FIRST corpus must analyze the first corpus's rows.
+    job = client.post("/api/jobs", json={
+        "project_id": project["id"], "corpus_id": first["id"],
+        "construct_id": flow["construct"]["id"], "text_column": "text",
+        "model_name": "fake-deterministic",
+    }).json()
+    job = wait_for_job(client, job["id"])
+    assert job["status"] == "completed", job.get("error")
+    # Assert on the EXPORTED rows (what was actually analyzed), not on DB
+    # metadata - the broken path kept correct metadata while scoring the
+    # other corpus's texts.
+    export = client.get(f"/api/jobs/{job['id']}/export").text
+    data_lines = [l for l in export.splitlines()[1:] if l.strip()]
+    assert len(data_lines) == 3, export
+    assert "alpha" in export and "beta" not in export

frontend/index.html CHANGED Viewed

@@ -3,7 +3,7 @@
   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <title>CCR Platform — Contextualized Construct Representations</title>
   </head>
   <body>
     <div id="root"></div>

   <head>
     <meta charset="UTF-8" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+    <title>CCR Platform - Contextualized Construct Representations</title>
   </head>
   <body>
     <div id="root"></div>

sample_data/README.md CHANGED Viewed

@@ -19,7 +19,7 @@ select language English and model MiniLM.
 | `xlsx_upload_demo.xlsx` | Excel ingestion path | Uploads and parses like a CSV |
 | `large_demo.csv` | Anonymous upload caps (800 rows) | Signed out: blocked with the sign-in hint. Signed in: runs |
 | `construct_items_demo.csv` | Construct upload: item + reverse columns | + Custom construct > upload: 6 items parse, last one flagged reverse-scored |
-| `construct_items_marker_demo.csv` | Construct upload: (R) markers, blank + duplicate rows | 4 items parse (blank and duplicate skipped, notes say so); "(R)" line flagged |
 | `construct_items_demo.xlsx` | Construct upload from Excel | 4 items, two flagged reverse via (R) markers |
 Notes:

 | `xlsx_upload_demo.xlsx` | Excel ingestion path | Uploads and parses like a CSV |
 | `large_demo.csv` | Anonymous upload caps (800 rows) | Signed out: blocked with the sign-in hint. Signed in: runs |
 | `construct_items_demo.csv` | Construct upload: item + reverse columns | + Custom construct > upload: 6 items parse, last one flagged reverse-scored |
+| `construct_items_marker_demo.csv` | Construct upload: (R) markers, blank + duplicate rows | 4 items parse; notes report the skipped duplicate (the blank line is dropped silently at read); "(R)" line flagged |
 | `construct_items_demo.xlsx` | Construct upload from Excel | 4 items, two flagged reverse via (R) markers |
 Notes: