MANUAL_TESTING.md

Manual testing guide

Everything built so far, as click-through scenarios. Each scenario says what to do and exactly what you should see. Files referenced live in sample_data/ (see sample_data/README.md for what each one triggers).

0. Setup

cd backend
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8000

Open http://127.0.0.1:8000. First start downloads MiniLM (~90 MB) on the first real run; set CCR_WARM_MODEL=1 to preload it at startup instead.

To test quickly without models: CCR_FAKE_EMBEDDINGS=1 uvicorn ... (scores are fake but every flow works; never use for real analysis).

1. Projects and sidebar

1. Create three projects. They appear under "Today", newest activity first.
2. Type in the sidebar search box: list filters as you type.
3. Archive a project (project header > Archive): it moves into the collapsed "Archived" group; Unarchive brings it back. No data is lost either way.
4. Delete a project: requires typing the project name; removes its datasets, runs, and files permanently.

2. Upload paths (Step 1 card)

Upload	Expect
sample_corpus.csv	Parses, 60 rows, text column suggested
multi_column_demo.csv	5 columns; comment_text marked "(suggested)"
semicolon_delimited_demo.csv	Parses into exactly id + text (commas inside texts intact)
latin1_encoding_demo.csv	Parses with a ⚠ note: decoded as latin-1; fiancée/café render correctly
xlsx_upload_demo.xlsx	Parses like a CSV
a .txt or .pdf file	Rejected: unsupported file type

Anonymous limits (signed out): the Step 1 hint shows 2 MB / 200 rows and says uploads are deleted after analysis. Upload large_demo.csv (800 rows): rejected with a "Sign in (top right)" message. Sign in and retry: accepted.

3. Construct selection (Step 2 card)

1. Open the picker: search field + panel below it, library grouped by category, with "Recently used" pinned on top after your first runs.
2. Type "GAD" or "empathy": matches by name and category; Arrow keys + Enter work.
3. Select any imported construct: items listed, plus the "not yet verified verbatim" notice (expected for the whole imported library for now).

Multi-construct runs (new)

1. Pick a construct, then open the picker again and pick a second (selected ones show a ✓; clicking again removes). Each selected construct is listed as a collapsible block with its items and a "remove" link.
2. The run button reads "Run CCR analysis (2 constructs)"; a run scores every construct on ONE pass over the corpus, so 2 constructs take roughly as long as 1. The cap is 10 per run.
3. The runs table shows "Construct A + Construct B"; results open with a "Construct interrelations" card - a Pearson correlation matrix of the per-text scores (this is the point of the feature: how constructs interrelate in YOUR texts) - plus a collapsible per-construct section each with histogram, item loadings, and top/bottom texts.
4. Downloads: the CSV has per-construct prefixed columns (satisfaction_with_life_sim_item_1, satisfaction_with_life_ccr_score, …) and no bare ccr_score; metadata JSON carries every construct snapshot (output_schema_version 1.1); the reproduction script embeds the corpus once, scores all constructs, and prints the same correlation matrix. Single-construct runs are unchanged (flat columns, schema 1.0).

Custom construct, typed

1. "+ New construct" > "Type or paste" card > name it, paste items one per line.
2. Append (R) to one line: the form shows "1 item(s) marked reverse-scored".
3. Save: it appears in the picker under "My custom constructs"; run metadata will carry the reverse flag (check via Results > metadata download).

Custom construct, from file (new)

1. "+ New construct" > the "Upload a file" card.
2. Try a CSV with item,reverse columns (1/true/yes/R = reverse) or a single-column file with (R) markers.
3. Expect: items fill the textarea ((R) appended where flagged), the filename becomes the suggested name, and parse notes list skipped duplicates (and blank cells in multi-column files; fully blank lines are dropped at read). Nothing is saved until you review and press Save. Item files are never retained on the server.

Custom construct, drafted with AI (new)

Needs a generation key on the instance (GROQ_API_KEY or ANTHROPIC_API_KEY); without one the "Draft with AI" tab is hidden entirely. Signed-in only, 20/day (tune locally with CCR_USER_MAX_GENERATIONS_PER_DAY=2 to test the cap fast).

1. Happy path: sign in > "+ New construct" > "Draft with AI" card. Enter a name + a 1-2 sentence description, press "Draft items". Expect ~10 first-person, positively-worded items in the textarea (no (R) items - by design), a "N of 20 used today" counter, the amber "AI-generated · not validated - drafted by " notice, and (sometimes) model notes.
2. Review-edit-save: edit one item, delete one, Save. Picker shows the construct under "My custom constructs" with the "AI-generated · not validated" tag; the selected-construct block shows the caution paragraph. The tag persists despite the edits (seed was AI; item hash records edits).
3. Library guardrail: name it "Satisfaction with Life" - expect the warning pointing at the existing library scale before you generate.
4. Vague input: nonsense name + vague description still returns items, with model notes explaining the problem.
5. Run + provenance: run a corpus against the saved AI construct. Results page shows the caution line; metadata JSON has source_type: "llm_generated", generation (model, prompt_version, generated_at), and items_source_note.
6. Cap: with CCR_USER_MAX_GENERATIONS_PER_DAY=2, the third draft returns a friendly 429 ("resets at midnight UTC"); failed drafts do NOT burn quota.
7. Signed out: the tab shows a sign-in nudge; POST /api/constructs/generate-items returns 401.
8. No key set: unset both keys - the tab disappears; the endpoint returns 503.
9. One-shot live check without the UI: cd backend && GROQ_API_KEY=... .venv/bin/python ../scripts/smoke_test_generation.py

4. Language, models, and warnings (Step 3 card + results)

Run each of these and open the results page; the amber warnings panel should show exactly:

Corpus	Selection	Expected warnings
warnings_showcase.csv	en + MiniLM	EMPTY_ROWS_DROPPED (2), DUPLICATE_TEXTS (2), TEXT_TOO_SHORT (3), TEXTS_MAYBE_TRUNCATED (2); no language warnings
french_demo.csv	en + MiniLM	LANGUAGE_MISMATCH (detected fr, 100%)
french_demo.csv	fr + MiniLM	MODEL_LANGUAGE_UNSUPPORTED
french_demo.csv	fr + Multilingual E5	no language warnings
mixed_language_demo.csv	en + MiniLM	LANGUAGE_UNCERTAIN (majority 50%)
long_documents_demo.csv	en + MiniLM	TEXTS_MAYBE_TRUNCATED (4) + LANGUAGE_UNCERTAIN (only 10 rows, below the 20-row minimum - by design)

Warnings are per-run snapshots: changing language/model requires a NEW run; old result pages don't update.

5. Results and reproducibility

1. Run moral_foundations_demo.csv against two different MFQ-2 foundations: top texts change per foundation; the 6 neutral rows sink to the bottom.
2. Results page: histogram, mean/SD/min/max, per-item loadings, top/bottom texts.
3. Downloads: results CSV (input columns + sim_item_N + ccr_score), metadata JSON (model revision, construct snapshot + item hash, language block, environment pins), reproduction script + requirements file.
4. Reproduction check: both downloads carry your run's id, e.g. pip install -r requirements-repro_<run-id>.txt, then python reproduce_analysis_<run-id>.py your_corpus.csv on a machine with no platform access; the exact commands (with your run id and CSV name) are in the script's header. Values should match the export (target ~1e-5 with real models).

6. Accounts

1. Sign in (top right) > "Create a free account" > email + password (min 8 chars).
2. You're signed in immediately; header shows your name.
3. Sign out, sign back in; wrong password gives "Incorrect email or password"; registering the same email again gives "already exists".
4. Email is case-insensitive. There is no self-service password reset yet (interim local accounts; Google/Supabase swap planned) - reset = admin action.

7. Anonymous tiers (test signed OUT)

1. Upload caps: see section 2.
2. Run limit: run 3 analyses (default). The Step 3 card counts "X of 3 free runs used today". The 4th run is refused with a sign-in prompt (HTTP 429). Counter resets next day (UTC). Signing in removes the limit.
3. Delete-after-analysis: run any corpus, open results (fine, downloadable), note the info warning "uploaded file was deleted after this analysis". Re-running that same corpus: refused ("upload again, or sign in").
4. TTL purge: with CCR_ANON_TTL_HOURS=24 (deployment default; 0 = off in local dev), anonymous projects older than 24h are deleted entirely, startup + hourly.

8. Signed-in tier

1. Sign in, upload, run: no ANONYMOUS_DATA_REMOVED warning; re-running the same corpus works (file kept).
2. Saved-run cap: Step 3 card shows "N of 15 saved runs used". At the cap, new runs are refused until you delete old runs/projects (nothing is auto-deleted).
3. Ownership: your projects are invisible to signed-out visitors and other accounts (they get 403 on any modification). Anonymous projects stay shared.

9. Performance behaviors

1. Corpus-embedding cache: run the SAME corpus with a second construct (signed in, same model): the run skips document embedding and completes in seconds; metadata shows "doc_embeddings_from_cache": true.
2. Duplicate texts are embedded once (warnings_showcase.csv has 2 dupes): identical scores for identical texts, less compute.
3. API responses are gzip-compressed (check the response headers).

10. Robustness

1. Restart the server mid-run: the orphaned job is marked failed with an explanation, never stuck at "running".
2. A DB from an older version gains new columns automatically at startup (additive auto-migration) - no more "no such column" 500s.
3. Tampered session cookie = treated as signed out, no error.

11. Deployment (container)

cd frontend && npm run build && cd ..
docker build -t ccr-platform .
docker run -p 7860:7860 \
  -e CCR_SESSION_SECRET=$(python3 -c "import secrets;print(secrets.token_hex(32))") \
  -e CCR_DATA_DIR=/data -e CCR_COOKIE_SECURE=1 \
  -v ccr_data:/data ccr-platform

Checklist before giving the URL to real users:

• CCR_SESSION_SECRET set (sessions survive restarts)
• CCR_COOKIE_SECURE=1 (HTTPS only)
• CCR_DATA_DIR on a persistent volume (default /tmp is ephemeral)
• CCR_ANON_TTL_HOURS=24 (default in the image)
• Supabase: RLS shows enabled on all public tables (the app enables it automatically at startup; the Supabase dashboard linter should report zero rls_disabled_in_public issues)
• Smoke test: sections 2, 4, 6, 7 above

12. Admin page (/admin)

Requires ADMIN_EMAILS to include your signed-in email (see .env.example), OR a pi/maintainer role. The env allowlist is bootstrap + break-glass; the PI role carries the same escalation rights in-app. The full access model is documented at /product.

1. Sign in with an allowlisted or staff account: an "Admin" link appears in the header; non-admins (and signed-out visitors) see an access notice.
2. Overview: account/run/project counters plus scales awaiting verification.
3. Users - four tiers (external user, lab member, maintainer, PI):
   • Set a user to "lab member" or above: their saved-run cap disappears (check /api/auth/me shows max_saved_runs null).
   • Set a user to "maintainer" or "PI": they also get this admin page.
   • Escalation needs PI or env-admin rights: a maintainer gets 403 on granting pi/maintainer, and on resetting the password of, changing the role of, or deleting a pi/maintainer account. A PI-by-role CAN do all of that (no env entry needed). Nobody can change their own role.
   • Reset a password (temporary password shown once; old one stops working), delete a user (removes all their data; self-deletion refused).
4. Access before sign-in:
   • Pre-assign a role to an email (staff roles need PI/env rights): register with that email afterwards - the account lands at that tier, the assignment shows "claimed". Works for Google sign-ins too.
   • Create an invite link (lab member or external only): open it in a private window - the signup form announces the invite; registering through it grants the role. The invites table shows each link's status (active/expired/revoked) and who signed up through it ("used by"); Copy link re-copies an active link, Revoke kills it immediately. Expired/garbage/revoked tokens refuse registration.
5. Construct verification is maintainer-only: PI/env admins see the queue read-only (no action buttons; the API returns 403), a maintainer can mark scales verified. Statuses are applied back to the library YAML before production.
6. Audit trail (PI/env admins only; maintainers get 403 and don't see the card): every role change, reset, deletion, invite, pre-assignment, requeue, and verification appears with actor, target, and time.
7. Failed runs: a failed job lists with its error tail; Requeue re-runs it (refused when the corpus file is already gone - anonymous retention).
8. Verification: mark a scale Verified; its "unverified" flag disappears platform-wide. Statuses are applied back to the library YAML before production.

Step 2 layout + Save gating (PI feedback 2026-08-07)

1. One "+ New construct" button next to the picker opens a contained panel with three radio-cards (Type or paste / Upload a file / Draft with AI - the AI card only when a generation key is configured); only the chosen card's fields render. On the AI card, the items box + Save appear after drafting; Reference is hidden (no publication exists for an AI draft).
2. Save is DISABLED until the form has a name and at least one item; a line under the button says what's missing. No more silent dead click.
3. After "Draft items", the page scrolls to the items box, and the notice reads "N items drafted - they're in the box below."