Testing guide

Everything built so far, as click-through scenarios: what to do and exactly what you should see. Every dataset referenced is downloadable right here - all texts are synthetic, written for this kit; no real participant data anywhere. This is the lab's internal testing guide; the plain how-to-use walkthrough for everyone lives at the public guide.

Quickstart: run YOUR study in 5 minutes

1. Sign in (top right - Google or email+password). Lab accounts have unlimited access.
2. Create a project named after your study.
3. Upload your corpus - any CSV/XLSX with one text per row. Other columns (IDs, conditions, demographics) are fine; they pass through untouched into the results file.
4. Pick your construct(s) - search the library (94 scales from the lab's collection), or click "+ New construct" and choose a card: type/paste items, upload a file, or draft items with AI for a construct with no validated questionnaire. Pick several (up to 10) to score them together in a single run and see how they correlate in your texts.
5. Run. Read the warnings panel first (it tells you if something about your data is off), then check the top/bottom scoring texts - if those don't make sense for your construct, trust that instinct and report it.
6. Export the results CSV (your columns + per-item similarities + CCR score) and, if you want, the Python script that reproduces the run on your own machine.

The sections below are structured test scenarios with sample data - useful for systematically stress-testing the platform, but your real data is the best test we have.

Limits at a glance

Accounts come in four tiers, set by the admins: external user (the default on sign-up), lab member, maintainer, and PI. Signing in lifts the anonymous caps; lab members and above have no saved-run cap. The row limit is usually what you hit first, not the file size.

Every model below is available on all three, including signed out.

Models available

Chosen on the Step 3 card. Scores are only comparable within the same model - never across model families. Larger models are slower and, the first time anyone selects one on this instance, take a couple of extra minutes to download before the run starts; after that they stay warm.

0. Sample datasets

One file per platform behavior. To test something, download the file named for it and follow its scenario below. Unless stated otherwise, select language English and model MiniLM.

1. Projects and sidebar

1. Create three projects. They appear under "Today", newest activity first.
2. Type in the sidebar search box: the list filters as you type.
3. Archive a project (project header > Archive): it moves into the collapsed "Archived" group; Unarchive brings it back. No data is lost either way.
4. Delete a project: requires typing the project name; removes its datasets, runs, and files permanently.

2. Upload paths (Step 1 card)

Anonymous limits (signed out): the Step 1 hint shows 5 MB / 200 rows and says uploads are deleted after analysis. Upload large_demo.csv (800 rows): rejected with a "Sign in (top right)" message. Sign in and retry: accepted.

3. Construct selection (Step 2 card)

1. Open the picker: search field + panel below it, library grouped by category, with "Recently used" pinned on top after your first runs.
2. Type "GAD" or "empathy": matches by name and category; Arrow keys + Enter work.
3. Select any imported construct: items listed, plus the "not yet verified verbatim" notice (expected for the whole imported library for now).

Multi-construct runs (new)

1. Pick a construct, then open the picker again and pick a second - selected ones show a ✓ (clicking again removes). Each selection becomes a collapsible block with its items and a "remove" link. Up to 10 constructs per run.
2. The run button reads "Run CCR analysis (2 constructs)". All constructs are scored on one pass over the corpus, so two constructs take about as long as one.
3. Results open with a "Construct interrelations" card - the Pearson correlation between per-text scores, i.e. how the constructs co-occur in YOUR texts - plus a collapsible per-construct section (histogram, item loadings, top/bottom texts). Try moral_foundations_demo.csv with two MFQ-2 foundations.
4. The export CSV keeps one row per text with per-construct prefixed columns (mfq_care_sim_item_1 … mfq_care_ccr_score, …), so the correlations are fully recomputable; metadata and the reproduction script cover every construct in the run. Single-construct runs are unchanged.

Custom construct, typed

1. "+ New construct" > the "Type or paste" card > name it, paste items one per line.
2. Append (R) to one line: the form shows "1 item(s) marked reverse-scored".
3. Save: it appears in the picker under "My custom constructs"; run metadata carries the reverse flag (check via Results > metadata download).

Custom construct, from file

1. "+ New construct" > the "Upload a file" card.
2. Try construct_items_demo.csv (item,reverse columns - 1/true/yes/R = reverse), construct_items_marker_demo.csv (single column with (R) markers), or construct_items_demo.xlsx (Excel).
3. Expect: items fill the textarea ((R) appended where flagged), the filename becomes the suggested name, and parse notes list skipped duplicates. Nothing is saved until you review and press Save. Item files are never retained on the server.

Custom construct, drafted with AI (new)

For constructs with no validated questionnaire: the platform can draft candidate items from the construct's name and a short explanation. The draft is a starting point, not a validated scale - you review, edit, and delete before saving, and everything the construct touches is labeled "AI-generated · not validated". Signed-in users only, 20 drafts/day.

1. Happy path: sign in > "+ New construct" > the "Draft with AI" tab. Name: Digital overwhelm. Description: "Feeling that screens, notifications, and online demands exceed one's capacity to keep up." Press "Draft items". Expect in a few seconds: ~10 first-person, positively-worded items in the textarea (no (R) items - by design), a "0 of 20 used today" style counter, an amber "AI-generated · not validated - drafted by <model>" notice, and sometimes short model notes (e.g. which facets it covered).
2. Review is the point: edit one item, delete a weak one, then Save. The construct appears in the picker under "My custom constructs" with an AI-generated · not validated tag - the tag stays even though you edited, because the seed was AI (the item hash records your edits).
3. Library guardrail: on the same tab, type Satisfaction with Life as the name. Expect a warning that the library already has this scale with validated items - use that instead of generating.
4. It follows your definition: draft the same name twice with two different descriptions (e.g. define "resilience" once as bouncing back from setbacks, once as tolerating discomfort). The items should track YOUR wording, not a generic textbook version - that is the feature working.
5. Vague input: give a nonsense name (Zorblex) with a vague description. Expect items anyway, plus model notes explaining it could not identify a standard construct - refine the description and redraft.
6. Run + provenance: run any corpus against your saved AI construct. The results page shows a caution line; the metadata JSON download has source_type: "llm_generated", the drafting model + prompt version + date, and a machine-readable cautionary note. This travels into the reproduction script too.
7. Signed out: the "Draft with AI" tab shows a sign-in nudge instead of controls; the API refuses anonymous calls outright.

4. Language, models, and warnings (Step 3 card + results)

Run each of these and open the results page; the amber warnings panel should show exactly:

Warnings are per-run snapshots: changing language/model requires a NEW run; old result pages don't update.

5. Results and reproducibility

1. Run moral_foundations_demo.csv against two different MFQ-2 foundations (separately, or both in one multi-construct run): top texts change per foundation; the 6 neutral rows sink to the bottom.
2. Results page: histogram, mean/SD/min/max, per-item loadings, top/bottom texts; multi-construct runs add the correlation matrix up top.
3. Downloads: results CSV (input columns + sim_item_N + ccr_score; multi-construct runs prefix these per construct), metadata JSON (model revision, construct snapshot + item hash, language block, environment pins), reproduction script + requirements file.
4. Reproduction check: both downloads carry your run's id, e.g. pip install -r requirements-repro_<run-id>.txt, then python reproduce_analysis_<run-id>.py your_corpus.csv on a machine with no platform access; the exact commands (with your run id and CSV name) are in the script's header. Values should match the export (target ~1e-5 with real models).

6. Accounts

1. Sign in (top right) > "Create a free account" > email + password (min 8 chars) - or use "Continue with Google".
2. You're signed in immediately; the header shows your name.
3. Sign out, sign back in; wrong password gives "Incorrect email or password"; registering the same email again gives "already exists".
4. Email is case-insensitive. No self-service password reset yet - reset = admin action. (Accounts persist across restarts now - no need to re-register.)

7. Anonymous tiers (test signed OUT)

1. Upload caps: see section 2.
2. Run limit: run 3 analyses. The Step 3 card counts "X of 3 free runs used today". The 4th run is refused with a sign-in prompt. Counter resets next day (UTC). Signing in removes the limit.
3. Delete-after-analysis: run any corpus, open results (fine, downloadable), note the info warning "uploaded file was deleted after this analysis". Re-running that same corpus: refused ("upload again, or sign in").
4. TTL purge: anonymous projects older than 24 h are deleted entirely (startup + hourly).

8. Signed-in tier

1. Sign in, upload, run: no ANONYMOUS_DATA_REMOVED warning; re-running the same corpus works (file kept).
2. Saved-run budget: the Step 3 card shows "N of M saved runs used". Lab members and above have no cap; external accounts get 15. At the cap, new runs are refused until you delete old runs/projects (nothing is auto-deleted).
3. Ownership: your projects are invisible to signed-out visitors and other accounts. Anonymous projects stay shared.

9. Performance behaviors

1. Corpus-embedding cache: run the SAME corpus with a second construct (signed in, same model): the run skips document embedding and completes in seconds; metadata shows "doc_embeddings_from_cache": true.
2. Duplicate texts are embedded once (warnings_showcase.csv has 2 dupes): identical scores for identical texts, less compute.

10. Under the hood

Architecture, data flow, data retention, and the full access/roles model (tiers, invite links, pre-assigned roles, audit trail) live on their own page: Product & Architecture →. This guide stays focused on using and testing the platform.

11. Found something off?

Anything that doesn't match what this guide says it should do - or anything confusing, slow, or missing - post it in the lab's #ccr Slack channel: the 🐞 thread for bugs, the 💡 thread for ideas and feature requests. One line is enough; note the section number and what you saw; screenshots help. DMs to Deva work too, and email as a fallback (devaanand@umass.edu). Nothing is too small - "this button confused me" is exactly the kind of report we want.