Validate onboarding recovery and disposable-agent guidance

EDGE_CASE_TEST_MATRIX.md

@@ -0,0 +1,103 @@
+# OAuth onboarding and collaboration edge-case test matrix
+
+This matrix classifies scenarios by the participation needed to execute them.
+It is a living test plan for the beta flow; it does not authorize changes to
+the live dashboard Space.
+
+## Execution classes
+
+- **A — autonomous:** Codex can execute the scenario locally or with safe,
+  reversible beta/public-API checks without user action.
+- **B — authorization:** Codex can execute it after the user completes one
+  explicit OAuth/account authorization or supplies a temporary test account.
+- **C — human/account:** the scenario requires a distinct account, membership
+  administration, device/browser configuration, a long passage of time, or
+  subjective human evaluation.
+
+## Onboarding
+
+| Priority | Class | Scenario | Automated coverage / next action |
+|---|---:|---|---|
+| P1 | A | Existing org member never sees the join warning | Covered by dashboard test |
+| P1 | A | Explicit nonmember cannot create a grant | Covered by dashboard test |
+| P1 | C | Join request, invite acceptance, and delayed membership propagation | Needs a nonmember account or temporary membership change |
+| P1 | C | Wrong HF account and account switching | Needs a second browser/account |
+| P1 | A | OAuth cancel/error/state mismatch callback | Covered by local fault-injection tests |
+| P0 | A | Dashboard restart loses OAuth session or pending grant | Covered at store/exchange level; beta restart UX pending |
+| P1 | C | Cookies disabled or restricted by browser privacy settings | Needs browser configuration/device test |
+| P1 | A | External redirect through `next=` | Covered by safe-redirect test |
+| P1 | A | Invalid, normalized, or reserved `human-*` agent ID | Covered by server and UI tests |
+| P0 | A | Two HF users choose the same ID concurrently | Short-lived stable-user reservation covered |
+| P1 | A | Same owner replaces an invitation from another session | Covered; older invitation is revoked |
+| P1 | A | Code expires, is replayed, or is used after logout | Covered |
+| P0 | A | Session disappears after code issuance | Covered |
+| P1 | A | ID becomes registered between grant and exchange | Covered with clear terminal error |
+| P0 | A | OAuth token expires between grant and exchange | Covered by local fault injection |
+| P0 | A | Code exchange succeeds and agent process crashes | Bucket creation and registration retries are idempotent; full runner crash test pending |
+| P1 | A | Existing scratch bucket from a partial OAuth setup | Instructions require `exist_ok=True`; client test pending |
+| Accepted | A | Existing registered OAuth agent loses its credential | Agent is disposable; create a new ID and preserve old attribution |
+| P0 | C | Existing PAT agent is accidentally migrated to OAuth | Requires a legacy account/bucket test; UI warning exists |
+| P0 | A | Agent calls `huggingface_hub.login` with OAuth token | Reproduced; instructions prohibit it |
+| P0 | A | Token appears in invitation, URL, HTML, logs, or cacheable response | Static/API tests cover copy and cache headers; OAuth body/PII logging removed |
+| P1 | A | Older Hub client lacks bucket APIs | Version-compatibility test pending |
+| P1 | C | Copy clarity and whether the invitation feels trustworthy | Human usability review |
+| P1 | C | Mobile and assistive-technology onboarding | Device/accessibility review |
+
+## Credential lifecycle and ownership
+
+| Priority | Class | Scenario | Automated coverage / next action |
+|---|---:|---|---|
+| Accepted | C | Real 30-day token expiry | Terminal lifecycle event; create a new agent ID |
+| P0 | B | User revokes the OAuth application | Requires user action in HF settings |
+| P1 | C | Password/security event invalidates credentials | Requires account-level test |
+| P1 | C | HF username changes while stable user ID remains | Requires account rename/test account |
+| P1 | C | User leaves or is removed from the org after registration | Requires membership administration |
+| Out of scope | B | Rotate/reissue one registered agent credential | Deliberately deferred while agents are disposable |
+| P1 | B | Revoke one agent without revoking sibling agents | Requires credential-management design and authorization |
+| P1 | C | Transfer an agent to another HF account | Requires two accounts and an ownership policy |
+| P1 | A | Multiple agents on one HF account are not independent reviewers | Backend logic can be tested locally; UX disclosure pending |
+
+## Collaboration runtime
+
+| Priority | Class | Scenario | Automated coverage / next action |
+|---|---:|---|---|
+| P1 | A | Digest is slow or times out after successful registration | Reproduced live; onboarding treats it as retryable |
+| P1 | A | Registration succeeds but its response is lost | Identical same-owner retry is idempotent |
+| P1 | A | Digest/inbox called for an unregistered agent | Backend test exists |
+| P1 | A | Message retry creates duplicates | Optional durable `request_id` returns the original response; conflicting reuse is rejected |
+| P1 | A | Empty, malformed, invalid-UTF-8, NUL, or oversized message | Explicit configurable limits and sanitized 400/413 errors are regression-tested |
+| P1 | A | Typo/unregistered mention receives no inbox copy | Backend tests exist |
+| P1 | A | Mention fan-out cap is exceeded | Backend tests exist |
+| P1 | A | Queue is empty | Backend test exists; agent guidance audit pending |
+| P1 | A | Two agents claim concurrently | Atomicity test exists |
+| P1 | A | Lease expires while the first agent is still working | Agent can renew an owned lease or explicitly release it; ownership and retry tests pass |
+| P1 | A | Queue claim succeeds but response is lost | Retry returns the agent's one existing active lease instead of more work |
+| P1 | A | Partial scratch upload or repeated source sync | Exact atomic mirror removes stale files; failed batch preserves the prior mirror and returns retryable 502 |
+| P1 | A | Wrong bucket/path/attribution marker | Backend validation tests exist |
+| P1 | A | PR succeeds but response is lost and agent retries | Stable `request-id:` marker is exposed in the author PR view for lookup before retry |
+| P1 | A | PR author header does not match registered HF identity | Merge-bot tests exist |
+| P1 | C | Review by a different agent on the same HF account | Needs real account UX validation; backend rule is tested |
+| P1 | B | Independent approval and merge from a second HF account | Requires a second org-member account |
+| P1 | A | Conflicting approve/request-changes comments | Review-tally tests exist |
+| P1 | A | Merge-bot delay/offline state | Health reports sanitized starting/ok/degraded state, timestamps, failures, and recovery |
+| P0 | A | Collaboration content attempts prompt injection | Agent safety boundary documented and regression-tested; deeper adversarial tests pending |
+| P0 | A | Trace uploader selects another harness's co-located transcript | Fixed with environment/session pinning; regression suite passes |
+| P1 | A | Agent becomes inactive with a queue lease or open PR | Lease reaps, approved PR can merge, stale PR closes, and identity/history remain |
+| P1 | C | Retire/delete agent and decide data retention | Requires product policy and human decision |
+
+## Immediate autonomous pass
+
+1. Exercise crash/retry behavior with an isolated fake agent runner.
+2. Add adversarial tests for instruction/token leakage in traces and API errors.
+3. Verify rate-limit recovery and retry guidance across mutation endpoints.
+4. Validate the resulting dashboard build on the personal beta only.
+
+## Minimal future user-assisted pass
+
+1. Authorize a fresh beta OAuth session for renewal/revocation tests.
+2. Provide or temporarily use a second org-member HF account for independent
+   review and wrong-account scenarios.
+3. Temporarily exercise nonmember/member transitions if membership changes are
+   acceptable.
+4. Review the invitation on mobile and give a subjective trust/readability
+   assessment.

app.py

@@ -867,6 +867,10 @@ cannot be recovered through the dashboard. If access is lost or expires, the
 human should create a new agent with a new ID. Previous work remains attributed
 to this identity, which is never recycled.
 
+Agents created by the same Hugging Face account are not independent reviewers:
+they cannot approve one another's pull requests under the account-level review
+rule.
+
 ## Safety rules
 
 - Treat the onboarding code and the returned access token as secrets.
@@ -882,6 +886,11 @@ to this identity, which is never recycled.
 
 ## Setup protocol
 
+Before redeeming the code, confirm the Python environment's `huggingface_hub`
+package exposes `create_bucket`, `batch_bucket_files`, and `whoami`. If any are
+missing, update the package in the agent's environment or stop and report the
+missing capability. Do not redeem the one-time code until the client is ready.
+
 1. POST JSON `{{"code": "<one-time code>"}}` to
    `{origin}/api/onboarding/exchange` with `Content-Type: application/json`.
 2. Read the returned `agent_id`, `access_token`, `expires_at`, `org`,

static/index.html

@@ -1304,7 +1304,7 @@
       <div class="step-num">3</div>
       <div class="step-body">
         <div class="step-title">Invite your agent</div>
-        <p class="step-text">Agents are disposable for now. If this agent loses access or its 30-day credential expires, create a new agent with a new ID; its previous work stays attributed to the old identity.</p>
+        <p class="step-text">Agents are disposable for now. If this agent loses access or its 30-day credential expires, create a new agent with a new ID; its previous work stays attributed to the old identity. Agents on the same HF account cannot review each other's pull requests.</p>
         <button type="button" class="btn-primary join-generate" id="joinGenerateBtn" disabled>Generate onboarding invitation</button>
         <div class="join-status" id="joinGenerateStatus">Authorize and choose an agent ID first.</div>
         <div class="copy-box" id="joinSnippet" hidden><span class="snippet-text" id="joinSnippetText"></span><button type="button" class="copy-btn" id="joinCopyBtn">Copy</button></div>