audit/README.md

# `audit/` — the repo's self-verifying error/risk gate

One runnable command that asserts the repo is sound. **24 checks across 5 tiers.**
Every check traces to a real incident this project hit (LFS quota silent
failures, deleted-module import breakage, ChromaDB disk bloat, etc.) — the
auditor exists so those classes of failure cannot ship silently again.

The auditor itself is verified: `--selftest` reconstructs the broken state of
each incident in a temp fixture and asserts the matching check still flags it.
A check that no longer detects its incident is reported as "not self-verifying".

## CLI

```
tools/audit.sh --static|--build|--functional|--deploy|--all|--selftest [--json]
```

`tools/audit.sh` just execs `.venv/bin/python -m audit "$@"` from the repo root.
(Note: `tools/audit.sh` is the entrypoint — unrelated to the pre-existing
`tools/audit/` tool directory.)

`--json` emits the per-check results as a JSON array instead of the text
report (id / status / evidence / remediation).

## Tiers

| Flag | Runs | Checks | When to run |
|------|------|--------|-------------|
| `--static` | static | T1.* + T2.* (11) | **Pre-commit.** Seconds. Pure repo/AST/import/lint — no build, no network, no servers. |
| `--build` | static + build | + T3.* (14) | Before push. Runs the full `pytest` gate, `next build` (production static export), and boots the backend for `/api/health`. Minutes. |
| `--functional` | static + build + functional | + T4.* (20) | Needs a **local backend on :8000 and frontend** up. API smoke (health/coverage/chat/upload/profile) + Playwright E2E journeys. |
| `--deploy` | deploy only | T5.* (4) | **Read-only** prod/deploy safety: LFS pre-push simulation, Dockerfile coherence, deployed-SHA vs local, standing tripwires. |
| `--all` | static + build + functional + deploy | all 24 | Everything. Run with backend (and ideally frontend) up. |

## Exit codes

- Exit is **non-zero iff any check is `FAIL`**.
- `WARN` **never fails the gate** (it surfaces a soft / deferred condition,
  e.g. T5.3 deferred-deploy SHA mismatch or T2.3 a stale-doc note).
- `SKIP` (e.g. a prerequisite service is down) also does not fail the gate.

## `--selftest`

```
.venv/bin/python -m audit --selftest        # full: 24 checks
```

For every check, a fixture stands up the broken state from the original
incident; the check must return its `selftest_expect` status on that fixture
(`FAIL` for most; `WARN` for T5.3 / T5.4 which are advisory by design). Output:
`24 checks · 0 not self-verifying`. A non-zero "not self-verifying" count means
the auditor has silently rotted and is no longer catching its own incident.

The pytest gate (`tests/test_audit_selftest.py`) runs the selftest **scoped to
the static tier only** — `core.selftest(only_tiers={"static"})`. The
build/functional/deploy tiers are deliberately excluded from the unit gate:
T3.1's fixture shells `pytest`, which would recurse into that very test; T3.2
runs a multi-minute `next build`; Tier 4 needs a live backend. The full
24-check selftest is run via `tools/audit.sh --selftest` (manual / pre-push).

## Opt-in git hooks

`.githooks/` ships a `pre-commit` (`--static`) and a `pre-push`
(`--build` then `--deploy`). They are **opt-in** — enable per clone with:

```
git config core.hooksPath .githooks
```

This repo does not set that automatically; nothing changes your git config
until you run the command above.

## Tier 5 is read-only against prod

Every Tier 5 check is strictly read-only with respect to deployed
infrastructure: it inspects local artifacts (Dockerfile, LFS attributes,
tripwire state) and at most *reads* the deployed SHA for comparison. It never
pushes, deploys, mutates remote state, or touches the HF Space — so `--deploy`
and `--all` are safe to run at any time.