# Playbook — deployed data

## What "backed up" means here

The mutable production data — databases, user uploads, bucket contents,
queue state, anything that's irreplaceable if the VPS dies — is
**replicated to at least 1 off-host destination on a regular cadence**,
**and at least one restore has been successfully tested**.

Two halves to that:
1. **Replication** — fresh copies land somewhere off this host.
2. **Restoration tested** — at least once, you've verified the copies
   are usable. A backup you've never restored isn't a backup.

## When this playbook does NOT apply

Stateless projects (a static site, a stateless API behind a managed
DB) have no deployed data of their own. If that's the case, delete
this playbook and the BACKUP.md row. The managed DB's vendor backups
are *their* concern; verify those separately if you care, but it's
out of scope here.

## Verification steps

1. **Inventory what data this project has.**

   Read `_project/ARCHITECTURE.md`. Look for: a database section, file
   uploads, bucket / blob storage, persistent caches.

   If ARCHITECTURE.md doesn't mention any data, double-check by
   asking the user. A surprising number of projects "don't have data"
   right up until you remember the user upload directory.

2. **Find where the data lives on the VPS.**

   Common paths:

   ```bash
   vps-proxy run "ls -lh /var/lib/postgresql/ /var/lib/mysql/ 2>/dev/null"
   vps-proxy run "df -h --output=source,target,size,used | grep -v tmpfs | head -20"
   ```

   And from RUNBOOK.md: any `data/`, `uploads/`, `storage/` paths the
   app uses.

   - Found → continue.
   - No data on disk and no buckets used → playbook doesn't apply.

3. **Backup mechanism and cadence.** (Ask the user.)

   > For each data source above, what mechanism backs it up, where, and
   > how often?

   Examples of acceptable answers:
   - Postgres: `pg_dump` cron daily, uploaded to B2 via rclone, 30-day
     retention
   - User uploads: rsynced to a second VPS nightly, restic to S3 weekly
   - Bucket data: bucket has versioning + replication enabled to a
     second region

   Record per-source. A single "backups go to S3" answer covering five
   data sources is suspicious — verify each one.

4. **Confirm freshness.** (Ask the user, then spot-check.)

   For each data source, what's the timestamp of the most recent
   successful backup?

   When possible, verify via vps-proxy:

   ```bash
   # Example for an rclone-pushed pg_dump:
   vps-proxy run "ls -lt /var/backups/postgres/ | head -3"
   # For a backup script's log:
   vps-proxy run "tail -20 /var/log/backup.log 2>/dev/null"
   ```

   - Latest backup younger than the cadence → ok.
   - Latest older than 2× cadence → hard gap (silently broken).
   - Older than the project's tolerance → soft gap.

5. **Restore test on record.**

   Read the "Restore tests" section in `_project/BACKUP.md`. At least
   one entry?

   - Yes, within the last 6 months → ok.
   - Yes, > 6 months ago → soft gap. Schedule the next one.
   - None ever → hard gap. Even one test, even on a small subset, is
     the difference between "we have backups" and "we hope we have
     backups."

6. **Restore procedure documented.**

   Read `_project/RUNBOOK.md` for a restore section: what to run, in
   what order, with what credentials, to recover the data.

   - Section exists with concrete commands → ok.
   - Vague ("restore from B2") → soft gap.
   - Missing → soft gap.

## What counts as a gap

| Severity | Condition | Example finding line |
|---|---|---|
| Hard | Data source identified with no backup mechanism | `[backup] deployed-data: <source> has no backup configured` |
| Hard | Backup older than 2× cadence (silently broken) | `[backup] deployed-data: <source> last backup <date>, expected <cadence>` |
| Hard | No restore test ever performed | `[backup] deployed-data: no restore test on record — exercise restore for <source>` |
| Soft | Restore test > 6 months old | `[backup] deployed-data: last restore test was <date> — schedule a new one` |
| Soft | Restore procedure missing or vague in RUNBOOK | `[backup] deployed-data: RUNBOOK lacks concrete restore commands` |
| Soft | Data inventory not in ARCHITECTURE.md | `[backup] deployed-data: data sources not documented in ARCHITECTURE.md` |

## What to put in BACKUP.md

- Hard gaps unresolved → **gaps**
- Hard closed, soft may remain → **ok**
- Couldn't reach the user (e.g. running via `/loop` unattended) →
  **unverified** with a note that off-host confirmation needs a human.