deploy/PRODUCTION_RUNBOOK.md

Production Deployment Runbook

Moving the CCR Platform from the personal dev setup to the lab's own accounts, and putting it on a custom domain. Written for a one-sitting migration: the code is already unified in the lab GitHub repo, the app is feature-complete, and data already lives on managed services (Postgres + object storage). This runbook carries all of it over to lab-owned accounts.

Terminology: "the Space" is the Hugging Face Space that runs the app. "Custom domain" is whatever the PI picks (examples below use ccr.culturemoralitylab.org; swap in the chosen domain everywhere).

There are no em dashes in this file on purpose (project rule).

---

0. What is already done

• Code is unified in Culture-and-Morality-Lab/ccr-platform (public), with clean commit history.
• License texts are staged in the repo root (LICENSE-MIT, LICENSE-APACHE); one gets promoted to LICENSE once the PI picks.
• The app is one portable container (Dockerfile); the same image runs on any host, so "deploy" is a git push to the Space.
• Data services are already in use on the dev instance (Supabase Postgres via DATABASE_URL, Cloudflare R2 via the CCR_S3_* secrets), so this is a copy-to-lab-accounts job, not new infrastructure.

1. Accounts to create (lab email, free)

Service	Purpose	Cost
Hugging Face (account or org)	Hosts the Space	Free (cpu-basic)
Supabase	Postgres database + Google sign-in	Free tier
Cloudflare	Domain registrar + DNS + R2 storage + the reverse-proxy Worker	Free (domain ~20/yr; R2 needs a card on file but is free at this scale)
Groq	AI item-drafting key	Free tier
GitHub	Already done (lab org repo exists)	Free

Save every password and key in a password manager as you go. These are the lab's institutional credentials, not personal logins.

2. Secrets sheet (fill privately, never commit)

The new Space needs these. Collect the values as you complete each section below, then paste them into the Space in section 7.

Name	Store	Value source
CCR_SESSION_SECRET	Secret	Generate fresh: python3 -c "import secrets; print(secrets.token_hex(32))" (do NOT reuse the dev value)
DATABASE_URL	Secret	New Supabase project, Session pooler URI, with the DB password in it (section 3)
SUPABASE_URL	Secret	New Supabase project > Settings > API
SUPABASE_ANON_KEY	Secret	Same page (anon public key, NOT service_role)
CCR_STORAGE	Variable	s3
CCR_S3_ENDPOINT	Secret	New R2: https://<account_id>.r2.cloudflarestorage.com (section 5)
CCR_S3_BUCKET	Secret	New R2 bucket name
CCR_S3_ACCESS_KEY_ID	Secret	New R2 API token
CCR_S3_SECRET_ACCESS_KEY	Secret	New R2 API token
GROQ_API_KEY	Secret	New lab Groq key
CCR_APP_URL	Variable	https://<custom-domain> (section 8)
CCR_COOKIE_SECURE	Variable	1
CCR_ANON_TTL_HOURS	Variable	24
ADMIN_EMAILS	Variable	Comma-separated admin emails (confirm with PI)
CCR_MAX_ROWS	Variable	50000 (matches the dev instance)

Hugging Face keeps Variables and Secrets in two separate stores. A name defined in BOTH puts the Space into CONFIG_ERROR before it builds. Put each name in one store only, per the table.

3. Supabase project (database + sign-in)

1. New project under the lab's Supabase org. Match the region to the current dev project (read it from the dev DATABASE_URL: the pooler host contains it, e.g. aws-0-us-east-1... means US East). Generate a strong DB password and save it.
2. Turn OFF "Enable Data API" (the app talks to Postgres directly and never uses the REST API; leaving it off closes that surface). Leave "Enable automatic RLS" on. Do NOT connect GitHub (the app manages its own schema).
3. Settings > API: copy the Project URL and the anon key into the secrets sheet (SUPABASE_URL, SUPABASE_ANON_KEY).
4. Connect > Session pooler: copy the URI, insert the DB password, into DATABASE_URL.
5. Google sign-in (do this before launch; the button is hidden without it):
   • Google Cloud console (lab Google account) > OAuth consent screen (External)

     Credentials > Create OAuth client ID (Web application).

   • Supabase > Authentication > Providers > Google > Enable. Copy the shown callback (https://<PROJECT_REF>.supabase.co/auth/v1/callback) and add it as an authorized redirect URI in the Google client. Paste the Google client id and secret back into the Supabase Google form.
   • Supabase > Authentication > URL Configuration: add the redirect URLs http://127.0.0.1:8000/api/auth/google/callback and, once the domain is live, https://<custom-domain>/api/auth/google/callback.

4. Migrate the database (carry accounts and data over)

Run BEFORE the new Space boots for the first time, so the app finds the data already present rather than creating empty tables.

# Dump the app's tables from the current (dev) database. Use the dev
# DATABASE_URL from your local .env. --schema=public keeps it to the app's
# tables; --no-owner --no-acl avoids role mismatches between projects.
pg_dump "$DEV_DATABASE_URL" --schema=public --no-owner --no-acl -Fc -f ccr_public.dump

# Restore into the NEW Supabase project.
pg_restore --no-owner --no-acl --clean --if-exists \
  -d "$NEW_DATABASE_URL" ccr_public.dump

If pg_dump version-mismatches against Supabase, use the Postgres client that matches the server major version (or the supabase db dump CLI). The app's startup auto-migration adds any missing columns after this, so a slightly older dump still boots cleanly.

5. R2 object storage (uploaded files)

1. Cloudflare > R2 > create a bucket under the lab account (this is when R2 asks for a card on file; the 10 GB free tier covers this use).
2. Create an R2 API token (Object Read and Write) scoped to that bucket. Put the access key id, secret, endpoint, and bucket name into the secrets sheet.
3. Copy the existing files over (R2 is S3-compatible):

# From the dev R2 to the lab R2. Fill both endpoint/keys from each account.
aws s3 sync \
  --endpoint-url "$DEV_S3_ENDPOINT" "s3://$DEV_BUCKET" ./r2-migrate-tmp
aws s3 sync \
  --endpoint-url "$LAB_S3_ENDPOINT" ./r2-migrate-tmp "s3://$LAB_BUCKET"
# (or use rclone with two remotes: rclone sync dev:bucket lab:bucket)

6. Domain purchase (on the PI call, card needed)

1. Cloudflare > Domain Registration > register the chosen domain (recommended: buy at Cloudflare so domain, DNS, R2, and the Worker share one account). Availability was last checked recently and can change daily, so register the same day it is chosen.
2. If the domain is bought elsewhere, add it to Cloudflare and point its nameservers at Cloudflare, so the Worker route in section 8 can attach.

7. Create the lab Space and add secrets

1. New Space under the lab Hugging Face account: SDK = Docker, hardware = cpu-basic (free, same as dev). Give it the same repo.
2. Point the local repo's Space remote at the new Space and push:

git remote set-url hf https://huggingface.co/spaces/<lab-owner>/<space>
git push hf main    # use the lab HF username + a WRITE token when prompted

3. Space > Settings > Variables and secrets: enter every row from the secrets sheet (section 2), each in the store the table specifies.
4. The Space builds and boots. Because the database was restored in section 4, accounts and data are already there.

8. Wire the custom domain (reverse proxy)

Hugging Face Spaces cannot serve a custom domain directly, so a small Cloudflare Worker serves the Space at the lab domain and keeps the domain in the address bar.

1. Get the Space direct host: Space > Settings > Embed this Space > Direct URL, of the form https://<owner>-<space>.hf.space.
2. Open deploy/reverse-proxy-worker.js, set UPSTREAM_HOST to that host (without the https://).
3. Cloudflare > Workers & Pages > Create Worker > paste the file > Deploy.
4. That Worker > Settings > Domains & Routes > Add Custom Domain > ccr.<domain> (Cloudflare provisions TLS automatically).
5. Set the Space Variable CCR_APP_URL=https://ccr.<domain> and confirm the Supabase redirect URL from section 3.5 uses the same domain. Restart the Space so it picks up CCR_APP_URL.

9. Launch checklist (verify before announcing)

• Site loads on the custom domain; the address bar stays on the domain while navigating.
• Google sign-in completes and returns to the custom domain (not hf.space).
• Password sign-in works; an account created on the dev instance is present (confirms the data migration).
• Upload a small corpus, pick a construct, run it, open results.
• AI drafting: "Draft with AI" tab works with the lab Groq key; the drafted items appear, and a saved AI construct shows the "AI-generated, not validated" label through picker, results, and the metadata download.
• /guide loads; /admin is reachable for the emails in ADMIN_EMAILS.
• Promote the license: git mv LICENSE-MIT LICENSE && rm LICENSE-APACHE (or the reverse), commit, push. Needed because the site publicly says the tool is open-source.

10. Cut over and retire the old instance

• Pause (do not delete) the old personal Space once the new one is verified, so it is a fallback for a day or two.
• Keep the dev database and R2 until the new instance has run cleanly for a few days, then retire them.

---

Optional, post-launch

Keep the Space awake (free)

Free Spaces sleep after about 48 hours idle (first visitor then waits about a minute). A free uptime pinger (for example UptimeRobot) hitting the domain every few hours keeps it warm without paying. The durable fix is an always-on host (next item).

Always-on host (later)

The same container runs on any host (Fly, Railway, Hetzner, a UMass VM). Moving there removes the sleep behavior AND gives native custom-domain support, so the Worker in section 8 can be dropped. Not a launch blocker.

One-push deploys (GitHub Action)

Auto-deploy the Space on every merge to main, so nobody needs the Space remote. Add a GitHub Actions workflow that mirrors main to the Space, with an HF write token stored as a GitHub Actions secret (HF_TOKEN):

# .github/workflows/deploy-hf.yml
name: Deploy to Hugging Face Space
on:
  push:
    branches: [main]
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }
      - name: Push to Space
        env:
          HF_TOKEN: ${{ secrets.HF_TOKEN }}
        run: |
          git push "https://lab:${HF_TOKEN}@huggingface.co/spaces/<lab-owner>/<space>" main

Switch AI drafting to Claude Haiku (later)

Add ANTHROPIC_API_KEY as a Space secret and the app auto-selects Anthropic over Groq (CCR_GENERATION_PROVIDER can force either). Groq stays as the free fallback. The provenance stamp records which model drafted each construct, so constructs drafted during the Groq period stay traceable.