DocDoeAI / STORAGE_SETUP.md
asnannp's picture
deploy: sync backend to Space root (learn-lesson HF cache fix)
6515ef9
|
Raw
History Blame Contribute Delete
4.28 kB
# Cloud Storage Setup (R2 / S3-compatible)
DocDoe serves generated student audio and video through owner-authenticated API
routes. Keep any backing object store private; public bucket URLs are not an
acceptable delivery mechanism for student-generated artifacts.
## When to switch
| Env | STORAGE_PROVIDER | Notes |
|---|---|---|
| Local dev | `local` (default) | Files served from `/uploads`, `/generated/audio`, `/generated/videos` |
| Production (single-instance) | `r2` or `s3` | Required for stable public video URLs |
| Production (multi-instance) | `r2` or `s3` | Required β€” local volumes don't survive redeploys |
## Cloudflare R2 (recommended)
R2 is S3-compatible, zero egress fees, free tier up to 10GB.
### 1. Create bucket
1. Cloudflare Dashboard β†’ R2 β†’ Create bucket
2. Bucket name: `docdoe-media`
3. Region: auto
### 2. Enable public access
Option A β€” R2.dev subdomain (fastest):
- Bucket β†’ Settings β†’ "Public access" β†’ Allow access via R2.dev subdomain
- Public URL prefix: `https://pub-<hash>.r2.dev`
Option B β€” Custom domain (recommended for production):
- Bucket β†’ Settings β†’ "Public access" β†’ Connect custom domain
- Domain: `media.docdoe.ai`
- Cloudflare auto-creates CNAME
### 3. Create API token
1. Cloudflare Dashboard β†’ R2 β†’ "Manage R2 API Tokens"
2. Create token β†’ "Object Read & Write" permission
3. Apply to bucket: `docdoe-media`
4. Copy Access Key ID + Secret Access Key (shown once)
### 4. Set env vars
```bash
STORAGE_PROVIDER=r2
STORAGE_BUCKET=docdoe-media
STORAGE_ENDPOINT_URL=https://<your-account-id>.r2.cloudflarestorage.com
STORAGE_ACCESS_KEY_ID=<R2 access key>
STORAGE_SECRET_ACCESS_KEY=<R2 secret>
STORAGE_PUBLIC_BASE_URL=https://media.docdoe.ai # or https://pub-<hash>.r2.dev
```
`STORAGE_REGION` not needed for R2.
### 5. CORS on bucket
R2 β†’ Bucket β†’ Settings β†’ CORS Policy. Add:
```json
[
{
"AllowedOrigins": ["https://docdoe.ai", "https://www.docdoe.ai"],
"AllowedMethods": ["GET", "HEAD"],
"AllowedHeaders": ["*"],
"MaxAgeSeconds": 3600
}
]
```
For dev/preview, add Vercel preview origins or `https://*.vercel.app`.
## AWS S3 (alternative)
Same env vars, set `STORAGE_PROVIDER=s3` and `STORAGE_REGION=us-east-1` (or region of bucket). Endpoint URL not needed β€” boto3 derives it. Public URLs use `https://<bucket>.s3.<region>.amazonaws.com/<key>` unless `STORAGE_PUBLIC_BASE_URL` overrides.
### S3 bucket policy (public read)
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::docdoe-media/*"
}
]
}
```
Plus enable "Block all public access" β†’ uncheck for this bucket.
## Verification
After deploy:
```bash
# Backend health
curl -s https://api.docdoe.ai/health
# Generate a small test video via /video-generator
# Then list video jobs:
curl -s -H "Authorization: Bearer <jwt>" https://api.docdoe.ai/video/render-jobs
# Final video URL should look like:
# https://media.docdoe.ai/generated-videos/<job-id>/output.mp4
# NOT like:
# http://localhost:8000/generated/videos/<job-id>/output.mp4
```
If you see localhost URLs in `public_url` field, `STORAGE_PROVIDER` is still `local`. Fix env and redeploy.
## Object key layout
| Key prefix | Content |
|---|---|
| `generated-audio/<job-id>/scene-XXX.wav` | TTS per-scene audio files |
| `generated-videos/<job-id>/output.mp4` | Final rendered MP4 |
| `uploads/<user-id>/<doc-id>.<ext>` | Original student uploads |
Local provider rewrites keys into `/generated/audio/...`, `/generated/videos/...`, `/uploads/...` paths served by FastAPI `StaticFiles`. R2/S3 provider serves directly from `STORAGE_PUBLIC_BASE_URL`.
## Failure modes
| Symptom | Cause | Fix |
|---|---|---|
| Video plays in dev, 404 in production | `STORAGE_PROVIDER=local` in prod | Switch to `r2`/`s3` |
| `boto3 is not installed` error | Image built without `pip install boto3` | Rebuild Dockerfile.backend (boto3 in requirements.txt) |
| Cloud upload fails | Missing access key, secret, or bucket var | Check `/health/deep` JSON for storage status |
| Video plays but no audio | Render container missing ffmpeg | Use Dockerfile.backend (includes ffmpeg) |