Arag / .agent /PLATFORM_API_RULES.md
AuthorBot
Document feature rules for platform APIs, TOTP, and pricing.
34e47a7
|
Raw
History Blame Contribute Delete
6.76 kB
# Feature Rules β€” Platform API Credentials (SuperAdmin)
> Status: **Implemented** (`ad1b7c6`). Source of truth for retailer API key management.
## 1. Problem & scope
- **User story:** As platform operator, I enter optional paid retailer API keys in SuperAdmin so price/presence scans use reliable APIs instead of blocked free scraping β€” without exposing keys in env files or logs.
- **In scope:**
- SuperAdmin UI to create/update/disable/delete per-platform credentials
- Fernet encryption at rest in PostgreSQL
- Provider registry (Rainforest, ScrapingBee, Google Books)
- Priority resolution: SuperAdmin DB β†’ env fallback β†’ free logic
- Integration with `fetch_listing_price()` during platform presence scans
- Audit log on every mutation
- Masked read API (never return raw keys)
- **Out of scope:**
- Author-facing credential management (SuperAdmin only)
- Billing/metering of third-party API usage
- Presence scanning via paid APIs (future β€” pricing only today)
- Goodreads / Open Library paid APIs (none exist)
## 2. Architecture decisions
| Decision | Choice | Why |
|----------|--------|-----|
| Storage | `platform_api_credentials` table | Auditable, encrypted, UI-manageable |
| Encryption | Fernet derived from `SECRET_KEY` | R-174; no plaintext in DB |
| Resolution | `platform_api_registry.resolve_platform_credential()` | Single place for priority chain |
| Providers | Strategy in registry (`fetch_price_via_credential`) | Open/closed β€” add provider without changing routes |
| Cache | Redis `platform_api_cred:v1:{platform_id}`, TTL 300s | Performance; invalidate on write (R-179) |
| Routes | Thin router β†’ `PlatformApiService` | SRP; no logic in handlers |
| Env fallback | Documented in `PLATFORM_API_META` | Bootstrap only β€” not primary UX |
Principles:
- **Single responsibility** β€” `PlatformApiService` CRUD; registry resolves/fetches
- **Open/closed** β€” new provider = new branch in registry, not new endpoint
- **Dependency inversion** β€” pricing calls registry, not HTTP directly from presence
- **Thin client** β€” UI posts key once; server encrypts, masks, audits
- **Fail secure** β€” no key β†’ fall through; never log raw secrets
## 3. Data & security
- **Tables/columns:** `platform_api_credentials` (see migration `003_platform_api_credentials`)
- **Encryption at rest:** yes β€” `credential_crypto.encrypt_secret()` / Fernet
- **Who can read/write:** `role=superadmin` only; mutations require JWT + TOTP (`X-TOTP-Code`)
- **Audit log actions:** `upsert_platform_api`, `disable_platform_api`, `delete_platform_api`
- **Never log:** raw `api_key`, `api_secret`, decrypted values; audit uses `mask_secret()` only
## 4. API contract
| Method | Path | Auth | Request schema | Response |
|--------|------|------|----------------|----------|
| GET | `/api/super/platform-apis` | JWT + TOTP | β€” | `list[PlatformApiCredentialView]` |
| PUT | `/api/super/platform-apis/{platform_id}` | JWT + TOTP | `PlatformApiCredentialUpsert` | masked view |
| POST | `/api/super/platform-apis/{platform_id}/disable` | JWT + TOTP | β€” | masked view |
| DELETE | `/api/super/platform-apis/{platform_id}` | JWT + TOTP | β€” | masked view |
- Pydantic only (`PlatformApiCredentialUpsert`) β€” R-180
- `platform_id` in path must match body on PUT
## 5. UI
- **Location:** SuperAdmin β†’ **Platform Settings** β†’ **Retailer API Keys**
- **Complete UX path:**
1. Login (JWT)
2. Complete TOTP enrollment if not configured (see `SUPERADMIN_TOTP_RULES.md`)
3. Enter 6-digit TOTP in settings header
4. Per platform: select provider β†’ paste API key β†’ Save / Disable / Remove
5. Env-sourced keys show read-only `env` badge
- **Error states:** missing TOTP β†’ toast; invalid provider β†’ 400; unknown platform β†’ 400
## 6. Priority / fallback chains
For each `platform_id` on **price fetch**:
1. SuperAdmin DB row (`is_enabled=true`) β†’ `fetch_price_via_credential()`
2. Environment variable (see table below)
3. Built-in free logic (`platform_pricing.py`)
If step 1 fails (HTTP/parse), fall through to 2 then 3.
| platform_id | SuperAdmin providers | Env fallback | Free fallback |
|-------------|---------------------|--------------|---------------|
| amazon | rainforest, scrapingbee | `RAINFOREST_API_KEY` | US-locale HTML |
| google_books | google_books | `GOOGLE_BOOKS_API_KEY` | Anonymous Books API |
| barnes_noble | scrapingbee | β€” | HTML patterns |
| apple_books | scrapingbee | β€” | iTunes Lookup API |
| kobo | scrapingbee | β€” | HTML (often blocked) |
| thriftbooks | scrapingbee | β€” | HTML (often blocked) |
| abebooks | scrapingbee | β€” | HTML patterns |
| bookshop | scrapingbee | β€” | HTML (often blocked) |
| goodreads | β€” | β€” | from-price rollup |
| open_library | β€” | β€” | catalog "Free" |
**Never** instruct operators to use HF Secrets or `.env` as the primary key entry path. Env is emergency/bootstrap only.
## 7. Provider contracts
### `google_books`
- `GET https://www.googleapis.com/books/v1/volumes/{id}?key=...&country=US`
- ISBN fallback via `q=isbn:{isbn13}`
### `rainforest` (Amazon only)
- `GET https://api.rainforestapi.com/request?api_key=...&type=product&amazon_domain=amazon.com&asin={asin}`
### `scrapingbee`
- `GET https://app.scrapingbee.com/api/v1?api_key=...&url={listing_url}&premium_proxy=true&country_code=us`
- Parse via `extract_price_from_html()`
## 8. Tests
- [x] Unit β€” `tests/unit/test_platform_api_registry.py` (priority, crypto, parsers)
- [x] Unit β€” `tests/unit/test_platform_api_service.py` (upsert, audit, delete)
- [ ] Integration β€” SuperAdmin upsert happy path (mocked DB)
- [ ] BDD β€” see `.agent/BDD_SPECS.md` β†’ Platform API Credentials
## 9. Definition of done
- [x] Rules doc approved
- [x] Migration `003_platform_api_credentials`
- [x] FILE_MAP.md updated
- [x] End-to-end UI (requires TOTP enrollment)
- [x] Pushed to `hf/main`
## 10. Rule IDs
- **R-174:** Fernet encrypt API keys at rest
- **R-175:** Mask secrets in API/audit/logs (`β€’β€’β€’β€’` + last 4)
- **R-176:** Mutations require JWT + TOTP
- **R-177:** Audit every mutation
- **R-178:** SuperAdmin role only
- **R-179:** Redis cache TTL ≀ 300s; invalidate on write
- **R-180:** Pydantic schemas only on routes
## 11. File map
| File | Role |
|------|------|
| `app/core/crypto/credential_crypto.py` | Encrypt/decrypt/mask |
| `app/models/platform_api_credential.py` | ORM model |
| `app/repositories/platform_api_credential_repo.py` | DB access |
| `app/services/platform_api_registry.py` | Resolution + providers |
| `app/services/platform_api_service.py` | CRUD + audit |
| `app/superadmin/routers/platform_apis.py` | HTTP routes |
| `app/services/platform_pricing.py` | Consumes credential in `fetch_listing_price()` |