Arag / .agent /PLATFORM_API_RULES.md
AuthorBot
Document feature rules for platform APIs, TOTP, and pricing.
34e47a7
|
Raw
History Blame Contribute Delete
6.76 kB

Feature Rules β€” Platform API Credentials (SuperAdmin)

Status: Implemented (ad1b7c6). Source of truth for retailer API key management.

1. Problem & scope

  • User story: As platform operator, I enter optional paid retailer API keys in SuperAdmin so price/presence scans use reliable APIs instead of blocked free scraping β€” without exposing keys in env files or logs.
  • In scope:
    • SuperAdmin UI to create/update/disable/delete per-platform credentials
    • Fernet encryption at rest in PostgreSQL
    • Provider registry (Rainforest, ScrapingBee, Google Books)
    • Priority resolution: SuperAdmin DB β†’ env fallback β†’ free logic
    • Integration with fetch_listing_price() during platform presence scans
    • Audit log on every mutation
    • Masked read API (never return raw keys)
  • Out of scope:
    • Author-facing credential management (SuperAdmin only)
    • Billing/metering of third-party API usage
    • Presence scanning via paid APIs (future β€” pricing only today)
    • Goodreads / Open Library paid APIs (none exist)

2. Architecture decisions

Decision Choice Why
Storage platform_api_credentials table Auditable, encrypted, UI-manageable
Encryption Fernet derived from SECRET_KEY R-174; no plaintext in DB
Resolution platform_api_registry.resolve_platform_credential() Single place for priority chain
Providers Strategy in registry (fetch_price_via_credential) Open/closed β€” add provider without changing routes
Cache Redis platform_api_cred:v1:{platform_id}, TTL 300s Performance; invalidate on write (R-179)
Routes Thin router β†’ PlatformApiService SRP; no logic in handlers
Env fallback Documented in PLATFORM_API_META Bootstrap only β€” not primary UX

Principles:

  • Single responsibility β€” PlatformApiService CRUD; registry resolves/fetches
  • Open/closed β€” new provider = new branch in registry, not new endpoint
  • Dependency inversion β€” pricing calls registry, not HTTP directly from presence
  • Thin client β€” UI posts key once; server encrypts, masks, audits
  • Fail secure β€” no key β†’ fall through; never log raw secrets

3. Data & security

  • Tables/columns: platform_api_credentials (see migration 003_platform_api_credentials)
  • Encryption at rest: yes β€” credential_crypto.encrypt_secret() / Fernet
  • Who can read/write: role=superadmin only; mutations require JWT + TOTP (X-TOTP-Code)
  • Audit log actions: upsert_platform_api, disable_platform_api, delete_platform_api
  • Never log: raw api_key, api_secret, decrypted values; audit uses mask_secret() only

4. API contract

Method Path Auth Request schema Response
GET /api/super/platform-apis JWT + TOTP β€” list[PlatformApiCredentialView]
PUT /api/super/platform-apis/{platform_id} JWT + TOTP PlatformApiCredentialUpsert masked view
POST /api/super/platform-apis/{platform_id}/disable JWT + TOTP β€” masked view
DELETE /api/super/platform-apis/{platform_id} JWT + TOTP β€” masked view
  • Pydantic only (PlatformApiCredentialUpsert) β€” R-180
  • platform_id in path must match body on PUT

5. UI

  • Location: SuperAdmin β†’ Platform Settings β†’ Retailer API Keys
  • Complete UX path:
    1. Login (JWT)
    2. Complete TOTP enrollment if not configured (see SUPERADMIN_TOTP_RULES.md)
    3. Enter 6-digit TOTP in settings header
    4. Per platform: select provider β†’ paste API key β†’ Save / Disable / Remove
    5. Env-sourced keys show read-only env badge
  • Error states: missing TOTP β†’ toast; invalid provider β†’ 400; unknown platform β†’ 400

6. Priority / fallback chains

For each platform_id on price fetch:

  1. SuperAdmin DB row (is_enabled=true) β†’ fetch_price_via_credential()
  2. Environment variable (see table below)
  3. Built-in free logic (platform_pricing.py)

If step 1 fails (HTTP/parse), fall through to 2 then 3.

platform_id SuperAdmin providers Env fallback Free fallback
amazon rainforest, scrapingbee RAINFOREST_API_KEY US-locale HTML
google_books google_books GOOGLE_BOOKS_API_KEY Anonymous Books API
barnes_noble scrapingbee β€” HTML patterns
apple_books scrapingbee β€” iTunes Lookup API
kobo scrapingbee β€” HTML (often blocked)
thriftbooks scrapingbee β€” HTML (often blocked)
abebooks scrapingbee β€” HTML patterns
bookshop scrapingbee β€” HTML (often blocked)
goodreads β€” β€” from-price rollup
open_library β€” β€” catalog "Free"

Never instruct operators to use HF Secrets or .env as the primary key entry path. Env is emergency/bootstrap only.

7. Provider contracts

google_books

  • GET https://www.googleapis.com/books/v1/volumes/{id}?key=...&country=US
  • ISBN fallback via q=isbn:{isbn13}

rainforest (Amazon only)

  • GET https://api.rainforestapi.com/request?api_key=...&type=product&amazon_domain=amazon.com&asin={asin}

scrapingbee

  • GET https://app.scrapingbee.com/api/v1?api_key=...&url={listing_url}&premium_proxy=true&country_code=us
  • Parse via extract_price_from_html()

8. Tests

  • Unit β€” tests/unit/test_platform_api_registry.py (priority, crypto, parsers)
  • Unit β€” tests/unit/test_platform_api_service.py (upsert, audit, delete)
  • Integration β€” SuperAdmin upsert happy path (mocked DB)
  • BDD β€” see .agent/BDD_SPECS.md β†’ Platform API Credentials

9. Definition of done

  • Rules doc approved
  • Migration 003_platform_api_credentials
  • FILE_MAP.md updated
  • End-to-end UI (requires TOTP enrollment)
  • Pushed to hf/main

10. Rule IDs

  • R-174: Fernet encrypt API keys at rest
  • R-175: Mask secrets in API/audit/logs (β€’β€’β€’β€’ + last 4)
  • R-176: Mutations require JWT + TOTP
  • R-177: Audit every mutation
  • R-178: SuperAdmin role only
  • R-179: Redis cache TTL ≀ 300s; invalidate on write
  • R-180: Pydantic schemas only on routes

11. File map

File Role
app/core/crypto/credential_crypto.py Encrypt/decrypt/mask
app/models/platform_api_credential.py ORM model
app/repositories/platform_api_credential_repo.py DB access
app/services/platform_api_registry.py Resolution + providers
app/services/platform_api_service.py CRUD + audit
app/superadmin/routers/platform_apis.py HTTP routes
app/services/platform_pricing.py Consumes credential in fetch_listing_price()