docs/ai-studio.md

AI Studio

AI Studio is MediaRouter's authenticated, provider-neutral AI workspace. The frontend routes are /ai and /projects/{projectId}/ai; both call only the MediaRouter BFF/API. Provider URLs and credentials never enter browser state.

Implemented capability boundary

GET /v1/ai/capabilities is authoritative. It derives ready tools and models from the existing generation provider/model registries. A tool is visible only when the frontend feature flag, user permission, API contract, provider, and model are all available.

This phase exposes only operations backed by real adapters:

• generate_image: FLUX text-to-image and optional image transformation.
• generate_video: WAN image-to-video.

Voice, music, transform, transcription, OCR, multimodal understanding, and structured text creation are not advertised because no corresponding production adapter exists. There is no mock provider or placeholder output.

Each advertised tool includes its category, input/output media types, permission, project/asset support, available models, limits, and provider display metadata. The frontend registry adds presentation metadata only; it cannot make a backend operation available.

API

• GET /v1/ai/capabilities
• GET /v1/ai/jobs?offset=0&limit=25
• POST /v1/ai/jobs with Idempotency-Key
• GET /v1/ai/jobs/{generation_id}
• POST /v1/ai/jobs/{generation_id}/cancel

Submission is a discriminated Pydantic union. Image and video parameters are closed schemas with bounded prompts, dimensions, duration, steps, guidance, seed, project ID, and source asset IDs. Unknown fields are rejected.

The provider-neutral request is translated by AiStudioService into the existing typed generation request. The backend selects only a ready model of the requested modality. The browser cannot supply credentials, worker URLs, or unregistered provider payloads.

Generation and job lifecycle

AI Studio reuses generation_requests, generation_jobs, and the existing dispatcher:

queued -> processing -> completed | failed | cancelling -> cancelled

retrying is reported only when the durable generation state machine schedules a safe retry. Progress remains null because the current adapters do not provide trustworthy normalized progress. Completion is reported only when output validation and canonical asset registration have succeeded.

The product_surface = ai_studio marker isolates AI history from legacy /v1/generation requests without creating a second job system. Requests retain one logical job through retries. Workspace-scoped idempotency rejects reuse of a key with a different operation, parameters, asset, project, or product surface.

Cancellation is truthful: queued work can become cancelled immediately; running work remains cancelling until the provider confirms cancellation.

Projects and assets

Project context is optional globally and authoritative on the project route. The server verifies project membership and active status. Every source asset is resolved through CanonicalAssetService; it must belong to the authenticated workspace and, when project context is present, to that exact project.

Outputs are downloaded and validated by the existing generation ingestor, registered as canonical media_assets, and associated with the selected project. Source assets are never overwritten. The frontend stores asset IDs, not binaries, worker paths, credentials, or durable signed URLs. Completed outputs can be opened in the asset library and their project can be opened in Content Studio.

Authorization, limits, and audit

The API uses existing authentication, tenant sessions, security policy, and rate limiter. Scopes are:

• ai:read
• ai:generate
• ai:transform
• ai:analyze
• ai:create

Only ai:read and ai:generate currently authorize an advertised operation. The remaining scopes reserve explicit boundaries for future real adapters. Rate limiting uses the existing job classification and workspace/user/API-key identity; no second limiter exists.

Audit events are bounded and omit prompts and provider payloads:

• ai.generation_requested
• ai.generation_completed
• ai.generation_failed
• ai.generation_cancelled

Structured logs contain generation/job IDs, operation/model, provider-neutral status/error category, and timing where available. Secrets, prompts, signed URLs, and full provider responses are excluded.

Usage and cost fields are present as null. They must remain null until a provider supplies authoritative usage or approved pricing metadata.

Frontend ownership and integration

TanStack Query owns capabilities, jobs, history, and output refresh. Zustand owns only selected category/tool/project and temporary prompt/source input. Active jobs poll by state; completed output invalidates project asset queries. AI jobs are merged into WorkspaceActivityDrawer.

Commands and global search entries are derived from discovered, available capabilities. Deep links use /ai?tool={operation}. Desktop uses tool navigation plus workspace/history; smaller layouts collapse to normal responsive grid/stack behavior. Cards and controls are keyboard accessible, focus visible, semantically labelled, and do not rely on color alone.

Provider abstraction and failure behavior

AI Studio delegates validation, submission, polling, cancellation, error normalization, output ingestion, and retry decisions to the existing generation adapters. Provider unavailable, timeout, rate limit, invalid request, rejection, network error, partial/invalid output, and cancellation retain truthful typed failure states. Provider response objects are not exposed.

MCP, SDK, and n8n

MCP registers ai.capabilities, ai.generate, ai.list_jobs, ai.get_job, and ai.cancel_job as thin authorized calls over AiStudioService.

The TypeScript and Python SDKs expose capabilities, models, generate, list, get-generation, and cancel-generation through client.ai.

The n8n package is intentionally unchanged. Its current generic operation node does not provide the capability-driven AI UX and explicit idempotency contract needed for safe generation. Adding a superficial static AI operation would violate the no-fake/no-duplicate architecture requirement.

Migration order

PostgreSQL migrations remain explicit and are never applied on production startup. Apply:

1. security migrations 0001 through 0004
2. project migrations 0001_projects_foundation.sql
3. 0002_project_resources.sql
4. 0003_editor_persistence_rendering.sql
5. 0004_ai_studio.sql
6. social migrations in their documented order

0004_ai_studio.sql is additive. It adds project context and product-surface classification to generation requests, ownership triggers, and query indexes. SQLite metadata creation remains limited to local/test configuration.

Deferred runtime certification

Provider credential tests, Docker/Linux runtime, Supabase PostgreSQL/RLS, FFmpeg, and Python 3.10 production-image certification are intentionally deferred to the final cross-product production certification phase. Production settings must not be weakened for Termux.