Spaces:
Running
Running
| # AI Studio | |
| AI Studio is MediaRouter's authenticated, provider-neutral AI workspace. The | |
| frontend routes are `/ai` and `/projects/{projectId}/ai`; both call only the | |
| MediaRouter BFF/API. Provider URLs and credentials never enter browser state. | |
| ## Implemented capability boundary | |
| `GET /v1/ai/capabilities` is authoritative. It derives ready tools and models | |
| from the existing generation provider/model registries. A tool is visible only | |
| when the frontend feature flag, user permission, API contract, provider, and | |
| model are all available. | |
| This phase exposes only operations backed by real adapters: | |
| - `generate_image`: FLUX text-to-image and optional image transformation. | |
| - `generate_video`: WAN image-to-video. | |
| Voice, music, transform, transcription, OCR, multimodal understanding, and | |
| structured text creation are not advertised because no corresponding | |
| production adapter exists. There is no mock provider or placeholder output. | |
| Each advertised tool includes its category, input/output media types, | |
| permission, project/asset support, available models, limits, and provider | |
| display metadata. The frontend registry adds presentation metadata only; it | |
| cannot make a backend operation available. | |
| ## API | |
| - `GET /v1/ai/capabilities` | |
| - `GET /v1/ai/jobs?offset=0&limit=25` | |
| - `POST /v1/ai/jobs` with `Idempotency-Key` | |
| - `GET /v1/ai/jobs/{generation_id}` | |
| - `POST /v1/ai/jobs/{generation_id}/cancel` | |
| Submission is a discriminated Pydantic union. Image and video parameters are | |
| closed schemas with bounded prompts, dimensions, duration, steps, guidance, | |
| seed, project ID, and source asset IDs. Unknown fields are rejected. | |
| The provider-neutral request is translated by `AiStudioService` into the | |
| existing typed generation request. The backend selects only a ready model of | |
| the requested modality. The browser cannot supply credentials, worker URLs, or | |
| unregistered provider payloads. | |
| ## Generation and job lifecycle | |
| AI Studio reuses `generation_requests`, `generation_jobs`, and the existing | |
| dispatcher: | |
| `queued -> processing -> completed | failed | cancelling -> cancelled` | |
| `retrying` is reported only when the durable generation state machine schedules | |
| a safe retry. Progress remains `null` because the current adapters do not | |
| provide trustworthy normalized progress. Completion is reported only when | |
| output validation and canonical asset registration have succeeded. | |
| The `product_surface = ai_studio` marker isolates AI history from legacy | |
| `/v1/generation` requests without creating a second job system. Requests retain | |
| one logical job through retries. Workspace-scoped idempotency rejects reuse of a | |
| key with a different operation, parameters, asset, project, or product surface. | |
| Cancellation is truthful: queued work can become cancelled immediately; | |
| running work remains cancelling until the provider confirms cancellation. | |
| ## Projects and assets | |
| Project context is optional globally and authoritative on the project route. | |
| The server verifies project membership and active status. Every source asset is | |
| resolved through `CanonicalAssetService`; it must belong to the authenticated | |
| workspace and, when project context is present, to that exact project. | |
| Outputs are downloaded and validated by the existing generation ingestor, | |
| registered as canonical `media_assets`, and associated with the selected | |
| project. Source assets are never overwritten. The frontend stores asset IDs, | |
| not binaries, worker paths, credentials, or durable signed URLs. Completed | |
| outputs can be opened in the asset library and their project can be opened in | |
| Content Studio. | |
| ## Authorization, limits, and audit | |
| The API uses existing authentication, tenant sessions, security policy, and | |
| rate limiter. Scopes are: | |
| - `ai:read` | |
| - `ai:generate` | |
| - `ai:transform` | |
| - `ai:analyze` | |
| - `ai:create` | |
| Only `ai:read` and `ai:generate` currently authorize an advertised operation. | |
| The remaining scopes reserve explicit boundaries for future real adapters. | |
| Rate limiting uses the existing job classification and workspace/user/API-key | |
| identity; no second limiter exists. | |
| Audit events are bounded and omit prompts and provider payloads: | |
| - `ai.generation_requested` | |
| - `ai.generation_completed` | |
| - `ai.generation_failed` | |
| - `ai.generation_cancelled` | |
| Structured logs contain generation/job IDs, operation/model, provider-neutral | |
| status/error category, and timing where available. Secrets, prompts, signed | |
| URLs, and full provider responses are excluded. | |
| Usage and cost fields are present as `null`. They must remain null until a | |
| provider supplies authoritative usage or approved pricing metadata. | |
| ## Frontend ownership and integration | |
| TanStack Query owns capabilities, jobs, history, and output refresh. | |
| Zustand owns only selected category/tool/project and temporary prompt/source | |
| input. Active jobs poll by state; completed output invalidates project asset | |
| queries. AI jobs are merged into `WorkspaceActivityDrawer`. | |
| Commands and global search entries are derived from discovered, available | |
| capabilities. Deep links use `/ai?tool={operation}`. Desktop uses tool | |
| navigation plus workspace/history; smaller layouts collapse to normal | |
| responsive grid/stack behavior. Cards and controls are keyboard accessible, | |
| focus visible, semantically labelled, and do not rely on color alone. | |
| ## Provider abstraction and failure behavior | |
| AI Studio delegates validation, submission, polling, cancellation, error | |
| normalization, output ingestion, and retry decisions to the existing generation | |
| adapters. Provider unavailable, timeout, rate limit, invalid request, rejection, | |
| network error, partial/invalid output, and cancellation retain truthful typed | |
| failure states. Provider response objects are not exposed. | |
| ## MCP, SDK, and n8n | |
| MCP registers `ai.capabilities`, `ai.generate`, `ai.list_jobs`, `ai.get_job`, | |
| and `ai.cancel_job` as thin authorized calls over `AiStudioService`. | |
| The TypeScript and Python SDKs expose capabilities, models, generate, list, | |
| get-generation, and cancel-generation through `client.ai`. | |
| The n8n package is intentionally unchanged. Its current generic operation node | |
| does not provide the capability-driven AI UX and explicit idempotency contract | |
| needed for safe generation. Adding a superficial static AI operation would | |
| violate the no-fake/no-duplicate architecture requirement. | |
| ## Migration order | |
| PostgreSQL migrations remain explicit and are never applied on production | |
| startup. Apply: | |
| 1. security migrations `0001` through `0004` | |
| 2. project migrations `0001_projects_foundation.sql` | |
| 3. `0002_project_resources.sql` | |
| 4. `0003_editor_persistence_rendering.sql` | |
| 5. `0004_ai_studio.sql` | |
| 6. social migrations in their documented order | |
| `0004_ai_studio.sql` is additive. It adds project context and product-surface | |
| classification to generation requests, ownership triggers, and query indexes. | |
| SQLite metadata creation remains limited to local/test configuration. | |
| ## Deferred runtime certification | |
| Provider credential tests, Docker/Linux runtime, Supabase PostgreSQL/RLS, | |
| FFmpeg, and Python 3.10 production-image certification are intentionally | |
| deferred to the final cross-product production certification phase. Production | |
| settings must not be weakened for Termux. | |