Hugging Face's logo

Spaces:
firebird-technologies
/
blog2video-test
Running on CPU Upgrade

App Files Community
blog2video-test / progress.md
faisalnazeer67's picture
faisalnazeer67
chore: update fixes
92e8f98
preview code
|
Raw
History Blame Contribute Delete
61.7 kB

AI Custom Templates — Progress Tracker

Master Plan

Replace the fixed UniversalScene.tsx renderer (9 layouts, 10 element types) with AI-generated unique React components per brand. Each template becomes a code string stored in DB, compiled at runtime for preview, written to temp files for rendering.

No fallback to old system. UniversalScene.tsx, styleEngine.ts, and the old custom template types will be deleted as a final cleanup step after all phases are implemented, tested, and stable. No dual-path code long-term — the new system must stand on its own so bugs are visible, not hidden behind a fallback.

Full plan: /Users/faisalnazir/Desktop/Blog2Video - AI Generated Custom Templates Plan.md

Phases Overview

Phase Description Status
1 DB Models + Enhanced Scraping DONE
2 AI Code Generation + Preview DONE
3 Chat-Based Editing DEFERRED — removed from codebase, will revisit in a future phase
4 Render Pipeline Update DONE
5 Polish & Preview System DONE
6 Cleanup — Remove old UniversalScene/styleEngine/fallback code DONE
H1 Rate Limiting on Claude API Endpoints DONE
H2 Code Validator Hardening DONE
H3 Template Cache TTL DONE
H4 Render Workspace Locking DONE
H5 Frontend Code Sandbox (CSP + Error Boundary) DONE
H6 Self-healing current_version_id DONE
F1 BrandKit Population During Theme Extraction DONE
F2 Brand Asset Uploads (Logo) DONE
F5 Version Snapshots on Chat Edits DEFERRED — depends on Phase 3
F6 custom_prompt_builder.py Cleanup DONE
F3 Enhanced Scraping with Firecrawl Branding Format DONE
F4 Preview Thumbnails for Template Gallery DONE
F7 Multiple Unique Content Scene Variants DONE
OPT Performance & Quality Optimization (5 fixes) DONE
V2 Content-Aware Architecture (feat/customv2) DONE

Phase 1: DB Models + Enhanced Scraping — DONE

What Was Done

New file created:

  • backend/app/models/brand_kit.py — BrandKit model (brand_kits table)
    • Fields: id, user_id, source_url, brand_name, colors (JSON), fonts (JSON), design_language (JSON), logos (JSON), images (JSON), timestamps
    • Relationships: user, custom_templates

Files modified:

  • backend/app/models/custom_template.py — Added:

    • component_code (Text, nullable) — will hold AI-generated React code
    • intro_code (Text, nullable) — intro scene variant
    • outro_code (Text, nullable) — outro scene variant
    • brand_kit_id (Integer FK → brand_kits.id, nullable, indexed)
    • brand_kit relationship

  • backend/app/models/scene.py — Added:

    • scene_type (String(20), nullable) — "intro"/"content"/"outro"; NULL = "content"

  • backend/app/models/user.py — Added:

    • brand_kits relationship (cascade delete)

  • backend/app/models/__init__.py — Added BrandKit import and export

  • backend/app/database.py — Updated:

    • _migrate_sqlite(): added component_code, intro_code, outro_code, brand_kit_id to ct_migrations; added scene_type to scene_migrations
    • init_db(): added BrandKit to model imports

  • backend/alembic/env.py — Added BrandKit import

  • backend/alembic/versions/phase1_brand_kit_and_schema_fields.py — New Alembic migration:

    • Creates brand_kits table
    • Adds 4 columns to custom_templates
    • Adds scene_type to scenes
    • Down revision: 7ed594338d45

  • backend/app/services/theme_scraper.py — Enhanced:

    • ScrapedThemeData now has logo_urls, og_image, screenshot_url fields
    • New _extract_logo_urls() helper extracts logos from HTML (favicon links + img tags with "logo" in attributes)
    • Tries Firecrawl screenshot format with graceful fallback
    • Extracts OG image from metadata

Backward Compatibility

  • All new columns nullable — existing data unaffected
  • No API endpoint changes — responses identical
  • No frontend changes
  • ScrapedThemeData new fields are additive — ThemeExtractor ignores them

Verification Steps

  1. Start backend: cd backend && uvicorn app.main:app --reload --port 8000
  2. Confirm no startup errors
  3. Check brand_kits table exists in SQLite
  4. Check new columns on custom_templates and scenes
  5. Test existing custom template flow (paste URL → extract → save)

Phase 2: AI Code Generation + Preview — DONE

What Was Done

New files created:

  • backend/app/services/code_validator.py — Validates generated code (blocklist for dangerous APIs, SceneComponent declaration check)
  • backend/app/services/code_generator.py — Claude API-based code generation (claude-sonnet-4-5-20250929). Generates 3 variants (intro/content/outro) in one call using XML delimiters. Retry logic on validation failure.
  • frontend/src/utils/compileComponent.ts — @babel/standalone JIT compiler. Lazy-loads Babel (~3MB), transpiles JSX, creates component via Function() factory with injected Remotion APIs.
  • frontend/src/components/RemotionPreviewPlayer.tsx — Remotion Player wrapper. Compiles code string, renders live preview with sample props. Error state with retry button.

Files modified:

  • backend/app/routers/custom_templates.py — Added:

    • POST /{template_id}/generate-code endpoint (calls code_generator, validates, stores in DB)
    • GET /{template_id}/code endpoint (lightweight code-only response)
    • _serialize_template() now includes component_code, intro_code, outro_code
    • CustomTemplateOut schema updated with code fields

  • frontend/src/components/templatePreviews/CustomPreview.tsx — Added:

    • Optional componentCode prop
    • When componentCode exists, renders RemotionPreviewPlayer (lazy-loaded) instead of carousel
    • Existing carousel unchanged as fallback for templates without generated code

  • frontend/src/api/client.ts — Added:

    • component_code, intro_code, outro_code fields to CustomTemplateItem
    • generateTemplateCode(templateId) API method
    • getTemplateCode(templateId) API method

  • frontend/src/components/CustomTemplateCreator.tsx — Added:

    • Step 3: "Generating Template" — shows spinner during code generation, then live Remotion preview
    • After saving template (Step 2), automatically triggers code generation
    • Error handling with retry button
    • "Done" button to close modal

  • frontend/src/pages/CustomTemplates.tsx — Updated:

    • Template grid passes componentCode to CustomPreview for live Remotion previews

  • frontend/package.json — Added:

    • @babel/standalone dependency
    • @types/babel__standalone dev dependency

Architecture Decisions

  • Claude API for code gen (not DSPy) — raw code output doesn't fit DSPy's structured fields
  • Separate endpoint — code gen is slow (10-30s), doesn't block template creation
  • One LLM call for 3 variants — intro/content/outro in one call for visual coherence
  • Lazy-loaded Babel — ~3MB chunk only loaded when viewing templates with generated code
  • No fallback — compilation errors surface inline with retry option

Verification Steps

  1. Start backend: cd backend && uvicorn app.main:app --reload --port 8000
  2. Create a custom template via existing flow (paste URL → extract → save)
  3. Step 3 should auto-trigger code generation with spinner
  4. After generation, live Remotion preview should play in the modal
  5. On Custom Templates page, templates with generated code show live previews
  6. npm run build passes ✓

Phase 3: Chat-Based Editing — DONE

What Was Done

New files created:

  • backend/app/models/code_edit_history.py — CodeEditHistory model (code_edit_history table)

    • Fields: id, template_id (FK, CASCADE), code_type ("component"/"intro"/"outro"), previous_code, instruction, summary, created_at
    • Composite index on (template_id, code_type)
    • Relationship: template (back to CustomTemplate)

  • backend/alembic/versions/phase3_code_edit_history.py — Alembic migration

    • Creates code_edit_history table
    • Down revision: phase1_brand_kit

  • frontend/src/components/TemplateCodeEditor.tsx — Full-screen code editor overlay

    • Two-panel layout: Remotion preview (left) + chat panel (right)
    • Variant tabs (Intro/Content/Outro) to switch which code is previewed/edited
    • Chat messages reconstructed from edit history on mount
    • Typing indicator dots during Claude processing
    • Undo button per variant (reverts to previous code version)
    • Suggestion chips for common edit requests
    • Error display for failed edits

Files modified:

  • backend/app/services/code_generator.py — Added:

    • EDIT_SYSTEM_PROMPT — system prompt for code editing (modify existing code, keep everything else intact)
    • _parse_edit_response() — parses <CODE> and <SUMMARY> XML tags from response
    • edit_component_code(existing_code, instruction) — calls Claude to edit code, validates, retries on failure. Returns { code, summary }

  • backend/app/routers/custom_templates.py — Added:

    • EditCodeRequest / UndoEditRequest Pydantic schemas
    • VARIANT_FIELD_MAP helper dict
    • POST /{template_id}/edit-code — sends existing code + instruction to Claude, saves edit history, updates template code
    • POST /{template_id}/undo-edit — reverts to previous code version from history, deletes history row
    • GET /{template_id}/edit-history — returns list of edits (instruction + summary, no code) for chat reconstruction

  • backend/app/models/custom_template.py — Added:

    • edit_history relationship (cascade delete-orphan)

  • backend/app/models/__init__.py — Added CodeEditHistory import and export

  • backend/app/database.py — Added CodeEditHistory to init_db imports (SQLite auto-creates via create_all)

  • backend/alembic/env.py — Added CodeEditHistory import

  • frontend/src/api/client.ts — Added:

    • CodeVariant type ("component" | "intro" | "outro")
    • CodeEditHistoryItem interface
    • editTemplateCode(templateId, instruction, variant) API method
    • undoTemplateEdit(templateId, variant) API method
    • getTemplateEditHistory(templateId) API method

  • frontend/src/pages/CustomTemplates.tsx — Added:

    • codeEditTarget state for full-screen code editor
    • "Customize" button on template cards (only shown when component_code exists, purple filled)
    • TemplateCodeEditor rendering when codeEditTarget is set
    • preloadBabel() call on page mount
    • handleCodeEditorClose() to update template in grid after editing

Architecture Decisions

  • Separate edit function — edit_component_code() uses a different system prompt than generation (focused on modifying existing code, not creating from scratch)
  • Per-variant editing — user selects which variant (Intro/Content/Outro) to edit; edits apply to one variant at a time (saves cost, more predictable)
  • Edit history table — follows SceneEditHistory pattern; stores previous_code for multi-level undo
  • Chat reconstruction — edit history reconstructed into chat messages on mount (no separate chat persistence)
  • No new npm dependencies — reuses existing @babel/standalone, @remotion/player, RemotionPreviewPlayer
  • Separate from CustomTemplateEditor — existing editor kept for name/style changes; new TemplateCodeEditor is full-screen for code editing

Verification Steps

  1. Start backend: cd backend && uvicorn app.main:app --reload --port 8000 — no errors, code_edit_history table created
  2. Create a template with generated code (Phase 2 flow)
  3. On Custom Templates page, "Customize" button appears on templates with code
  4. Click "Customize" → full-screen editor opens with preview + chat
  5. Type "make the title bigger" → Claude edits code → preview updates live
  6. Click "Undo" → code reverts to previous version → preview updates
  7. Close editor → template card on grid shows updated preview
  8. npm run build passes ✓

Phase 4: Render Pipeline Update — DONE

What Was Done

New files created:

  • remotion-video/src/templates/generated/types.ts — TypeScript types for generated video data (GeneratedSceneProps, GeneratedVideoData, GeneratedSceneData)
  • remotion-video/src/templates/generated/SceneIntro.tsx — Placeholder intro scene component (overwritten with AI code at render time)
  • remotion-video/src/templates/generated/SceneContent.tsx — Placeholder content scene component (overwritten with AI code at render time)
  • remotion-video/src/templates/generated/SceneOutro.tsx — Placeholder outro scene component (overwritten with AI code at render time)
  • remotion-video/src/templates/generated/GeneratedVideo.tsx — New Remotion composition:
    • Reads data.json (including brandColors and sceneType per scene)
    • Statically imports SceneIntro, SceneContent, SceneOutro components
    • Maps each scene to the right component based on sceneType (explicit from DB, or inferred: first=intro, last=outro, middle=content)
    • Handles font loading via delayRender/continueRender
    • Sequences scenes with Audio and LogoOverlay (same pattern as CustomVideo)
    • calculateGeneratedMetadata for dynamic duration/dimensions

Files modified:

  • remotion-video/src/Root.tsx — Added:

    • Import of GeneratedVideo + calculateGeneratedMetadata
    • New <Composition id="GeneratedVideo" .../> registration

  • backend/app/services/template_service.py — Updated:

    • _load_custom_template_data() now includes has_generated_code, component_code, intro_code, outro_code in cached result
    • _get_custom_meta() passes has_generated_code to build_custom_meta

  • backend/app/services/custom_prompt_builder.py — Updated:

    • build_custom_meta() accepts has_generated_code parameter
    • Returns composition_id: "GeneratedVideo" when template has AI code, "CustomVideo" otherwise

  • backend/app/services/remotion.py — Updated:

    • provision_workspace() calls _write_generated_scene_files() for custom templates with AI code
    • _write_generated_scene_files() (NEW) — loads component code from DB, wraps in proper .tsx module with imports, writes to workspace src/templates/generated/ directory
    • _wrap_generated_code() (NEW) — wraps raw AI-generated code (const SceneComponent = ...) in a proper ESM module with Remotion imports + default export
    • write_remotion_data() — adds brandColors and sceneType per scene to data.json when template has generated code

Architecture Decisions

  • Static file overwrite — generated code is written as actual .tsx files in the workspace, overwriting placeholders. Vite bundles them normally — no runtime compilation, no @babel/standalone needed server-side.
  • Placeholder pattern — the repo contains stub scene components so Root.tsx always compiles. At render time, these stubs are replaced with the real AI-generated code.
  • composition_id routing — GeneratedVideo vs CustomVideo is selected via build_custom_meta based on whether the template has component_code. Old templates without generated code continue to use UniversalScene.
  • brandColors in data.json — derived from theme colors, passed to GeneratedSceneProps. Generated components use props.brandColors directly.
  • sceneType tagging — each scene gets sceneType ("intro"/"content"/"outro") from the DB scene_type field, falling back to positional inference (first=intro, last=outro).
  • Code wrapping — raw AI code (const SceneComponent = (props) => { ... }) is wrapped with Remotion imports and export default SceneComponent. This makes the generated files proper ES modules that Vite can bundle.

Verification Steps

  1. Start backend: cd backend && uvicorn app.main:app --reload --port 8000 — no errors
  2. Create a template with generated code (Phase 2 flow)
  3. Trigger a video render for a project using the generated template
  4. Check logs: should see [REMOTION] GeneratedVideo: brandColors and sceneTypes set for N scenes
  5. Check workspace: src/templates/generated/SceneIntro.tsx etc. should contain the AI-generated code wrapped in proper imports
  6. Render should produce an MP4 using the AI-generated components
  7. Templates without generated code should still render via CustomVideo + UniversalScene
  8. TypeScript check passes: cd remotion-video && npx tsc --noEmit ✓

Phase 5: Polish & Preview System — DONE

What Was Done

New file created:

  • backend/app/models/template_version.py — TemplateVersion model (template_versions table)

    • Fields: id, template_id (FK, CASCADE), component_code, intro_code, outro_code, label, created_at
    • Relationship: template (back to CustomTemplate)

  • backend/alembic/versions/phase5_template_versions.py — Alembic migration

    • Creates template_versions table
    • Adds current_version_id column to custom_templates
    • Down revision: phase3_code_edit_history

Files modified:

  • backend/app/models/custom_template.py — Added:

    • current_version_id (Integer, nullable) — pointer to active version
    • versions relationship (cascade delete-orphan, ordered by created_at desc)

  • backend/app/models/__init__.py — Added TemplateVersion import and export

  • backend/app/database.py — Added:

    • TemplateVersion to init_db imports
    • current_version_id to SQLite migrations for custom_templates

  • backend/alembic/env.py — Added TemplateVersion import

  • backend/app/routers/custom_templates.py — Added:

    • _save_version() helper — snapshots code fields into a new TemplateVersion row
    • Updated generate_code() — saves "Initial generation" version after first code gen
    • POST /{template_id}/regenerate-code — regenerates all code variants from scratch; snapshots current code before overwriting, saves new "Regenerated" version
    • GET /{template_id}/versions — lists all saved versions (newest first) with current_version_id
    • POST /{template_id}/versions/{version_id}/rollback — restores code from a previous version; snapshots current state as "Before rollback" first
    • CustomTemplateOut and _serialize_template() now include current_version_id

  • frontend/src/api/client.ts — Added:

    • current_version_id to CustomTemplateItem interface
    • TemplateVersionItem and TemplateVersionsResponse interfaces
    • regenerateTemplateCode(templateId) API method
    • getTemplateVersions(templateId) API method
    • rollbackTemplateVersion(templateId, versionId) API method

  • frontend/src/pages/CustomTemplates.tsx — Added:

    • regeneratingId state for tracking which template is regenerating
    • handleRegenerate() function — calls regenerate API, updates template in grid
    • Regenerate icon button on each template card (next to Customize), with spinning SVG animation during regeneration

  • frontend/src/components/TemplateCodeEditor.tsx — Added:

    • Version history state: versions, currentVersionId, showVersions, isRegenerating, isRollingBack
    • loadVersions() — fetches version list from API on mount
    • handleRegenerate() — regenerates code from within editor, clears chat, reloads versions
    • handleRollback(versionId) — restores a previous version, reloads version list
    • Regenerate button in preview panel header (purple outline, with spinner)
    • "Versions (N)" toggle button in chat header — opens collapsible version history panel
    • Version history panel: lists all versions with labels, timestamps, "(current)" badge, and "Restore" button per version

Architecture Decisions

  • Version snapshots — each version stores a full copy of all 3 code variants (intro/component/outro). Not incremental diffs — simpler, no reconstruction needed.
  • Automatic versioning — versions are created automatically on generate, regenerate, and rollback. No manual "save version" step needed.
  • Pre-rollback snapshots — before a rollback overwrites code, the current state is saved as a "Before rollback" version, so users can always undo a rollback.
  • current_version_id — pointer on the template to its active version. Used by frontend to highlight "(current)" in the version list. Not a FK constraint (flexible).
  • No preview thumbnails yet — this phase focused on regeneration + versioning. Server-side mini-renders for gallery thumbnails can be added later as an optimization.

Verification Steps

  1. Start backend: cd backend && uvicorn app.main:app --reload --port 8000 — no errors, template_versions table created
  2. Create a template with generated code (Phase 2 flow) — a version is saved automatically
  3. On Custom Templates page, hover a template card → Regenerate button (refresh icon) appears next to Customize
  4. Click Regenerate → spinner shows → new code generated → preview updates → old version preserved
  5. Click Customize → full-screen editor opens → "Versions (N)" button in chat header
  6. Click "Versions (N)" → version history panel opens with all versions listed
  7. Click "Restore" on an older version → code rolls back → preview updates
  8. TypeScript check passes (no new errors) ✓

Phase 6: Cleanup — Remove Old UniversalScene/StyleEngine/Fallback Code — DONE

What Was Done

Files deleted:

  • remotion-video/src/templates/custom/ — entire directory removed:

    • UniversalScene.tsx (36KB) — old fixed layout engine (9 arrangements, 10 element types)
    • CustomVideo.tsx (10KB) — old Remotion composition using UniversalScene
    • types.ts (3KB) — old CustomTheme, SceneLayoutConfig types
    • utils/styleEngine.ts (11KB) — theme style → CSS decoration mapper
    • utils/normalize.ts (7KB) — layout config normalization helpers

  • frontend/src/components/remotion/custom/ — entire directory removed:

    • UniversalScene.tsx — frontend copy of server UniversalScene
    • CustomVideoComposition.tsx — frontend copy of CustomVideo
    • types.ts — frontend copy of custom types
    • utils/ — frontend copy of utils

Files modified:

  • remotion-video/src/Root.tsx — Removed:

    • CustomVideo + calculateCustomMetadata import
    • <Composition id="CustomVideo" .../> registration
    • Only GeneratedVideo composition remains for custom templates

  • backend/app/services/remotion.py — Updated:

    • Removed has_generated_code conditional in write_remotion_data() — brandColors and sceneType are now always set for custom templates
    • Replaced print() calls with logger.info()/logger.warning()

  • backend/app/services/custom_prompt_builder.py — Updated:

    • Removed has_generated_code parameter from build_custom_meta()
    • composition_id is now always "GeneratedVideo" — no fallback to "CustomVideo"

  • backend/app/services/template_service.py — Updated:

    • Removed has_generated_code kwarg from build_custom_meta() call

  • frontend/src/components/remotion/remotionAdapters.tsx — Removed:

    • UniversalScene import from @remotion-video/templates/custom/UniversalScene
    • CustomTheme, SceneLayoutConfig type imports from @remotion-video/templates/custom/types
    • RemotionCustomSceneInput interface
    • RemotionCustomVideoCompositionProps interface
    • RemotionCustomVideoComposition component (entire ~100 line composition)

  • frontend/src/components/remotion/templateConfig.tsx — Removed:

    • CustomVideoComposition import
    • CUSTOM_ARRANGEMENTS set
    • custom entry from TEMPLATE_REGISTRY
    • custom_ routing in getTemplateConfig() (no longer routes to old system)
    • RemotionCustomVideoComposition from the remotion source override chain

  • frontend/src/types/remotion-video.d.ts — Removed:

    • @remotion-video/templates/custom/UniversalScene module declaration
    • @remotion-video/templates/custom/types module declaration

  • frontend/src/components/templatePreviews/CustomPreview.tsx — Rewritten:

    • Removed 3-slide carousel fallback (SlideHero, SlideContent, SlideTypo, SlideDots, ScaledCanvas)
    • Removed all old style helpers (getStyleBg, getCardStyle, getDecorations)
    • Now only renders RemotionPreviewPlayer when componentCode exists
    • Shows simple placeholder when componentCode is missing

  • frontend/src/components/SceneEditModal.tsx — Updated:

    • Changed comment from "UniversalScene handles sizing" to "Custom template arrangements"

Architecture Changes

  • Single rendering path — All custom templates now use GeneratedVideo composition exclusively. No fallback to CustomVideo + UniversalScene.
  • ~107KB of dead code removed — UniversalScene (36KB), styleEngine (11KB), normalize (7KB), CustomVideo (10KB × 2 copies), types (3KB × 2 copies), carousel preview (~400 lines).
  • No dual-path conditionals — build_custom_meta() always returns composition_id: "GeneratedVideo". write_remotion_data() always writes brandColors + sceneType for custom templates.

Verification Steps

  1. cd remotion-video && npx tsc --noEmit — passes ✓
  2. cd frontend && npx tsc --noEmit — passes (only pre-existing @babel/standalone type issue from missing npm install) ✓
  3. Backend imports: python -c "from app.services.remotion import provision_workspace; ..." — passes ✓
  4. Custom templates page shows live Remotion previews (no carousel fallback)
  5. Video rendering for custom templates uses GeneratedVideo composition exclusively

Phase H1: Rate Limiting on Claude API Endpoints — DONE

What Was Done

File modified:

  • backend/app/routers/custom_templates.py — Added:
    • _ai_call_counts in-memory dict: user_id → (date_string, count)
    • AI_DAILY_LIMIT = 5 constant
    • _check_ai_rate_limit(user_id) helper — checks daily count, resets on new day, raises HTTP 429 if exceeded
    • Rate limit check added at top of generate_code(), regenerate_code(), and edit_code() endpoints

Design Decisions

  • In-memory counter — matches existing _render_progress pattern. Single Cloud Run instance, so in-memory is fine.
  • Combined limit — 5 calls/day total across generate + regenerate + edit (not 5 per endpoint).
  • Daily reset — counter resets on date change, no cron needed.
  • Future extensibility — AI_DAILY_LIMIT can be made per-plan (Free: 5, Standard: 20, Pro: 50).

Phase H2: Code Validator Hardening — DONE

What Was Done

File modified:

  • backend/app/services/code_validator.py — Enhanced:
    • Added DANGEROUS_REGEX list — 18 compiled regex patterns with word-boundary matching for eval, Function, fetch, document, window, process, globalThis, require, import, __proto__, constructor[], Proxy, Reflect, XMLHttpRequest, WebSocket, localStorage, sessionStorage, cookie
    • Added MAX_CODE_LENGTH = 15,000 — rejects code exceeding this limit
    • Added MAX_NESTING_DEPTH = 20 — rejects excessively nested brace structures
    • Added balanced braces check — ensures { and } counts match
    • Two-pass validation: fast substring check first, then thorough regex pass

Design Decisions

  • Regex word boundaries — \beval\b catches eval but not evaluation, and is harder to bypass with string concatenation tricks
  • New blocked APIs — Proxy, Reflect (metaprogramming), WebSocket (network), localStorage/sessionStorage/cookie (storage)
  • Structural checks — max length prevents abuse via massive code payloads; nesting depth catches obfuscated code; balanced braces catch malformed output

Phase H3: Template Cache TTL — DONE

What Was Done

Files modified:

  • backend/app/services/template_service.py — Updated:

    • Cache type changed from dict[str, data] to dict[str, (data, timestamp)]
    • Added _CACHE_TTL = 30.0 seconds constant
    • _load_custom_template_data() now checks time.monotonic() age on cache hit — refetches from DB if expired
    • Explicit invalidation (invalidate_custom_template_cache()) still works for same-process freshness

  • backend/app/services/remotion.py — Updated:

    • provision_workspace() calls invalidate_custom_template_cache(template_id) before writing scene files, ensuring fresh DB data

Design Decisions

  • 30s TTL — short enough for multi-worker consistency, long enough to avoid hammering DB during a single request lifecycle
  • time.monotonic() — immune to system clock changes
  • Explicit invalidation preserved — same-process updates still get instant cache busting

Phase H4: Render Workspace Locking — DONE

What Was Done

File modified:

  • backend/app/services/remotion.py — Added:
    • _workspace_locks: dict[int, threading.Lock] — per-project lock registry
    • _get_workspace_lock(project_id) — returns or creates a lock for a project
    • provision_workspace() body wrapped in with _get_workspace_lock(project_id):

Design Decisions

  • Per-project locks — different projects can provision concurrently; only same-project requests serialize
  • Same pattern as _render_progress — uses module-level dict for lock storage
  • threading.Lock — sufficient for single-process multi-thread scenarios (uvicorn workers)

Phase H5: Frontend Code Sandbox — DONE

What Was Done

Files modified:

  • frontend/index.html — Added:

    • CSP meta tag: script-src 'self' 'unsafe-eval' (needed for Babel), connect-src 'self' https://*.blog2video.app (blocks generated code from fetching external URLs), restricted style-src, img-src, media-src, font-src

  • frontend/src/components/RemotionPreviewPlayer.tsx — Added:

    • PlayerErrorBoundary class component — catches React render errors from AI-generated components
    • Shows friendly error message with "Retry" button instead of crashing the app
    • Player JSX wrapped in <PlayerErrorBoundary>

Design Decisions

  • CSP as first line of defense — even if generated code contains fetch(), the browser blocks it from reaching external URLs
  • unsafe-eval required — @babel/standalone needs eval for JIT compilation; CSP restricts everything else
  • Error boundary on Player — catches runtime errors from compiled AI components; shows error UI instead of white screen

Phase H6: Self-healing current_version_id — DONE

What Was Done

File modified:

  • backend/app/routers/custom_templates.py — Updated:
    • list_versions() now checks if tpl.current_version_id exists in the fetched versions set
    • If not found (dangling pointer), auto-sets to the newest version's ID and commits

Design Decisions

  • Lazy self-healing — only checks when versions are listed (read path), not on every template access
  • Newest version wins — if pointer is dangling, the most recent version is the safest default

Phase F1: BrandKit Population During Theme Extraction — DONE

What Was Done

Files modified:

  • backend/app/routers/custom_templates.py — Updated:

    • ExtractThemeResponse now includes logo_urls, og_image, screenshot_url from ScrapedThemeData
    • CreateCustomTemplateRequest now accepts optional logo_urls, og_image, screenshot_url
    • extract_theme() passes scraped logo/image data through to the response
    • create_custom_template() now creates a BrandKit row from theme data (colors, fonts, design_language, logos, images) and links it via brand_kit_id

  • frontend/src/api/client.ts — Updated:

    • ExtractThemeResponse interface includes logo_urls, og_image, screenshot_url
    • createCustomTemplate() accepts optional logo_urls, og_image, screenshot_url

  • frontend/src/components/CustomTemplateCreator.tsx — Updated:

    • Stores scraped logo_urls, og_image, screenshot_url from extract response in state
    • Passes them to createCustomTemplate() call so BrandKit is populated

Design Decisions

  • BrandKit auto-created on template creation — every new template gets a linked BrandKit, populating data that code_generator.py already reads (lines 199-207)
  • Logo URLs stored as JSON list — scraped logo URLs are raw strings; F2 adds structured upload with primary/secondary types
  • OG image stored in BrandKit.images — serves as the initial brand image until user uploads custom ones

Phase F2: Brand Asset Uploads (Logo) — DONE

What Was Done

Files modified:

  • backend/app/services/r2_storage.py — Added:

    • brand_asset_key(user_id, brand_kit_id, filename) helper for R2 key generation

  • backend/app/routers/custom_templates.py — Added:

    • POST /{template_id}/upload-logo endpoint — validates file type (PNG/JPEG/WebP/SVG) and size (2MB), auto-creates BrandKit if missing, uploads to R2, updates bk.logos JSON with primary logo entry
    • FastAPI UploadFile, File imports

  • backend/app/services/template_service.py — Updated:

    • _load_custom_template_data() now includes brand_kit dict (colors, fonts, logos, design_language, images) in cached result

  • backend/app/services/remotion.py — Updated:

    • write_remotion_data() reads brand_kit logos from custom_data, downloads primary logo to workspace public/ folder, sets data["brandLogo"] for GeneratedVideo access via staticFile()

  • frontend/src/api/client.ts — Added:

    • uploadTemplateLogo(templateId, file) API method

  • frontend/src/components/TemplateCodeEditor.tsx — Added:

    • Hidden file input + "Logo" button in preview panel header
    • handleLogoUpload() — uploads file, updates template state, shows success/error message in chat

Design Decisions

  • Auto-create BrandKit on upload — if template was created before F1, uploading a logo still works by auto-creating a minimal BrandKit
  • Logo format migration — handles old BrandKit.logos format (list of URL strings) and new format (list of dicts with type/url)
  • Primary logo replacement — uploading a new logo replaces the previous primary, preserving scraped logos as type "scraped"
  • Brand logo in data.json — downloaded to workspace public/brand-logo.png for staticFile() access in GeneratedVideo

Phase F5: Version Snapshots on Chat Edits — DONE

What Was Done

File modified:

  • backend/app/routers/custom_templates.py — Updated:
    • edit_code() now counts total edits for the template after each edit
    • Every 3rd edit, auto-saves a version snapshot via _save_version(tpl, f"After {count} edits", db)

Design Decisions

  • Every 3rd edit — balances version granularity vs. list clutter; users get checkpoints at edits 3, 6, 9, etc.
  • Combined with existing versioning — these auto-versions appear alongside generate/regenerate/rollback versions in the version panel
  • Non-blocking — version save happens in the same DB transaction as the edit, no extra overhead

Phase F6: custom_prompt_builder.py Cleanup — DONE

What Was Done

File modified:

  • backend/app/services/custom_prompt_builder.py — Updated:
    • build_custom_meta() simplified: removed valid_arrangements, preferred_arrangements, arrangements_without_image, hero_arrangement, fallback_arrangement
    • Kept valid_layouts, layouts_without_image (used by SceneEditModal), hero_layout, fallback_layout (used by scene generation)
    • Added docstring explaining dual purpose: build_custom_prompt() is for DSPy AI script generation, build_custom_meta() is for pipeline routing

Design Decisions

  • Prompt preserved — build_custom_prompt() still generates the full 300-line prompt for DSPy scene generation (narration, visual hints) even though GeneratedVideo doesn't use arrangement/element layout data
  • Meta simplified — removed arrangement-specific fields that GeneratedVideo ignores; kept layout fields that the API pipeline (SceneEditModal dropdown) still uses

Phase F3: Enhanced Scraping with Firecrawl Branding Format — DONE

What Was Done

Files modified:

  • backend/app/services/theme_scraper.py — Updated:

    • ScrapedThemeData has new branding: dict | None field
    • scrape_for_theme() tries formats=["branding", "html", "markdown", "screenshot"] first
    • Falls back to standard formats if branding not available on the Firecrawl plan
    • Normalizes branding_data to dict (handles SDK object or raw dict)

  • backend/app/dspy_modules/theme_extractor.py — Added:

    • _apply_branding_overrides() static method on ThemeExtractor
    • After DSPy theme extraction, merges Firecrawl branding data as high-confidence overrides
    • Overrides: colors.primary→accent, colors.background→bg, colors.text→text, colors.secondary→surface
    • Overrides: typography.headingFont→fonts.heading, typography.bodyFont→fonts.body
    • Called in extract_theme() only when scraped.branding is present and is a dict

Design Decisions

  • Graceful fallback — branding format may not be available on all Firecrawl plans; try/except ensures zero regression
  • Override, not replace — Firecrawl branding values are merged on top of DSPy-extracted theme; DSPy still provides style, animation, patterns, category
  • Only override when present — each branding field is checked individually; missing fields are left as DSPy extracted them

Phase F4: Preview Thumbnails for Template Gallery — DONE

What Was Done

New file created:

  • backend/app/services/thumbnail_renderer.py — Renders single-frame PNG thumbnail:
    • render_template_thumbnail(template_id, user_id) — provisions temporary workspace, writes mock data.json with 1 intro scene, runs npx remotion still GeneratedVideo at 480p, uploads PNG to R2
    • Uses negative project IDs for temp workspace to avoid collision
    • Cleanup: removes temp workspace in finally block

Files modified:

  • backend/app/routers/custom_templates.py — Updated:

    • _render_and_store_thumbnail() background task helper — calls renderer, stores URL in tpl.preview_image_url
    • generate_code() and regenerate_code() now accept BackgroundTasks and kick off thumbnail render after code gen
    • _serialize_template() now includes preview_image_url

  • frontend/src/api/client.ts — Updated:

    • CustomTemplateItem includes preview_image_url: string | null

  • frontend/src/components/templatePreviews/CustomPreview.tsx — Rewritten:

    • If previewImageUrl exists and user hasn't hovered, shows static image with play icon overlay (instant load)
    • On hover, lazy-loads the full RemotionPreviewPlayer (Babel JIT compilation)
    • Suspense fallback uses thumbnail image if available

  • frontend/src/pages/CustomTemplates.tsx — Updated:

    • Passes previewImageUrl={tpl.preview_image_url} to CustomPreview in template grid

Design Decisions

  • Background task — thumbnail rendering is non-critical; failure doesn't affect code generation response
  • remotion still — uses Remotion's built-in still image command (frame 0) instead of full video render; much faster
  • 480p resolution — sufficient for gallery thumbnails, keeps file size small
  • Hover-to-play — template grid loads instantly with static images; @babel/standalone (~3MB) only loaded when user interacts
  • Thumbnail as Suspense fallback — when Player is loading, the thumbnail serves as a smooth placeholder instead of a blank rectangle

Phase F7: Multiple Unique Content Scene Variants — DONE

What Was Done

Previously, Claude generated only 3 variants (1 intro, 1 content, 1 outro). All content scenes reused the same component — different text/image props but identical layout. This was the same fundamental limitation as the old UniversalScene system.

Now, Claude decides how many unique scene variants to create (typically 5-8 total). Each content scene gets a visually distinct layout, and scenes cycle through variants at render time.

Before: Intro → Content (same layout × N) → Outro After: Intro → Scene1 → Scene2 → Scene3 → Scene4 → Scene5 → Outro (all unique layouts)

Files modified:

  • backend/app/models/custom_template.py — Added content_codes (Text, nullable) — JSON array of content variant code strings
  • backend/app/models/template_version.py — Added content_codes (Text, nullable) — version snapshots include all content variants
  • backend/app/database.py — Added content_codes to SQLite migrations for both custom_templates and template_versions
  • backend/app/services/code_generator.py — Major rewrite:
    • New SYSTEM_PROMPT: asks Claude to generate 5-8 scene variants using <SCENE_0>, <SCENE_1>, ..., <SCENE_N> XML tags
    • Claude decides how many unique layouts the brand needs
    • Scene 0 = intro, last scene = outro, all middle scenes = unique content variants
    • _parse_scene_variants() replaces _parse_variants() — parses dynamic numbered XML tags
    • Returns content_codes: list[str] instead of single component_code
    • max_tokens increased from 8000 to 16000 for multiple variants
  • backend/app/routers/custom_templates.py — Updated:
    • EditCodeRequest.variant now accepts content_N format (e.g., content_0, content_3)
    • _parse_variant() helper parses variant strings into (type, index) tuples
    • _get_variant_code() / _set_variant_code() handle reading/writing content codes by index
    • generate_code() and regenerate_code() store content_codes as JSON
    • _save_version() snapshots content_codes
    • Rollback restores content_codes
    • _serialize_template() includes content_codes (parsed from JSON)
  • backend/app/services/template_service.py — Updated:
    • _load_custom_template_data() parses and caches content_codes
    • has_generated_code checks both component_code and content_codes
  • backend/app/services/remotion.py — Updated:
    • _write_generated_scene_files() now writes SceneContent0.tsx, SceneContent1.tsx, ..., SceneContentN.tsx
    • Generates contentRegistry.ts — exports CONTENT_VARIANTS array with all content components
    • Generates SceneContent.tsx (re-exports Content0 for backward compat)
    • write_remotion_data() adds contentVariantIndex and contentVariantCount to data.json
    • Content scenes are assigned variant indices cyclically (scene[i] uses variant[i % numVariants])

New files created:

  • remotion-video/src/templates/generated/contentRegistry.ts — Placeholder; at render time, overwritten with imports of all content variant components
  • remotion-video/src/templates/generated/SceneContent0.tsx — Placeholder; re-exports SceneContent

Files rewritten:

  • remotion-video/src/templates/generated/GeneratedVideo.tsx — Now imports from contentRegistry.ts instead of single ContentScene. getSceneComponent() picks variant by contentVariantIndex from data.json.
  • remotion-video/src/templates/generated/types.ts — Added contentVariantIndex and contentVariantCount fields
  • frontend/src/components/TemplateCodeEditor.tsx — Dynamic variant tabs built from template.content_codes. Shows "Intro | Scene 1 | Scene 2 | ... | Scene N | Outro" tabs. Each tab can be independently edited via chat.
  • frontend/src/api/client.ts — CodeVariant type changed from "component" | "intro" | "outro" to "intro" | "outro" | \content_${number}`. CustomTemplateItemincludescontent_codes: string[] | null`.

Architecture Decisions

  • Claude decides count — The prompt asks for "at least 5 variants" but lets Claude decide the right number (5-8) based on the brand personality. Complex brands might get more variety.
  • Cyclic assignment — If there are 5 content variants and 8 content scenes, scenes cycle: 0,1,2,3,4,0,1,2. Simple, predictable, no AI needed at render time.
  • Backward compatibility — component_code is always set to the first content variant. Old code that reads component_code still works (preview, gallery, etc).
  • Per-variant editing — User selects a specific scene tab (e.g., "Scene 3") and edits just that variant. Edits are scoped and precise.
  • contentRegistry.ts pattern — Instead of dynamic imports (which Vite can't bundle), we generate a registry file at render time that statically imports all content components. Vite bundles them normally.
  • max_tokens increase — 8000 → 16000 to accommodate 5-8 full React components in one API call.

Verification Steps

  1. Start backend: cd backend && uvicorn app.main:app --reload --port 8000 — no errors, content_codes column auto-created
  2. Create a template and generate code — should see "Generated N scene variants (1 intro + X content + 1 outro)" in backend logs
  3. On Custom Templates page, click "Customize" — editor should show tabs: "Intro | Scene 1 | Scene 2 | ... | Outro"
  4. Click each tab — different preview for each scene variant
  5. Edit a specific variant via chat — only that variant changes
  6. Trigger a video render — check workspace: should have SceneContent0.tsx through SceneContentN.tsx
  7. Rendered video should have visually different layouts for each content scene
  8. TypeScript check passes: cd remotion-video && npx tsc --noEmit ✓

Phase OPT: Custom Template Performance & Quality Optimization — DONE

Overview

5 fixes targeting custom template speed (246s → ~140s), animation quality, and UX. Previously code generation was blocking, scenes were 500-800 lines with repetitive animations, and CSS/logo scraping was broken.

Fix 1: CSS & Logo Scraping Improvements — DONE

Files modified:

  • backend/app/services/theme_scraper.py
    • _extract_css_content() now fetches external <link rel="stylesheet"> CSS files (first 2-3, up to _MAX_CSS_CHARS) in addition to inline <style> blocks
    • Logo fallbacks improved: og:image added to logos list, SVG logo detection via logo-like class names, favicon.ico fallback
    • Note: SPA sites (e.g. noon.com) use CSS-in-JS, so CSS=0 is expected for them — the AI reads inline styles from the rendered HTML instead

Fix 2: Theme Extraction Speed — DONE

Files modified:

  • backend/app/services/theme_scraper.py
    • _MAX_HTML_CHARS reduced: 40,000 → 15,000 (AI only needs enough HTML for colors/fonts/layout patterns)
    • _MAX_MARKDOWN_CHARS reduced: 5,000 → 2,000
  • backend/app/dspy_modules/__init__.py
    • get_theme_lm() model changed: anthropic/claude-sonnet-4-6 → anthropic/claude-haiku-4-5-20251001 (structured JSON extraction doesn't need Sonnet)

Impact: Theme extraction ~25s → ~13.8s (tested on noon.com)

Fix 3: Code Generation Speed — DONE

Files modified:

  • backend/app/services/code_generator.py
    • Line target in prompt: "There is NO line limit" → "Target 200-350 lines. Spend tokens on animation logic, not verbose inline styles."
    • Brevity reward in _scene_reward(): +0.05 for 200-350 lines, -0.10 for >500 lines
  • backend/app/dspy_modules/__init__.py
    • get_custom_lm() max_tokens: 8192 → 5120 (~300-400 lines of JSX, matching built-in template complexity)
    • All 5 content variants kept (NUM_CONTENT_VARIANTS=5)

Impact: Scene sizes reduced from 500-800 lines → 242-354 lines. Fewer truncation retries.

Fix 4: Background Code Generation — DONE

Files modified:

  • backend/app/routers/custom_templates.py
    • POST /{template_id}/generate-code now returns 202 immediately, runs generation in background thread via asyncio.get_event_loop().run_in_executor()
    • In-memory progress dict: _codegen_progress[template_id] with status/step/running/error (same pattern as _pipeline_progress in pipeline.py)
    • New GET /{template_id}/generation-status endpoint for polling
  • frontend/src/components/CustomTemplateCreator.tsx
    • After 202 response, polls generation-status every 2s
    • Shows progress steps, allows closing modal during generation
  • frontend/src/api/client.ts
    • Added getCodeGenerationStatus(templateId) API function

Impact: Frontend no longer blocks during code generation. User can navigate away.

Fix 5: Animation Quality & Creative Freedom — DONE

Files modified:

  • backend/app/services/code_generator.py
    • Deleted _VARIANT_SPECIALIZATIONS dict (~150 lines of hardcoded JSX recipes that killed creativity)
    • Added _CONTENT_VARIANT_ROLES — lightweight content-type assignments (name, content_types, role description) giving Claude full creative freedom
    • Rewrote _build_creative_direction() — assigns content specialization but lets Claude design layout/animation freely
    • Enhanced GenerateSceneCode signature:
      • Animation technique reference section (word-by-word spring, image glow entrance, decorative orb) — examples not recipes
      • Quality rules: ≥3 different spring configs, staggered text reveal, decorative background element, proper Img handling
      • Forbidden: defaulting to simple opacity+translateY for everything
    • Strengthened _scene_reward():
      • +0.10 for diverse springs (≥3 unique damping/stiffness configs)
      • +0.10 for staggered text animation (.map + spring + stagger timing)
      • +0.10 for Img + animation (spring/interpolate near Img usage)
      • +0.05 for glow/blur effects
      • -0.10 for opacity+translateY only (no other animation types)
      • -0.15 for scenes that completely ignore imageUrl (images won't render)
    • Quality debug logging: each scene logs techniques used (staggered-text, blur, radial-glow, scale, rotate, translateX, clip-path, easing) and spring configs

Impact: Generated scenes now have diverse animations (staggered-text, blur, radial-glow, scale, rotate, easing), 4 unique spring configs per scene, and scores of 1.00-1.30.

Design System Optimization — DONE

Problem: Design system generation took 43.6s (Sonnet) producing 6323 chars including spring configs. All 7 scenes copied the same springs, producing identical motion feel.

Files modified:

  • backend/app/services/code_generator.py
    • GenerateDesignSystem signature slimmed: 7 categories → 3 (background treatment, card/container CSS, text treatment)
    • Explicitly excludes: "Do NOT include spring configs, animation physics, decorative elements, or entrance patterns. Those are creative choices each scene makes independently."
    • Output constrained to ~1500 chars
    • Uses get_theme_lm() (Haiku) instead of get_custom_lm() (Sonnet) — structured CSS extraction doesn't need creative reasoning
    • GenerateSceneCode prompt updated: "Follow the design_system for visual styling (colors, cards, backgrounds, text treatment). Invent your OWN unique spring physics and animation timing — each scene should have distinctly different motion feel."

Impact: Design system ~43.6s → ~8-12s (Haiku + shorter output). Springs now vary per scene instead of being identical across all 7 scenes.

Content Variant Diversity — DONE

Problem: When all blog content is the same type (e.g. noon.com: all bullets), every scene got routed to the same variant (variant 0, lists specialist). All scenes looked identical.

Files modified:

  • backend/app/services/remotion.py
    • Variant routing updated: specialist variant used for the first occurrence of each content type only
    • Subsequent scenes with the same content type cycle through other variants for visual diversity
    • Example: 5 bullet scenes → variant 0 (specialist), 1 (cycling), 2, 3, 4 — all different layouts

Impact: Even when all content is the same type, every scene gets a different visual treatment.

Test Results (noon.com)

Metric Before After Target
CSS chars 0 0 (expected for SPA) > 0 for traditional sites
Logos 0 1 > 0
Theme AI time 25s 13.8s < 12s
Codegen total 246s 177.7s (pending design system fix) < 140s
Scene lines 500-800 242-354 200-350
Spring configs 1 identical 4 unique per scene ≥ 3
Animation techniques opacity+translateY staggered-text, blur, radial-glow, scale, rotate, translateX, easing diverse
Frontend blocking yes (60-250s) 202 + polling non-blocking

Phase V2: Content-Aware Architecture (feat/customv2) — DONE

Problem

Custom templates produced repetitive, content-blind videos. Built-in templates look great because each scene gets a layout matched to its content. Custom templates were generic because:

  • 5 hardcoded layout families — every brand got the same structural layouts
  • Blind cycling — scenes rotated through variants regardless of content (metrics scene might get kinetic_typography)
  • 16 wasted DSPy calls per video generating layoutConfig that the renderer ignored
  • Images broken — only one layout family rendered <Img>, others silently ignored images
  • Preview broken — contentVariantIndex written to data.json but not DB

What Was Done

1. AI-Decided Brand Scene Types (DecideBrandSceneTypes DSPy Signature)

Removed: CONTENT_ARCHETYPES (10 hardcoded archetypes), _LAYOUT_FAMILIES, NUM_CONTENT_VARIANTS=5, _build_creative_direction(), _VARIANT_SPECIALIZATIONS, _CONTENT_VARIANT_ROLES

Added: DecideBrandSceneTypes DSPy signature — one call per brand that outputs 6-8 scene types tailored to the brand.

  • Example: Metal (fundraising SaaS) → brand_hook_intro, feature_spotlight, how_it_works, metrics_proof, vs_traditional, fundraising_timeline, founder_voice, cta_outro
  • Example: Al Jazeera (news) → completely different set of scene types
  • Each scene type has best_for tags mapping to content types (bullets, metrics, quote, etc.)

Files: backend/app/services/code_generator.py — complete rewrite of generation flow

2. Content Extraction & Archetype Matching

New file: backend/app/services/content_classifier.py

  • extract_structured_content_batch() — ONE cheap Haiku call classifies ALL scene narrations into contentType (metrics/bullets/quote/comparison/timeline/steps/code/plain) and extracts structured data
  • match_scenes_to_archetypes() — deterministic matching: contentType → best archetype based on best_for tags, with anti-repeat logic so consecutive scenes never get the same archetype

Files modified:

  • backend/app/routers/pipeline.py — custom template branch now uses batch extraction instead of 16 per-scene DSPy calls
  • backend/app/services/remotion.py — content-aware matching replaces blind cycling, persists contentVariantIndex to DB
  • frontend/src/components/VideoPreview.tsx — reads contentVariantIndex from DB instead of CONTENT_TYPE_TO_VARIANT

3. Alembic Migration for content_archetype_ids

New file: backend/alembic/versions/phase8_content_archetype_ids.py

  • Adds content_archetype_ids TEXT column to custom_templates table
  • Merges 3 divergent alembic heads into one

4. Preview System Enrichment

File: frontend/src/components/templatePreviews/CustomPreview.tsx — complete rewrite

  • Replaced simple buildSampleData() with archetype-aware buildArchetypeSampleData() returning rich structured data per content type (metrics with values, bullets, quotes, comparison, timeline, steps, code)
  • Scene crossfade transitions with dot navigation
  • Pre-compiles ALL scene codes on mount (eliminates per-scene "Compiling preview..." flash)

File: frontend/src/components/templatePreviews/CustomPreviewLandscape.tsx

  • Delegates to CustomPreview when generated scene code exists
  • Falls back to FallbackSlides component when no code

File: frontend/src/components/BlogUrlForm.tsx

  • Template picker passes full template data (introCode, outroCode, contentCodes, contentArchetypeIds, logoUrls, ogImage)
  • Selected template preview shows real generated scenes instead of hardcoded slides

5. Image & Logo Rendering (MANDATORY in AI-generated scenes)

Problem: AI-generated components ignored props.imageUrl and props.logoUrl — pipeline delivered images correctly but components didn't render them.

Root cause: Not the pipeline — GeneratedVideo.tsx correctly maps scene.images[0] → imageUrl, and VideoPreview.tsx correctly passes imageUrl from sceneImageMap. The issue was that AI-generated React code never referenced these props.

Fix — Prompt (code_generator.py GenerateSceneCode):

  • Images & Logo section marked MANDATORY with explicit JSX patterns
  • Image techniques from built-in templates: Ken Burns zoom, radial vignette reveal, clipPath slit reveal, gradient overlays (3-layer: vignette + bottom gradient + accent wash)
  • Adaptive layout: const hasImage = !!props.imageUrl — with image: split layout or full-bleed overlay; without image: text expands, larger fonts, particles/gradient as visual interest
  • Both modes must look intentionally designed

Fix — Text animations:

  • Word-by-word / line-by-line reveals with staggered springs
  • Typewriter effect with blinking cursor
  • Scale-punch for key words (bouncy overshoot spring)
  • Bullet stagger (slide from right, delay=20+i*10)
  • Exit animations 20-30 frames before durationInFrames

Fix — Scene motion:

  • Multiple spring configs with different damping/stiffness/mass
  • Metric count-ups, card fly-ins, decorative corner shapes, accent line grows
  • Parallax depth, ambient gradient shifts, pulsing accent glows
  • Reference configs: fast={damping:22,stiffness:140,mass:1.2}, bouncy={damping:14,stiffness:220,mass:1.1}, smooth={damping:20,stiffness:70}

Fix — Reward function penalties:

  • -0.2 if code doesn't reference logoUrl
  • -0.2 if code doesn't reference imageUrl
  • -0.3 for non-monotonic interpolate() inputRange (catches runtime crashes like [0,60,120,90])
  • Combined with existing checks (overflow:hidden, hardcoded data, visible contentType/sceneIndex)

6. Code Generator Architecture (current state) 

generate_component_code(template)
  │
  ├─ _build_brand_context() — raw data only (colors, fonts, patterns, brand kit)
  │
  ├─ _decide_brand_scene_types() — ONE DSPy call → 6-8 brand-specific scene types
  │
  ├─ _generate_design_system() — ONE Haiku call → concise CSS design system (<2000 chars)
  │
  └─ asyncio.gather(*tasks) — ALL scenes in PARALLEL via ThreadPoolExecutor(max_workers=8)
       └─ _generate_single_scene() → dspy.Refine(ChainOfThought(GenerateSceneCode), reward_fn=_scene_reward)
            └─ Up to 3 attempts per scene (1 initial + 2 retries if score < 0.75)

Output: { intro_code, outro_code, content_codes: list[str], archetype_ids: list[dict] }

Files Changed Summary

File Change
backend/app/services/code_generator.py Complete rewrite: DecideBrandSceneTypes, stripped prescriptive code, enriched animation/image prompt, reward penalties for missing image/logo
backend/app/services/content_classifier.py NEW: batch content extraction (Haiku) + deterministic archetype matching
backend/app/routers/pipeline.py Custom template branch uses batch extraction
backend/app/services/remotion.py Content-aware matching, passes structuredContent to scenes
backend/app/routers/custom_templates.py content_archetype_ids serialization, regeneration endpoint
backend/app/routers/projects.py Scene regeneration for custom templates
backend/app/models/custom_template.py content_archetype_ids column
backend/app/database.py SQLite migration for content_archetype_ids
backend/app/services/template_service.py Loads and caches archetype metadata
backend/alembic/versions/phase8_content_archetype_ids.py PostgreSQL migration
frontend/src/components/templatePreviews/CustomPreview.tsx Archetype-aware previews with crossfade
frontend/src/components/templatePreviews/CustomPreviewLandscape.tsx Delegates to CustomPreview for generated code
frontend/src/components/BlogUrlForm.tsx Template picker passes full template data
frontend/src/components/VideoPreview.tsx Reads contentVariantIndex from DB
frontend/src/components/CustomTemplateCreator.tsx Centered step indicator
frontend/src/pages/CustomTemplates.tsx Passes archetype data to previews
frontend/src/pages/ProjectView.tsx Layout label from archetype name
frontend/src/api/client.ts content_archetype_ids types

Test Results

Three brands tested end-to-end (template creation → scene generation → video creation):

Brand Scene Types Decided Content Extraction Archetype Matching Generation Time
Metal (fundraising SaaS) 8 types: brand_hook_intro, feature_spotlight, how_it_works, metrics_proof, vs_traditional, fundraising_timeline, founder_voice, cta_outro All scenes classified correctly Anti-repeat working ~137s
Al Jazeera (news) Unique news-focused types Classified correctly Working ~130s
Nestlé Pakistan (FMCG) Unique consumer brand types Classified correctly Working ~135s

All regenerated scenes score 1.00 (image/logo handling verified by reward function).

Next Improvements (TODO)

# Improvement Priority Description
N2 brandImages Pass-through in frontend preview Low VideoPreview.tsx doesn't pass brandImages to scene props (Remotion pipeline does). Add it for preview parity.
N7 Per-Plan AI Limits Low AI_DAILY_LIMIT per-plan: Free: 5, Standard: 20, Pro: 50.