docs/embed-studio.md

# Embed Studio - Architecture Document

## Overview

The Embed Studio is a dedicated UI mode within the editor for creating, editing, and previewing HTML embed visualizations (D3.js charts). It isolates the dataviz workflow from the article editing flow, providing a focused chat + preview experience similar to the standalone [dataviz-agent-space](https://huggingface.co/spaces/tfrere/dataviz-agent-space).

## Context

The research-article-template uses `<HtmlEmbed>` components to embed self-contained D3.js charts into articles. These are `.html` files in `app/src/content/embeds/` with strict conventions (scoped CSS, IIFE scripts, ColorPalettes, responsive, etc.) documented in `.ai/skills/create-html-embed/directives.md`.

In the editor, users need to create and iterate on these charts without leaving the editor. The Embed Studio solves this by providing a dedicated panel with an AI assistant specialized in D3 chart generation.

## Storage

### Y.Map("embeds")

Embed HTML content is stored in a collaborative Yjs Map, keyed by filename:

```
Y.Map("embeds") = {
  "d3-scaling-chart.html": "<div class='d3-scaling-chart'>...</div>",
  "d3-performance.html": "<div class='d3-performance'>...</div>"
}
```

The ProseMirror node (`htmlEmbed`) only stores the `src` attribute as a reference key. The actual HTML lives in the shared Y.Map, enabling real-time collaboration on embed content.

### Node attributes

The `htmlEmbed` TipTap node stores:

| Attribute | Type | Description |
|-----------|------|-------------|
| `src` | string | Filename key into Y.Map("embeds") |
| `title` | string | Chart title (displayed above) |
| `desc` | string | Chart description |
| `wide` | boolean | Wide layout mode |
| `downloadable` | boolean | Show download button |
| `height` | number | Last known content height (pixels) |

The `height` attribute eliminates layout jumps: once a chart reports its height, it is persisted and used as the iframe's initial height on subsequent loads.

## UI: Two Modes

### 1. Inline Preview (article view)

When the user is editing the article, the `htmlEmbed` NodeView shows a read-only preview:

```
┌─────────────────────────────────────────┐
│ 📊 Chart Title              [Edit] [⋮] │
├─────────────────────────────────────────┤
│                                         │
│         <iframe preview>                │
│                                         │
└─────────────────────────────────────────┘
```

- The iframe renders the chart using `srcdoc` with the full HTML document (buildDoc wrapper)
- Height comes from the stored `height` attribute (default: 400px)
- Clicking "Edit" opens the Embed Studio panel
- No code editing in this mode

### 2. Embed Studio Panel (creation/editing)

A full-width panel (drawer or overlay) opens with a split layout:

```
┌────────────────────────────┬─────────────────────────────┐
│  Chat (D3 context)         │  Live Preview               │
│                            │                             │
│  System prompt includes:   │  ┌─────────────────────┐    │
│  - D3 embed directives     │  │                     │    │
│  - ColorPalettes API       │  │   [rendered chart]   │    │
│  - Current chart HTML      │  │                     │    │
│                            │  └─────────────────────┘    │
│  User: "make a bar chart   │                             │
│   showing model sizes"     │  Toggle: [Preview] [Code]   │
│                            │                             │
│  AI: Creating chart...     │                             │
│                            │       [Save & Close]        │
└────────────────────────────┴─────────────────────────────┘
```

**Key design decisions:**

- The chat in this panel has a **separate system prompt** with D3 directives injected. This avoids bloating the main article chat with 500+ lines of D3 conventions.
- The chat history is **per-embed** (scoped to the `src` key), so each chart has its own conversation thread.
- The live preview uses **double-buffered iframes** (A/B swap with cross-fade) from the dataviz-agent pattern to avoid flashes on update.
- An optional "Code" toggle shows the raw HTML for power users (future enhancement).

## AI Tools

The Embed Studio provides three tools to the AI (following the dataviz-agent pattern):

### createEmbed(src, html, title, source)

Create or fully replace the HTML for an embed. Writes to `Y.Map("embeds")`.

### patchEmbed(src, search, replace)

Exact string replacement in the current HTML. More efficient than full rewrite for small changes (color tweaks, label updates, data changes). Reads from and writes to `Y.Map("embeds")`.

### readEmbed(src)

Read the current HTML content. The AI should call this before patching to verify exact content.

## Preview Infrastructure

### buildDoc(html, isDark, primaryColor)

Wraps a chart HTML fragment into a complete HTML document with:

- CSS variables for theming (`--primary-color`, `--text-color`, `--surface-bg`, etc.)
- `data-theme="dark"` attribute when in dark mode
- ColorPalettes polyfill (provides `window.ColorPalettes.getColors()`, `.getPrimary()`, etc.)
- Height reporting script (see below)
- Base styles (box-sizing, font stack, padding)

### Height reporting

A script injected by `buildDoc()` observes the chart container and reports its height to the parent:

```js
// Injected into every chart iframe
new ResizeObserver(entries => {
  const height = Math.ceil(entries[0].contentRect.height);
  window.parent.postMessage({ type: 'embedResize', height }, '*');
}).observe(document.body);
```

The NodeView listens for this message and updates the node's `height` attribute:

```js
window.addEventListener('message', (e) => {
  if (e.data?.type === 'embedResize') {
    updateAttributes({ height: e.data.height });
  }
});
```

On subsequent renders, the iframe starts at the stored height, eliminating layout jumps.

### Preview strategy: srcdoc first

Initial implementation uses `<iframe srcdoc="...">` directly. This avoids backend changes and keeps the architecture simple.

If we hit limitations (CSP restrictions, large HTML payloads, script execution issues), we migrate to a server-side preview route (`POST /api/preview` returning an ID, iframe loads `/api/preview/:id`), following the dataviz-agent pattern.

## Export

### Updated export API

The `toMdx()` function returns an object instead of a plain string:

```typescript
interface ExportResult {
  mdx: string;                        // The MDX content with frontmatter
  embeds: Record<string, string>;     // filename -> HTML content
}
```

The caller is responsible for writing the embed files to `app/src/content/embeds/`.

### MDX output

Each embed in the article exports as:

```mdx
<HtmlEmbed src="d3-scaling-chart.html" title="Chart Title" desc="Description" />
```

The HTML files are exported separately from the Y.Map("embeds") contents.

## System Prompt for D3 Generation

The Embed Studio injects the D3 directives into the AI's system prompt. The content comes from two sources:

1. **Tool descriptions** (from dataviz-agent): create/patch/read tools with usage guidelines
2. **Embed conventions** (from research-article-template): structure, ColorPalettes, CSS variables, mount guard, D3 CDN loading, legends, controls, tooltips, responsiveness, error handling, accessibility, checklist

These are only injected when the Embed Studio is open, keeping the main article chat lightweight.

## Implementation Plan

### Phase 1: Core infrastructure

1. Create `Y.Map("embeds")` in `Editor.tsx` and pass to the embed store
2. Create `EmbedStore` (similar to FrontmatterStore) with get/set/observe/patch operations
3. Replace the current atomic `htmlEmbed` NodeView with an iframe-based preview
4. Implement `buildDoc()` with CSS variables and ColorPalettes polyfill
5. Implement height reporting via postMessage

### Phase 2: Embed Studio panel

6. Create `EmbedStudio.tsx` - the split-panel UI (chat + preview)
7. Create a dedicated chat hook (`useEmbedChat`) with D3 system prompt
8. Implement `createEmbed`, `patchEmbed`, `readEmbed` AI tools (backend + frontend)
9. Double-buffered iframe preview (A/B swap)
10. Wire "Edit" button on inline NodeView to open the studio

### Phase 3: Polish

11. Per-embed chat history persistence
12. Code view toggle
13. Export API update (`toMdx` returns `{ mdx, embeds }`)
14. Data file upload support (CSV/JSON stored in Y.Map)
15. Screenshot-based validation (Playwright, optional)

## References

- [dataviz-agent-space](../../../dataviz-agent-space/) - Standalone D3 chart generation agent
- [research-article-template embeds skill](../../../research-article-template/.ai/skills/create-html-embed/) - Embed authoring conventions
- [ChartFrame.jsx](../../../dataviz-agent-space/frontend/src/components/ChartFrame.jsx) - Double-buffered iframe + buildDoc pattern