NextJS_Personal_Caption_Studio_PRD.md

# PRD: Personal Video Caption Studio
## Next.js App (Private Use)
### Version: 1.0 | Date: July 5, 2026

---

## 1. WHAT THIS IS

A personal web app that burns animated, word-by-word captions onto videos. 
You upload a video + text (or SRT), style the captions, preview, and download the result.

**Inspiration**: The viral TikTok/Reels caption style — bold text, black stroke, bounce animation.

---

## 2. HOW IT WORKS (USER FLOW)

```
Upload Video ──► Paste Text / Upload SRT ──► Style Captions ──► Preview ──► Export MP4
     │                  │                        │                │            │
     ▼                  ▼                        ▼                ▼            ▼
  Drag & drop      Type or paste           Pick font,          Play with     Download
  or file picker   words manually          color, size,        captions      to disk
                   or import .srt          position,           burned in
                                           animation speed
```

---

## 3. FEATURES

### 3.1 Must-Have (MVP)

| # | Feature | How It Works |
|---|---------|-------------|
| 1 | **Upload video** | Drag & drop or click. Accept MP4, MOV, WebM. Max 500MB, 5 min. |
| 2 | **Add captions** | Textarea: type/paste words. Or upload `.srt` file. |
| 3 | **Auto-sync timing** | If no SRT: split text by words, estimate ~200ms per word + pauses. |
| 4 | **Style captions** | Font, size, fill color, stroke color, stroke width, shadow. |
| 5 | **Position** | Top/center/bottom + fine-tune Y offset. |
| 6 | **Preview** | Play video with real-time caption overlay in browser. |
| 7 | **Export** | Server-side FFmpeg renders video + captions → MP4 download. |

### 3.2 Nice-to-Have (v2)

| # | Feature | Notes |
|---|---------|-------|
| 8 | **Upload audio** | Separate voiceover track |
| 9 | **Auto-transcribe** | Whisper API (OpenAI) — costs money, optional |
| 10 | **Timeline editor** | Drag word timings visually |
| 11 | **Presets** | Save style combos ("Viral Yellow", "Minimal White", etc.) |
| 12 | **Batch export** | Queue multiple videos |

---

## 4. TECH STACK

```
Frontend (Browser)
├── Next.js 15 (App Router)
├── React 19 + TypeScript
├── Tailwind CSS 4
├── shadcn/ui (components)
├── Framer Motion (caption animations in preview)
└── React Player / native <video> (video playback)

Backend (Next.js API Routes + Server Actions)
├── FFmpeg (ffmepg-static or system install)
├── Canvas API / node-canvas (render caption frames)
├── Sharp (image processing, optional)
└── tmp (temp file cleanup)

Storage
├── Local filesystem (uploads + exports in /tmp or ./storage)
└── No database needed — state is ephemeral
```

---

## 5. DATA MODELS

```typescript
// types/index.ts

interface Project {
  id: string;                    // uuid
  videoPath: string;             // /tmp/uploads/abc123.mp4
  videoMeta: {
    width: number;
    height: number;
    duration: number;            // seconds
    fps: number;
  };
  captions: Caption[];
  style: CaptionStyle;
  layout: CaptionLayout;
  status: 'draft' | 'rendering' | 'done' | 'error';
  exportPath?: string;           // /tmp/exports/abc123_captioned.mp4
  createdAt: Date;
}

interface Caption {
  id: string;
  text: string;                  // single word or short phrase
  startMs: number;               // when to appear
  endMs: number;                 // when to disappear/dim
}

interface CaptionStyle {
  fontFamily: string;            // "Impact", "Oswald", "Anton", etc.
  fontSize: number;              // px, relative to video height
  fillColor: string;             // #FFD700
  strokeColor: string;           // #000000
  strokeWidth: number;           // px
  highlightColor?: string;       // #FFFFFF top highlight
  shadow: {
    color: string;
    blur: number;
    offsetX: number;
    offsetY: number;
  };
  uppercase: boolean;
  letterSpacing: number;
}

interface CaptionLayout {
  position: 'top' | 'center' | 'bottom';
  yOffset: number;               // px from position anchor
  maxWidthPercent: number;       // 0.9 = 90% of video width
  maxLines: number;              // how many lines visible at once
}
```

---

## 6. PAGE BREAKDOWN

### Page 1: `/` — Upload

```
┌─────────────────────────────────────────┐
│  🎬 Caption Studio          [Settings]  │
├─────────────────────────────────────────┤
│                                         │
│         ┌─────────────────┐             │
│         │                 │             │
│         │   DROP VIDEO    │             │
│         │    HERE         │             │
│         │   or click      │             │
│         │                 │             │
│         └─────────────────┘             │
│                                         │
│    MP4, MOV, WebM • Max 500MB • 5 min   │
│                                         │
└─────────────────────────────────────────┘
```

**Actions:**
- Drag & drop video → upload to `/tmp/uploads/{uuid}.mp4`
- Extract metadata via FFmpeg: `ffprobe -v error -select_streams v:0 -show_entries stream=width,height,r_frame_rate,duration`
- Redirect to `/editor?id={uuid}`

---

### Page 2: `/editor` — Captions + Style

```
┌─────────────────────────────────────────────────────────────┐
│  ← Back              Caption Studio              [Export]   │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  ┌─────────────────────────┐  ┌─────────────────────────┐  │
│  │                         │  │  CAPTION TEXT            │  │
│  │    ┌─────────────┐      │  │  ┌─────────────────┐    │  │
│  │    │             │      │  │  │ Type or paste   │    │  │
│  │    │   VIDEO     │      │  │  │ your captions   │    │  │
│  │    │   PREVIEW   │      │  │  │ here...         │    │  │
│  │    │             │      │  │  │                 │    │  │
│  │    │  [captions  │      │  │  │ Or upload .srt  │    │  │
│  │    │   overlay]  │      │  │  │                 │    │  │
│  │    │             │      │  │  └─────────────────┘    │  │
│  │    └─────────────┘      │  │                         │  │
│  │    ▶️ ◀️ ▶▶ ⏸️          │  │  [Auto-sync] [Clear]    │  │
│  │                         │  │                         │  │
│  └─────────────────────────┘  │  STYLE                  │  │
│                               │  ─────────────────────  │  │
│                               │  Font: [Impact ▼]       │  │
│                               │  Size: [━━━●━━━━] 48px  │  │
│                               │  Fill: [🟨] #FFD700     │  │
│                               │  Stroke: [⬛] #000 4px  │  │
│                               │  Highlight: [⬜] #FFF   │  │
│                               │  Shadow: [✓] blur:8     │  │
│                               │  Uppercase: [✓]         │  │
│                               │                         │  │
│                               │  POSITION               │  │
│                               │  [Top] [Center] [Bottom]│  │
│                               │  Y-offset: [━━●━━━━] 0  │  │
│                               │                         │  │
│                               │  ANIMATION              │  │
│                               │  Speed: [━━━●━━━] 1.0x  │  │
│                               │  Style: [Bounce ▼]      │  │
│                               │                         │  │
│                               │  [💾 Save Preset]       │  │
│                               └─────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘
```

**Left Panel — Preview:**
- `<video>` element playing uploaded video
- Overlay div with Framer Motion animated captions
- Captions rendered as absolutely-positioned `<span>` elements
- Animation mimics final FFmpeg output (for WYSIWYG)

**Right Panel — Editor:**
- **Text input**: `<textarea>` for raw text
- **Auto-sync button**: Split text by whitespace → assign timings:
  ```
  wordDuration = 180ms + (chars * 40ms)
  gapBetweenWords = 60ms
  ```
- **SRT upload**: Parse `HH:MM:SS,mmm --> HH:MM:SS,mmm` format
- **Style controls**: Color pickers, sliders, toggles
- **Presets**: JSON blobs saved to `localStorage`

**Caption Preview Animation (Framer Motion):**
```tsx
// components/CaptionWord.tsx
<motion.span
  initial={{ scale: 0.5, opacity: 0, y: 20 }}
  animate={{ 
    scale: isActive ? [1, 1.05, 1] : isPast ? 0.95 : 1,
    opacity: isPast ? 0.5 : 1,
    y: 0
  }}
  transition={{ 
    type: "spring", 
    stiffness: 400, 
    damping: 15,
    duration: 0.3 
  }}
>
  {word.text.toUpperCase()}
</motion.span>
```

---

### Page 3: `/export` — Render & Download

```
┌─────────────────────────────────────────┐
│  ← Back           Export Video          │
├─────────────────────────────────────────┤
│                                         │
│         ┌─────────────────┐             │
│         │                 │             │
│         │   PREVIEW       │             │
│         │   (low-res)     │             │
│         │                 │             │
│         └─────────────────┘             │
│                                         │
│  Settings:                              │
│  Resolution: [Original ▼] 1080x1920     │
│  Quality: [High ▼]                      │
│  Format: [MP4 (H.264) ▼]                │
│                                         │
│  ┌─────────────────────────────────┐    │
│  │  🎬 Rendering...                │    │
│  │  ████████████░░░░░░  67%        │    │
│  │  ~12 seconds remaining          │    │
│  │                                 │    │
│  │  [Cancel]                       │    │
│  └─────────────────────────────────┘    │
│                                         │
│         [📥 Download MP4]               │
│                                         │
└─────────────────────────────────────────┘
```

---

## 7. SERVER-SIDE RENDERING PIPELINE

### The Big Picture

```
Browser                              Server (Next.js API)
   │                                        │
   │── POST /api/upload ───────────────────►│
   │   (multipart video)                    │
   │◄─ 200 { id, meta } ────────────────────│
   │                                        │
   │── POST /api/render ───────────────────►│
   │   { id, captions, style, layout }      │
   │                                        │
   │   Server does:                         │
   │   1. Extract frames (FFmpeg)           │
   │   2. Render captions on each frame     │
   │      (Canvas 2D / node-canvas)         │
   │   3. Encode frames + audio → MP4       │
   │      (FFmpeg)                          │
   │                                        │
   │◄─ 200 { downloadUrl } ◄───────────────│
   │                                        │
   │── GET /api/download/{id} ─────────────►│
   │◄─ Stream MP4 ◄─────────────────────────│
```

### Step-by-Step Render Process

```typescript
// app/api/render/route.ts

import { NextRequest, NextResponse } from 'next/server';
import { exec } from 'child_process';
import { promisify } from 'util';
import { createCanvas, loadImage, registerFont } from 'canvas';
import fs from 'fs/promises';
import path from 'path';

const execAsync = promisify(exec);

export async function POST(req: NextRequest) {
  const { id, captions, style, layout } = await req.json();

  const uploadDir = `/tmp/uploads/${id}`;
  const renderDir = `/tmp/renders/${id}`;
  const outputPath = `/tmp/exports/${id}_captioned.mp4`;

  await fs.mkdir(renderDir, { recursive: true });

  // ── STEP 1: Extract frames ─────────────────────────────
  // Extract at video FPS, e.g. 30fps
  await execAsync(`
    ffmpeg -i ${uploadDir}/video.mp4 
    -vf "fps=30,scale=1080:-1:flags=lanczos" 
    -q:v 2 
    ${renderDir}/frame_%06d.png
  `);

  const frames = await fs.readdir(renderDir);
  const fps = 30;
  const frameDurationMs = 1000 / fps;

  // ── STEP 2: Render captions on each frame ──────────────
  for (let i = 0; i < frames.length; i++) {
    const currentMs = i * frameDurationMs;
    const framePath = path.join(renderDir, frames[i]);

    // Load frame
    const img = await loadImage(framePath);
    const canvas = createCanvas(img.width, img.height);
    const ctx = canvas.getContext('2d');

    // Draw original frame
    ctx.drawImage(img, 0, 0);

    // Determine which captions are visible
    const visibleCaptions = captions.filter(c => 
      c.startMs <= currentMs && c.endMs + 200 >= currentMs
    );

    // Render each visible caption word
    for (const cap of visibleCaptions) {
      const isActive = currentMs >= cap.startMs && currentMs < cap.endMs;
      const isPast = currentMs >= cap.endMs;

      renderCaptionWord(ctx, cap, isActive, isPast, style, layout, img.width, img.height);
    }

    // Save rendered frame
    const buffer = canvas.toBuffer('image/png');
    await fs.writeFile(framePath, buffer);
  }

  // ── STEP 3: Encode final video ─────────────────────────
  await execAsync(`
    ffmpeg -i ${uploadDir}/video.mp4 -i ${renderDir}/frame_%06d.png 
    -filter_complex "[0:a]acopy[audio];[1:v]format=yuv420p[video]" 
    -map "[video]" -map "[audio]" 
    -c:v libx264 -preset fast -crf 23 
    -c:a aac -b:a 128k 
    -movflags +faststart 
    -y ${outputPath}
  `);

  // Cleanup temp frames
  await fs.rm(renderDir, { recursive: true });

  return NextResponse.json({ 
    downloadUrl: `/api/download/${id}`,
    filename: `captioned_${id}.mp4`
  });
}

// Helper: render single word with stroke, fill, shadow
function renderCaptionWord(
  ctx: CanvasRenderingContext2D,
  caption: Caption,
  isActive: boolean,
  isPast: boolean,
  style: CaptionStyle,
  layout: CaptionLayout,
  videoW: number,
  videoH: number
) {
  const text = style.uppercase ? caption.text.toUpperCase() : caption.text;
  const fontSize = (videoH * style.fontSize) / 1080; // scale relative to 1080p

  ctx.font = `900 ${fontSize}px "${style.fontFamily}"`;
  ctx.textAlign = 'center';
  ctx.textBaseline = 'middle';

  // Calculate position
  const x = videoW / 2;
  let y = videoH * 0.85; // default bottom
  if (layout.position === 'top') y = videoH * 0.15;
  if (layout.position === 'center') y = videoH * 0.5;
  y += layout.yOffset;

  // Animation state
  let scale = 1;
  let opacity = 1;

  if (isActive) {
    // Bounce in effect — calculate based on time into word
    const progress = Math.min(1, (Date.now() - caption.startMs) / 300);
    scale = 0.5 + (0.5 * easeOutElastic(progress));
    if (progress > 0.8) scale = 1 + 0.05 * Math.sin(Date.now() / 150);
  } else if (isPast) {
    scale = 0.95;
    opacity = 0.5;
  }

  ctx.save();
  ctx.translate(x, y);
  ctx.scale(scale, scale);
  ctx.globalAlpha = opacity;

  // Shadow
  if (style.shadow) {
    ctx.shadowColor = style.shadow.color;
    ctx.shadowBlur = style.shadow.blur;
    ctx.shadowOffsetX = style.shadow.offsetX;
    ctx.shadowOffsetY = style.shadow.offsetY;
  }

  // Stroke (outline)
  ctx.strokeStyle = style.strokeColor;
  ctx.lineWidth = style.strokeWidth * (videoH / 1080);
  ctx.lineJoin = 'round';
  ctx.strokeText(text, 0, 0);

  // Fill
  ctx.fillStyle = style.fillColor;
  ctx.fillText(text, 0, 0);

  // Highlight (top edge)
  if (style.highlightColor) {
    ctx.fillStyle = style.highlightColor;
    ctx.globalAlpha = opacity * 0.3;
    ctx.fillText(text, 0, -2);
  }

  ctx.restore();
}

// Elastic ease-out for bounce
function easeOutElastic(x: number): number {
  const c4 = (2 * Math.PI) / 3;
  return x === 0 ? 0 : x === 1 ? 1 : Math.pow(2, -10 * x) * Math.sin((x * 10 - 0.75) * c4) + 1;
}
```

---

## 8. PROJECT STRUCTURE

```
my-caption-app/
├── app/
│   ├── page.tsx                    # Upload page
│   ├── editor/
│   │   └── page.tsx                # Editor + preview
│   ├── export/
│   │   └── page.tsx                # Render progress + download
│   ├── api/
│   │   ├── upload/
│   │   │   └── route.ts            # POST: save uploaded video
│   │   ├── render/
│   │   │   └── route.ts            # POST: FFmpeg render pipeline
│   │   ├── progress/
│   │   │   └── route.ts            # GET: SSE or poll render progress
│   │   └── download/
│   │       └── [id]/
│   │           └── route.ts        # GET: stream MP4 file
│   ├── layout.tsx
│   └── globals.css
├── components/
│   ├── ui/                         # shadcn components
│   ├── video-upload.tsx            # Drag & drop zone
│   ├── video-preview.tsx           # Video + caption overlay
│   ├── caption-word.tsx            # Animated word (Framer Motion)
│   ├── caption-editor.tsx          # Text input + SRT upload
│   ├── style-panel.tsx             # Font, color, size controls
│   ├── timeline.tsx                # Visual word timeline (v2)
│   └── export-progress.tsx         # Progress bar
├── lib/
│   ├── utils.ts                    # cn(), helpers
│   ├── ffmpeg.ts                   # FFmpeg command builders
│   ├── caption-renderer.ts         # Canvas rendering logic
│   ├── srt-parser.ts               # Parse .srt files
│   └── timing.ts                   # Auto-sync algorithms
├── types/
│   └── index.ts                    # TypeScript interfaces
├── public/
│   └── fonts/                      # Impact, Oswald, Anton (WOFF2)
├── next.config.js
├── tailwind.config.ts
└── package.json
```

---

## 9. KEY DEPENDENCIES

```json
{
  "dependencies": {
    "next": "^15.0.0",
    "react": "^19.0.0",
    "react-dom": "^19.0.0",
    "typescript": "^5.5.0",
    "tailwindcss": "^4.0.0",
    "@tailwindcss/postcss": "^4.0.0",
    "framer-motion": "^11.0.0",
    "canvas": "^2.11.0",
    "ffmpeg-static": "^5.2.0",
    "ffprobe-static": "^3.1.0",
    "uuid": "^9.0.0",
    "tmp": "^0.2.0",
    "class-variance-authority": "^0.7.0",
    "clsx": "^2.1.0",
    "tailwind-merge": "^2.3.0"
  },
  "devDependencies": {
    "@types/node": "^20.0.0",
    "@types/react": "^19.0.0",
    "@types/uuid": "^9.0.0",
    "@types/tmp": "^0.2.0"
  }
}
```

---

## 10. ENVIRONMENT SETUP

```bash
# .env.local
# No API keys needed for MVP!
# Just make sure FFmpeg is installed:

# macOS
brew install ffmpeg

# Ubuntu/Debian
sudo apt-get install ffmpeg

# Or use ffmpeg-static (bundled)
```

---

## 11. CRITICAL IMPLEMENTATION NOTES

### 11.1 Font Loading for Canvas
```typescript
// lib/fonts.ts
import { registerFont } from 'canvas';
import path from 'path';

export function loadFonts() {
  registerFont(path.join(process.cwd(), 'public/fonts/Impact.ttf'), {
    family: 'Impact'
  });
  registerFont(path.join(process.cwd(), 'public/fonts/Oswald-Bold.ttf'), {
    family: 'Oswald'
  });
  // ... etc
}
```

### 11.2 Temp File Cleanup
```typescript
// Run periodically or on shutdown
import tmp from 'tmp';

// Auto-cleanup on app exit
tmp.setGracefulCleanup();

// Or explicit cleanup after render
await fs.rm(`/tmp/uploads/${id}`, { recursive: true, force: true });
```

### 11.3 Progress Tracking (SSE)
```typescript
// app/api/render/route.ts (streaming progress)

export async function POST(req: NextRequest) {
  const encoder = new TextEncoder();
  const stream = new ReadableStream({
    async start(controller) {
      for (let i = 0; i < totalFrames; i++) {
        // render frame i...
        const progress = { percent: (i / totalFrames) * 100 };
        controller.enqueue(encoder.encode(`data: ${JSON.stringify(progress)}\n\n`));
      }
      controller.close();
    }
  });

  return new Response(stream, {
    headers: { 'Content-Type': 'text/event-stream' }
  });
}
```

### 11.4 Performance Optimization
- Extract frames at target resolution (not 4K if exporting 1080p)
- Use `ffmpeg -threads 4` for multi-core encoding
- Render frames in batches (100 at a time) to avoid memory bloat
- For long videos: stream process instead of loading all frames to RAM

---

## 12. QUICK START CHECKLIST

```
□ npm create next-app@latest my-caption-app --typescript --tailwind --app
□ cd my-caption-app
□ npx shadcn@latest init
□ npm install framer-motion canvas ffmpeg-static ffprobe-static uuid tmp
□ npm install -D @types/uuid @types/tmp
□ mkdir -p public/fonts && copy your fonts
□ brew install ffmpeg (or apt-get)
□ Implement /app/page.tsx (upload)
□ Implement /app/editor/page.tsx (editor + preview)
□ Implement /app/api/upload/route.ts
□ Implement /app/api/render/route.ts
□ Test with a 30s sample video
□ Polish styles, add presets
□ Done!
```

---

## 13. ESTIMATED BUILD TIME

| Phase | Time | Deliverable |
|-------|------|-------------|
| Setup + upload | 2 hrs | Video uploads, metadata extraction |
| Editor UI | 4 hrs | Split pane, controls, state management |
| Preview (browser) | 3 hrs | Framer Motion captions synced to video |
| Server render | 6 hrs | FFmpeg pipeline, Canvas rendering |
| Export + download | 2 hrs | Progress tracking, file serving |
| Polish | 3 hrs | Presets, error handling, cleanup |
| **Total** | **~20 hrs** | **Working MVP** |

---

*Built for personal use. No auth, no payments, no analytics. Just you and your captions.*