Spaces:

dvijaykrishnan
/

vault-video-processor

Runtime error

File size: 14,076 Bytes

d03d74d

# Vault - Project Overview

**Generated:** 2026-02-11  
**Project Type:** SaaS / Web Application  
**Domain:** E-Commerce / Creator Economy  
**Complexity:** Medium-High

---

## Executive Summary

Vault is an AI-powered e-commerce platform that transforms YouTube video archives into shoppable experiences. The platform automatically detects products featured in videos, matches them to marketplace listings (Amazon, eBay, Etsy), and creates passive revenue streams for content creators through affiliate marketing.

### Core Value Proposition

- **For Creators:** Monetize historical video content without manual tagging
- **For Viewers:** Discover and purchase products from trusted creators
- **For Marketplaces:** High-intent traffic from engaged audiences

---

## Business Goals

### Success Criteria

| Metric | Target |
|--------|--------|
| First affiliate revenue (video >12 months old) | Within 7 days of archive activation |
| Viewer product discovery | Under 3 clicks to find specific item |
| Creator passive revenue (Standard tier) | >$100/month average (12-month goal) |
| AI matching accuracy | >85% precision |
| Vault grid load time | <1.5 seconds on 4G |
| Archive processing | 1,000 videos within 48 hours |

### Innovation: "Lost Intent" Capture

Vault captures unmet demand for discontinued or out-of-stock items mentioned in historical media. When products are unavailable, viewers can click "I want this" to join a waitlist, creating valuable demand data for:
- Creators (sourcing opportunities)
- Resellers (market insights)
- Brands (demand signals)

---

## Architecture Overview

### High-Level Architecture

```
┌─────────────────────────────────────────────────────────────────┐
│                        VAULT PLATFORM                           │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────┐        │
│  │   Vercel    │    │   Railway   │    │   Supabase  │        │
│  │  (Frontend) │    │  (Workers)  │    │  (Database) │        │
│  └──────┬──────┘    └──────┬──────┘    └──────┬──────┘        │
│         │                  │                  │                │
│         └──────────────────┼──────────────────┘                │
│                            │                                   │
│  ┌─────────────────────────┴─────────────────────────┐        │
│  │                   Inngest                          │        │
│  │         (Durable Workflow Orchestration)           │        │
│  └─────────────────────────┬─────────────────────────┘        │
│                            │                                   │
│  ┌─────────────────────────┴─────────────────────────┐        │
│  │              External Integrations                 │        │
│  │  YouTube │ Amazon │ eBay │ Etsy │ Gemini │ Redis  │        │
│  └───────────────────────────────────────────────────┘        │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘
```

### Technology Decisions

| Decision | Choice | Rationale |
|----------|--------|-----------|
| Framework | Next.js 15 (App Router) | SSR for SEO, Server Actions for mutations |
| Database | Supabase Postgres | Managed Postgres with built-in auth options |
| ORM | Drizzle | Type-safe, lightweight, migration support |
| Background Jobs | Inngest | Durable workflows, automatic retries, observability |
| Caching | Upstash Redis | TTL-based caching, serverless-friendly |
| Auth | Better Auth | Next.js 15 App Router compatibility, YouTube OAuth |
| AI Vision | Gemini + HuggingFace | Multi-provider flexibility, cost optimization |
| Deployment | Vercel + Railway | Hybrid: Edge frontend + persistent workers |

---

## Feature Modules

### 1. Discovery Pipeline (`src/features/discovery/`)

**Purpose:** AI-powered video scanning and object detection

**Components:**
- YouTube OAuth integration
- Video archive scanning (Inngest workflow)
- Frame extraction (FFmpeg)
- AI vision analysis (Gemini/HuggingFace)
- Thumbnail storage (Supabase Storage)

**Key Files:**
- [`src/inngest/functions/scan-video-archive.ts`](../src/inngest/functions/scan-video-archive.ts) - Archive scanning workflow
- [`src/inngest/functions/detect-objects.ts`](../src/inngest/functions/detect-objects.ts) - Object detection workflow
- [`src/features/discovery/services/ai-vision.service.ts`](../src/features/discovery/services/ai-vision.service.ts) - Vision provider abstraction

### 2. Marketplace Integration (`src/features/marketplace/`)

**Purpose:** Product matching and affiliate link generation

**Supported Marketplaces:**
- Amazon (PA-API 5.0)
- eBay (Finding API)
- Etsy (Open API v3)

**Key Files:**
- [`src/features/marketplace/services/amazon.service.ts`](../src/features/marketplace/services/amazon.service.ts)
- [`src/features/marketplace/services/ebay.service.ts`](../src/features/marketplace/services/ebay.service.ts)
- [`src/features/marketplace/services/etsy.service.ts`](../src/features/marketplace/services/etsy.service.ts)
- [`src/inngest/functions/match-marketplace.ts`](../src/inngest/functions/match-marketplace.ts) - Matching workflow

### 3. Vault Experience (`src/features/vault/`)

**Purpose:** Public-facing product discovery interface

**Features:**
- Mobile-first responsive grid
- Category filtering
- Product search
- Affiliate redirect tracking
- Interest capture for out-of-stock items

**Key Files:**
- [`src/features/vault/services/vault.service.ts`](../src/features/vault/services/vault.service.ts) - Data fetching with caching
- [`src/features/vault/components/vault-grid.tsx`](../src/features/vault/components/vault-grid.tsx) - Main grid component
- [`src/features/vault/components/product-card.tsx`](../src/features/vault/components/product-card.tsx) - Product display

### 4. Moderation System (`src/features/moderation/`)

**Purpose:** Creator review and curation of AI detections

**Features:**
- Approval/rejection workflow
- Bulk operations
- Detection editing (name, category, links)
- Manual product addition
- Admin moderation for high-ambiguity items

**Key Files:**
- [`src/features/moderation/services/moderation.service.ts`](../src/features/moderation/services/moderation.service.ts)
- [`src/features/moderation/components/moderation-queue.tsx`](../src/features/moderation/components/moderation-queue.tsx)

### 5. Interest Capture (`src/features/interest/`)

**Purpose:** "Lost Intent" demand capture

**Features:**
- Email waitlist for unavailable items
- GDPR/CCPA compliant consent
- Duplicate prevention (email hashing)
- Notification system

**Key Files:**
- [`src/features/interest/components/interest-capture-modal.tsx`](../src/features/interest/components/interest-capture-modal.tsx)

### 6. Analytics (`src/features/analytics/`)

**Purpose:** Creator insights and metrics

**Features:**
- Click tracking
- Interest pledge counts
- Revenue attribution
- Key metrics dashboard

**Key Files:**
- [`src/features/analytics/components/key-metrics-row.tsx`](../src/features/analytics/components/key-metrics-row.tsx)

---

## User Roles

| Role | Description | Key Permissions |
|------|-------------|-----------------|
| **Anonymous Viewer** | Public visitor | Browse vaults, search products, click affiliate links |
| **Registered Creator** | Content creator | Connect YouTube, moderate detections, view analytics |
| **Admin** | Platform operator | All creator permissions + admin moderation, user management |

---

## Data Flow

### Video Processing Pipeline

```
YouTube Video
     │
     ▼
┌─────────────┐
│   Scan      │ ◄── Inngest: scan-video-archive
│   Archive   │
└─────┬───────┘
      │
      ▼
┌─────────────┐
│   Extract   │ ◄── FFmpeg frame extraction
│   Frames    │
└─────┬───────┘
      │
      ▼
┌─────────────┐
│   Detect    │ ◄── Inngest: detect-objects
│   Objects   │     (Gemini/HuggingFace)
└─────┬───────┘
      │
      ▼
┌─────────────┐
│   Match     │ ◄── Inngest: match-marketplace
│  Products   │     (Amazon/eBay/Etsy)
└─────┬───────┘
      │
      ▼
┌─────────────┐
│  Moderate   │ ◄── Creator review
│   Queue     │
└─────┬───────┘
      │
      ▼
┌─────────────┐
│   Public    │ ◄── Vault grid display
│   Vault     │
└─────────────┘
```

---

## Security & Compliance

### Authentication
- Google OAuth with YouTube readonly scope
- Better Auth session management
- Secure cookie handling (production)

### Data Protection
- PII encryption (emails) at rest
- AES-256 encrypted email storage
- SHA-256 hashing for deduplication
- Unsubscribe tokens for GDPR compliance

### API Security
- YouTube API ToS compliance
- Marketplace API rate limit handling
- Sentry error tracking
- Input validation on all server actions

---

## Performance Targets

| Metric | Target | Strategy |
|--------|--------|----------|
| LTI (Load-to-Interactivity) | <1.5s on 4G | SSR + ISR + Edge caching |
| Search latency | <300ms | Redis caching, indexed queries |
| Traffic spike handling | 10x surge <10% degradation | CDN, connection pooling |
| Uptime | 99.9% | Multi-region deployment, health checks |

---

## Deployment Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                      PRODUCTION                              │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  ┌──────────────────┐         ┌──────────────────┐         │
│  │     Vercel       │         │     Railway      │         │
│  │   (Edge/Server)  │         │   (Background)   │         │
│  │                  │         │                  │         │
│  │  • Next.js App   │         │  • Inngest       │         │
│  │  • SSR/ISR       │         │    Functions     │         │
│  │  • API Routes    │         │  • FFmpeg        │         │
│  │  • Server        │         │    Processing    │         │
│  │    Actions       │         │                  │         │
│  └────────┬─────────┘         └────────┬─────────┘         │
│           │                            │                    │
│           └────────────┬───────────────┘                    │
│                        │                                    │
│  ┌─────────────────────┴─────────────────────┐             │
│  │              External Services              │             │
│  │                                             │             │
│  │  • Supabase (Postgres + Storage)           │             │
│  │  • Upstash Redis (Caching)                 │             │
│  │  • Inngest Cloud (Workflow orchestration)  │             │
│  │  • Sentry (Error monitoring)               │             │
│  │  • Google Cloud (YouTube API, Gemini)      │             │
│  └─────────────────────────────────────────────┘             │
│                                                              │
└─────────────────────────────────────────────────────────────┘
```

---

## Related Documentation

- [Source Tree](./source-tree.md) - Detailed directory structure
- [Data Models](./data-models.md) - Database schema and relationships
- [API Contracts](./api-contracts.md) - Endpoints and server actions
- [Inngest Workflows](./inngest-workflows.md) - Background job documentation
- [Development Guide](./development-guide.md) - Setup and deployment