docs/inngest-workflows.md

Vault - Inngest Workflows

Generated: 2026-02-11
Framework: Inngest 3.49.1

---

Overview

Inngest provides durable workflow orchestration for long-running background tasks. All workflows are defined in src/inngest/.

Client Configuration

File: src/inngest/client.ts

import { Inngest } from 'inngest';

export const inngest = new Inngest({
  id: 'vault',
  name: 'Vault AI Discovery',
  eventKey: process.env.INNGEST_EVENT_KEY,
});

Handler Registration

File: src/app/api/inngest/route.ts

import { serve } from 'inngest/next';
import { inngest } from '@/inngest/client';
import { scanVideoArchive } from '@/inngest/functions/scan-video-archive';
import { detectObjects } from '@/inngest/functions/detect-objects';
import { matchMarketplace } from '@/inngest/functions/match-marketplace';
import { monitorLinkHealth, checkSingleLinkHealth } from '@/inngest/functions/monitor-link-health';
import { warmFeaturedCreatorsCache } from '@/inngest/functions/warm-featured-creators-cache';
import { healSocialMetadata } from '@/inngest/functions/social-healing';

export const { GET, POST, PUT } = serve({
  client: inngest,
  functions: [
    scanVideoArchive,
    detectObjects,
    matchMarketplace,
    monitorLinkHealth,
    checkSingleLinkHealth,
    warmFeaturedCreatorsCache,
    healSocialMetadata,
  ],
  signingKey: process.env.INNGEST_SIGNING_KEY,
});

---

Workflow: Scan Video Archive

Function ID: scan-video-archive
File: src/inngest/functions/scan-video-archive.ts
Event: youtube/video-archive.scan

Purpose

Scans a YouTube channel's video archive, fetching metadata and queuing videos for object detection.

Event Payload

interface ScanVideoArchiveEvent {
  name: 'youtube/video-archive.scan';
  data: {
    channelId: string;  // Internal DB channel ID
    userId: string;     // User who initiated scan
    scanJobId: string;  // Progress tracking ID
  };
}

Workflow Steps

┌─────────────────────────────────────────────────────────────┐
│                    scan-video-archive                        │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  Step 1: validate-channel                                    │
│  ├── Verify channel ownership                                │
│  ├── Update syncStatus to 'syncing'                          │
│  └── Return channel data                                     │
│                                                              │
│  Step 2: fetch-refresh-token                                 │
│  └── Get OAuth refresh token from accounts table             │
│                                                              │
│  Step 3: refresh-access-token                                │
│  └── Exchange refresh token for new access token             │
│                                                              │
│  Step 4: get-uploads-playlist                                │
│  └── Fetch channel's uploads playlist ID                     │
│                                                              │
│  Step 5: fetch-video-ids (paginated)                         │
│  └── Get all video IDs from playlist                         │
│                                                              │
│  Step 6: fetch-video-metadata (batched)                      │
│  └── Get metadata for each video                             │
│                                                              │
│  Step 7: store-videos                                        │
│  └── Insert/update videos in database                        │
│                                                              │
│  Step 8: trigger-detection (per video)                       │
│  └── Send 'youtube/video.detect-objects' event               │
│                                                              │
│  Step 9: update-scan-job                                     │
│  └── Mark scan job as completed                              │
│                                                              │
└─────────────────────────────────────────────────────────────┘

Error Handling

• NonRetriableError: Channel not found, access denied
• RetryAfterError: YouTube API rate limits
• onFailure: Update scan job status to 'failed', set channel syncStatus to 'errored'

---

Workflow: Detect Objects

Function ID: detect-objects
File: src/inngest/functions/detect-objects.ts
Event: youtube/video.detect-objects

Purpose

Extracts frames from a video and runs AI object detection on each frame.

Event Payload

interface DetectObjectsEvent {
  name: 'youtube/video.detect-objects';
  data: {
    videoId: string;   // Internal DB video ID
    videoUrl: string;  // Video URL for processing
  };
}

Workflow Steps

┌─────────────────────────────────────────────────────────────┐
│                      detect-objects                          │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  Step 1: update-scan-status-started                          │
│  └── Set video scanStatus to 'in_progress'                   │
│                                                              │
│  Step 2: analyze-video-content                               │
│  ├── Calculate adaptive frame interval                       │
│  │   ├── 0-2 min: every 5 seconds                            │
│  │   ├── 2-5 min: every 10 seconds                           │
│  │   ├── 5-10 min: every 15 seconds                          │
│  │   └── 10+ min: every 30 seconds                           │
│  ├── Extract frames at intervals (FFmpeg)                    │
│  ├── Process frames in batches of 3                          │
│  ├── Run AI detection on each frame                          │
│  ├── Generate thumbnails for detections                      │
│  └── Return all detection results                            │
│                                                              │
│  Step 3: store-detections                                    │
│  └── Insert detected objects into database                   │
│                                                              │
│  Step 4: trigger-marketplace-matching                        │
│  └── Send 'discovery/objects.match-marketplace' event        │
│                                                              │
│  Step 5: update-scan-status-completed                        │
│  └── Set video scanStatus to 'awaiting_approval'             │
│                                                              │
└─────────────────────────────────────────────────────────────┘

AI Vision Providers

Configured via VISION_PROVIDER environment variable:

Provider	Model	Best For
gemini	Gemini Flash	Complex scenes, good accuracy
huggingface	DETR	Specialized object detection, better free tier

Error Handling

• VideoUnavailableError: Video is private or deleted
• VisionRateLimitError: AI provider rate limits
• NonRetriableError: Missing videoId or videoUrl

---

Workflow: Match Marketplace

Function ID: match-marketplace
File: src/inngest/functions/match-marketplace.ts
Event: discovery/objects.match-marketplace

Purpose

Searches Amazon, eBay, and Etsy for products matching detected objects.

Event Payload

interface MatchMarketplaceEvent {
  name: 'discovery/objects.match-marketplace';
  data: {
    detectedObjectId?: string;      // Single ID (legacy)
    detectedObjectIds?: string[];   // Batch IDs (preferred)
  };
}

Workflow Steps

┌─────────────────────────────────────────────────────────────┐
│                    match-marketplace                          │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  For each detected object:                                   │
│                                                              │
│  Step 1: fetch-object-{id}                                   │
│  └── Get detection details from database                     │
│                                                              │
│  Step 2: search-amazon-{id}                                  │
│  ├── Check cache for existing matches                        │
│  ├── If not cached, search Amazon PA-API                     │
│  ├── Cache results                                           │
│  └── Return matches                                          │
│                                                              │
│  Step 3: search-ebay-{id}                                    │
│  ├── Check cache for existing matches                        │
│  ├── If not cached, search eBay Finding API                  │
│  ├── Cache results                                           │
│  └── Return matches                                          │
│                                                              │
│  Step 4: search-etsy-{id}                                    │
│  ├── Check cache for existing matches                        │
│  ├── If not cached, search Etsy Open API                     │
│  ├── Cache results                                           │
│  └── Return matches                                          │
│                                                              │
│  Step 5: store-matches-{id}                                  │
│  └── Insert marketplace matches into database                │
│                                                              │
└─────────────────────────────────────────────────────────────┘

Caching Strategy

• Cache Key: {objectId}:{marketplace}
• TTL: 24 hours
• Storage: Upstash Redis

Rate Limit Handling

• MarketplaceRateLimitError: Triggers RetryAfterError
• Automatic retry after specified delay

---

Workflow: Monitor Link Health

Function ID: monitor-link-health
File: src/inngest/functions/monitor-link-health.ts
Event: marketplace/links.health-check

Purpose

Periodically checks affiliate links for availability and updates status.

Event Payload

interface MonitorLinkHealthEvent {
  name: 'marketplace/links.health-check';
  data: {
    matchId?: string;       // Single match check
    checkAll?: boolean;     // Check all active links
  };
}

Workflow Steps

┌─────────────────────────────────────────────────────────────┐
│                   monitor-link-health                        │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  Step 1: fetch-links-to-check                                │
│  └── Get links with status 'ACTIVE' or 'CHECKING'            │
│                                                              │
│  For each link:                                              │
│                                                              │
│  Step 2: check-link-{id}                                     │
│  ├── HTTP HEAD request to affiliate URL                      │
│  ├── Record HTTP status                                      │
│  └── Return status                                           │
│                                                              │
│  Step 3: update-link-status-{id}                             │
│  ├── If 200: Set status 'ACTIVE'                             │
│  ├── If 404: Set status 'BROKEN'                             │
│  └── Increment checkAttempts                                 │
│                                                              │
└─────────────────────────────────────────────────────────────┘

Link Status Transitions

ACTIVE ──check──> CHECKING ──success──> ACTIVE
                     │
                     └──failure──> BROKEN

---

Workflow: Warm Featured Creators Cache

Function ID: warm-featured-creators-cache
File: src/inngest/functions/warm-featured-creators-cache.ts
Event: cache/warm-featured-creators

Purpose

Pre-warms cache for featured creators on the homepage.

Workflow Steps

1. Fetch featured creators from database
2. For each creator, fetch their vault data
3. Store in Redis cache with TTL

---

Workflow: Heal Social Metadata

Function ID: heal-social-metadata
File: src/inngest/functions/social-healing.ts
Event: social/metadata.heal

Purpose

Repairs inconsistent platform metadata for multi-platform content.

Workflow Steps

1. Find videos with platform mismatches
2. Update platform field based on video ID patterns
3. Update channel platform associations

---

Error Handling Patterns

NonRetriableError

For errors that should not be retried:

import { NonRetriableError } from 'inngest';

throw new NonRetriableError('Channel not found or access denied');

RetryAfterError

For rate-limited resources:

import { RetryAfterError } from 'inngest';

throw new RetryAfterError('Amazon rate limit exceeded', '60s');

Sentry Integration

All functions capture errors with Sentry:

import * as Sentry from '@sentry/nextjs';

onFailure: async ({ event, error }) => {
  Sentry.captureException(error, {
    tags: {
      source: 'inngest',
      function: 'function-name',
      type: 'function_failure',
    },
    extra: { event: event.data },
  });
}

---

Development & Testing

Local Development

1. Start Inngest Dev Server:

   npx inngest-cli dev
   

2. Application connects to dev server automatically in development mode.

Manual Triggers

Use the manual trigger endpoint for testing:

curl -X POST http://localhost:3000/api/manual-trigger \
  -H "Content-Type: application/json" \
  -d '{"function": "scan-video-archive", "data": {...}}'

Monitoring

• Inngest Cloud Dashboard: View function runs, retries, errors
• Sentry: Error tracking with context
• Logs: Console output captured in Inngest dashboard

---

Related Documentation

• API Contracts - Server actions and endpoints
• Data Models - Database schema
• Development Guide - Setup instructions