docs/inngest-workflows.md

# Vault - Inngest Workflows

**Generated:** 2026-02-11  
**Framework:** Inngest 3.49.1

---

## Overview

Inngest provides durable workflow orchestration for long-running background tasks. All workflows are defined in [`src/inngest/`](../src/inngest/).

### Client Configuration

**File:** [`src/inngest/client.ts`](../src/inngest/client.ts)

```typescript
import { Inngest } from 'inngest';

export const inngest = new Inngest({
  id: 'vault',
  name: 'Vault AI Discovery',
  eventKey: process.env.INNGEST_EVENT_KEY,
});
```

### Handler Registration

**File:** [`src/app/api/inngest/route.ts`](../src/app/api/inngest/route.ts)

```typescript
import { serve } from 'inngest/next';
import { inngest } from '@/inngest/client';
import { scanVideoArchive } from '@/inngest/functions/scan-video-archive';
import { detectObjects } from '@/inngest/functions/detect-objects';
import { matchMarketplace } from '@/inngest/functions/match-marketplace';
import { monitorLinkHealth, checkSingleLinkHealth } from '@/inngest/functions/monitor-link-health';
import { warmFeaturedCreatorsCache } from '@/inngest/functions/warm-featured-creators-cache';
import { healSocialMetadata } from '@/inngest/functions/social-healing';

export const { GET, POST, PUT } = serve({
  client: inngest,
  functions: [
    scanVideoArchive,
    detectObjects,
    matchMarketplace,
    monitorLinkHealth,
    checkSingleLinkHealth,
    warmFeaturedCreatorsCache,
    healSocialMetadata,
  ],
  signingKey: process.env.INNGEST_SIGNING_KEY,
});
```

---

## Workflow: Scan Video Archive

**Function ID:** `scan-video-archive`  
**File:** [`src/inngest/functions/scan-video-archive.ts`](../src/inngest/functions/scan-video-archive.ts)  
**Event:** `youtube/video-archive.scan`

### Purpose

Scans a YouTube channel's video archive, fetching metadata and queuing videos for object detection.

### Event Payload

```typescript
interface ScanVideoArchiveEvent {
  name: 'youtube/video-archive.scan';
  data: {
    channelId: string;  // Internal DB channel ID
    userId: string;     // User who initiated scan
    scanJobId: string;  // Progress tracking ID
  };
}
```

### Workflow Steps

```
┌─────────────────────────────────────────────────────────────┐
│                    scan-video-archive                        │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  Step 1: validate-channel                                    │
│  ├── Verify channel ownership                                │
│  ├── Update syncStatus to 'syncing'                          │
│  └── Return channel data                                     │
│                                                              │
│  Step 2: fetch-refresh-token                                 │
│  └── Get OAuth refresh token from accounts table             │
│                                                              │
│  Step 3: refresh-access-token                                │
│  └── Exchange refresh token for new access token             │
│                                                              │
│  Step 4: get-uploads-playlist                                │
│  └── Fetch channel's uploads playlist ID                     │
│                                                              │
│  Step 5: fetch-video-ids (paginated)                         │
│  └── Get all video IDs from playlist                         │
│                                                              │
│  Step 6: fetch-video-metadata (batched)                      │
│  └── Get metadata for each video                             │
│                                                              │
│  Step 7: store-videos                                        │
│  └── Insert/update videos in database                        │
│                                                              │
│  Step 8: trigger-detection (per video)                       │
│  └── Send 'youtube/video.detect-objects' event               │
│                                                              │
│  Step 9: update-scan-job                                     │
│  └── Mark scan job as completed                              │
│                                                              │
└─────────────────────────────────────────────────────────────┘
```

### Error Handling

- **NonRetriableError:** Channel not found, access denied
- **RetryAfterError:** YouTube API rate limits
- **onFailure:** Update scan job status to 'failed', set channel syncStatus to 'errored'

---

## Workflow: Detect Objects

**Function ID:** `detect-objects`  
**File:** [`src/inngest/functions/detect-objects.ts`](../src/inngest/functions/detect-objects.ts)  
**Event:** `youtube/video.detect-objects`

### Purpose

Extracts frames from a video and runs AI object detection on each frame.

### Event Payload

```typescript
interface DetectObjectsEvent {
  name: 'youtube/video.detect-objects';
  data: {
    videoId: string;   // Internal DB video ID
    videoUrl: string;  // Video URL for processing
  };
}
```

### Workflow Steps

```
┌─────────────────────────────────────────────────────────────┐
│                      detect-objects                          │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  Step 1: update-scan-status-started                          │
│  └── Set video scanStatus to 'in_progress'                   │
│                                                              │
│  Step 2: analyze-video-content                               │
│  ├── Calculate adaptive frame interval                       │
│  │   ├── 0-2 min: every 5 seconds                            │
│  │   ├── 2-5 min: every 10 seconds                           │
│  │   ├── 5-10 min: every 15 seconds                          │
│  │   └── 10+ min: every 30 seconds                           │
│  ├── Extract frames at intervals (FFmpeg)                    │
│  ├── Process frames in batches of 3                          │
│  ├── Run AI detection on each frame                          │
│  ├── Generate thumbnails for detections                      │
│  └── Return all detection results                            │
│                                                              │
│  Step 3: store-detections                                    │
│  └── Insert detected objects into database                   │
│                                                              │
│  Step 4: trigger-marketplace-matching                        │
│  └── Send 'discovery/objects.match-marketplace' event        │
│                                                              │
│  Step 5: update-scan-status-completed                        │
│  └── Set video scanStatus to 'awaiting_approval'             │
│                                                              │
└─────────────────────────────────────────────────────────────┘
```

### AI Vision Providers

Configured via `VISION_PROVIDER` environment variable:

| Provider | Model | Best For |
|----------|-------|----------|
| `gemini` | Gemini Flash | Complex scenes, good accuracy |
| `huggingface` | DETR | Specialized object detection, better free tier |

### Error Handling

- **VideoUnavailableError:** Video is private or deleted
- **VisionRateLimitError:** AI provider rate limits
- **NonRetriableError:** Missing videoId or videoUrl

---

## Workflow: Match Marketplace

**Function ID:** `match-marketplace`  
**File:** [`src/inngest/functions/match-marketplace.ts`](../src/inngest/functions/match-marketplace.ts)  
**Event:** `discovery/objects.match-marketplace`

### Purpose

Searches Amazon, eBay, and Etsy for products matching detected objects.

### Event Payload

```typescript
interface MatchMarketplaceEvent {
  name: 'discovery/objects.match-marketplace';
  data: {
    detectedObjectId?: string;      // Single ID (legacy)
    detectedObjectIds?: string[];   // Batch IDs (preferred)
  };
}
```

### Workflow Steps

```
┌─────────────────────────────────────────────────────────────┐
│                    match-marketplace                          │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  For each detected object:                                   │
│                                                              │
│  Step 1: fetch-object-{id}                                   │
│  └── Get detection details from database                     │
│                                                              │
│  Step 2: search-amazon-{id}                                  │
│  ├── Check cache for existing matches                        │
│  ├── If not cached, search Amazon PA-API                     │
│  ├── Cache results                                           │
│  └── Return matches                                          │
│                                                              │
│  Step 3: search-ebay-{id}                                    │
│  ├── Check cache for existing matches                        │
│  ├── If not cached, search eBay Finding API                  │
│  ├── Cache results                                           │
│  └── Return matches                                          │
│                                                              │
│  Step 4: search-etsy-{id}                                    │
│  ├── Check cache for existing matches                        │
│  ├── If not cached, search Etsy Open API                     │
│  ├── Cache results                                           │
│  └── Return matches                                          │
│                                                              │
│  Step 5: store-matches-{id}                                  │
│  └── Insert marketplace matches into database                │
│                                                              │
└─────────────────────────────────────────────────────────────┘
```

### Caching Strategy

- **Cache Key:** `{objectId}:{marketplace}`
- **TTL:** 24 hours
- **Storage:** Upstash Redis

### Rate Limit Handling

- **MarketplaceRateLimitError:** Triggers `RetryAfterError`
- Automatic retry after specified delay

---

## Workflow: Monitor Link Health

**Function ID:** `monitor-link-health`  
**File:** [`src/inngest/functions/monitor-link-health.ts`](../src/inngest/functions/monitor-link-health.ts)  
**Event:** `marketplace/links.health-check`

### Purpose

Periodically checks affiliate links for availability and updates status.

### Event Payload

```typescript
interface MonitorLinkHealthEvent {
  name: 'marketplace/links.health-check';
  data: {
    matchId?: string;       // Single match check
    checkAll?: boolean;     // Check all active links
  };
}
```

### Workflow Steps

```
┌─────────────────────────────────────────────────────────────┐
│                   monitor-link-health                        │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  Step 1: fetch-links-to-check                                │
│  └── Get links with status 'ACTIVE' or 'CHECKING'            │
│                                                              │
│  For each link:                                              │
│                                                              │
│  Step 2: check-link-{id}                                     │
│  ├── HTTP HEAD request to affiliate URL                      │
│  ├── Record HTTP status                                      │
│  └── Return status                                           │
│                                                              │
│  Step 3: update-link-status-{id}                             │
│  ├── If 200: Set status 'ACTIVE'                             │
│  ├── If 404: Set status 'BROKEN'                             │
│  └── Increment checkAttempts                                 │
│                                                              │
└─────────────────────────────────────────────────────────────┘
```

### Link Status Transitions

```
ACTIVE ──check──> CHECKING ──success──> ACTIVE
                     │
                     └──failure──> BROKEN
```

---

## Workflow: Warm Featured Creators Cache

**Function ID:** `warm-featured-creators-cache`  
**File:** [`src/inngest/functions/warm-featured-creators-cache.ts`](../src/inngest/functions/warm-featured-creators-cache.ts)  
**Event:** `cache/warm-featured-creators`

### Purpose

Pre-warms cache for featured creators on the homepage.

### Workflow Steps

1. Fetch featured creators from database
2. For each creator, fetch their vault data
3. Store in Redis cache with TTL

---

## Workflow: Heal Social Metadata

**Function ID:** `heal-social-metadata`  
**File:** [`src/inngest/functions/social-healing.ts`](../src/inngest/functions/social-healing.ts)  
**Event:** `social/metadata.heal`

### Purpose

Repairs inconsistent platform metadata for multi-platform content.

### Workflow Steps

1. Find videos with platform mismatches
2. Update platform field based on video ID patterns
3. Update channel platform associations

---

## Error Handling Patterns

### NonRetriableError

For errors that should not be retried:

```typescript
import { NonRetriableError } from 'inngest';

throw new NonRetriableError('Channel not found or access denied');
```

### RetryAfterError

For rate-limited resources:

```typescript
import { RetryAfterError } from 'inngest';

throw new RetryAfterError('Amazon rate limit exceeded', '60s');
```

### Sentry Integration

All functions capture errors with Sentry:

```typescript
import * as Sentry from '@sentry/nextjs';

onFailure: async ({ event, error }) => {
  Sentry.captureException(error, {
    tags: {
      source: 'inngest',
      function: 'function-name',
      type: 'function_failure',
    },
    extra: { event: event.data },
  });
}
```

---

## Development & Testing

### Local Development

1. Start Inngest Dev Server:
   ```bash
   npx inngest-cli dev
   ```

2. Application connects to dev server automatically in development mode.

### Manual Triggers

Use the manual trigger endpoint for testing:

```bash
curl -X POST http://localhost:3000/api/manual-trigger \
  -H "Content-Type: application/json" \
  -d '{"function": "scan-video-archive", "data": {...}}'
```

### Monitoring

- **Inngest Cloud Dashboard:** View function runs, retries, errors
- **Sentry:** Error tracking with context
- **Logs:** Console output captured in Inngest dashboard

---

## Related Documentation

- [API Contracts](./api-contracts.md) - Server actions and endpoints
- [Data Models](./data-models.md) - Database schema
- [Development Guide](./development-guide.md) - Setup instructions