vault-video-processor / docs /development-guide.md
dvijaykrishnan's picture
docs: Introduce comprehensive technical documentation and update planning artifacts to reflect MVP completion and detailed FR implementation status.
d03d74d
|
Raw
History Blame Contribute Delete
8.87 kB
# Vault - Development Guide
**Generated:** 2026-02-11
**Node.js:** 20+
**Package Manager:** npm
---
## Quick Start
### Prerequisites
- Node.js 20+ installed
- npm or yarn package manager
- Supabase account (for database)
- Google Cloud project (for OAuth + YouTube API)
- Inngest account (for background jobs)
- Upstash account (for Redis caching)
### Installation
```bash
# Clone the repository
git clone <repository-url>
cd vault
# Install dependencies
npm install
# Copy environment variables
cp .env.example .env.local
# Edit .env.local with your credentials
```
### Development Server
```bash
# Start Next.js development server with Turbopack
npm run dev
# In a separate terminal, start Inngest dev server
npx inngest-cli dev
# Open http://localhost:3000
```
---
## Environment Variables
### Required Variables
```bash
# Database (Supabase Postgres)
DATABASE_URL=postgresql://...@aws-0-us-east-1.pooler.supabase.com:6543/postgres
DIRECT_URL=postgresql://...@aws-0-us-east-1.pooler.supabase.com:5432/postgres
# Authentication
BETTER_AUTH_SECRET=your-secret-key-min-32-chars
BETTER_AUTH_URL=http://localhost:3000
GOOGLE_CLIENT_ID=your-google-client-id
GOOGLE_CLIENT_SECRET=your-google-client-secret
# Inngest
INNGEST_EVENT_KEY=your-event-key
INNGEST_SIGNING_KEY=your-signing-key
# Supabase Storage
NEXT_PUBLIC_SUPABASE_URL=https://your-project.supabase.co
NEXT_PUBLIC_SUPABASE_ANON_KEY=your-anon-key
# Redis (Upstash)
UPSTASH_REDIS_REST_URL=https://your-redis.upstash.io
UPSTASH_REDIS_REST_TOKEN=your-token
# AI Vision
GEMINI_API_KEY=your-gemini-api-key
HUGGINGFACE_API_KEY=your-hf-api-key
VISION_PROVIDER=gemini # or 'huggingface'
# Marketplaces
AMAZON_ACCESS_KEY=your-amazon-access-key
AMAZON_SECRET_KEY=your-amazon-secret-key
AMAZON_AFFILIATE_TAG=your-affiliate-tag
EBAY_APP_ID=your-ebay-app-id
EBAY_CERT_ID=your-ebay-cert-id
EBAY_CAMPAIGN_ID=your-campaign-id
ETSY_API_KEY=your-etsy-api-key
# Monitoring
SENTRY_ORG=your-sentry-org
SENTRY_PROJECT=your-sentry-project
# Admin
ADMIN_EMAILS=admin@example.com,admin2@example.com
```
### Optional Variables
```bash
# CI/CD
CI=false # Set to true in CI environments
# Development
NODE_ENV=development
```
---
## Database Setup
### Drizzle ORM
Database schema is defined in [`src/lib/db/schema.ts`](../src/lib/db/schema.ts).
### Migrations
```bash
# Generate migration from schema changes
npx drizzle-kit generate
# Run migrations
npx drizzle-kit migrate
# Push schema directly (development)
npx drizzle-kit push
# Open Drizzle Studio
npx drizzle-kit studio
```
### Supabase Setup
1. Create a new Supabase project
2. Get connection strings from Project Settings > Database
3. Create storage bucket named `detected-objects` for thumbnails
4. Run migrations against the database
---
## Testing
### Test Framework
Vitest with React Testing Library.
```bash
# Run all tests
npm run test
# Run tests in watch mode
npx vitest
# Run tests with coverage
npx vitest --coverage
```
### Test Structure
Tests are co-located with source files:
```
src/
β”œβ”€β”€ features/
β”‚ └── vault/
β”‚ β”œβ”€β”€ services/
β”‚ β”‚ └── vault.service.ts
β”‚ β”‚ └── __tests__/
β”‚ β”‚ └── vault.service.test.ts
β”‚ └── components/
β”‚ └── __tests__/
β”‚ └── vault-grid.test.tsx
```
### Writing Tests
```typescript
// Service test example
import { describe, it, expect, vi } from 'vitest';
import { VaultService } from '../vault.service';
describe('VaultService', () => {
it('should return vault for valid creator slug', async () => {
const vault = await VaultService.getCreatorVault('test-creator');
expect(vault).toBeDefined();
expect(vault?.channel.channelName).toBe('Test Creator');
});
});
```
```typescript
// Component test example
import { render, screen } from '@testing-library/react';
import { describe, it, expect } from 'vitest';
import { ProductCard } from '../product-card';
describe('ProductCard', () => {
it('should render product name', () => {
render(<ProductCard {...mockProps} />);
expect(screen.getByText('Test Product')).toBeInTheDocument();
});
});
```
---
## Code Quality
### Linting
ESLint with Next.js recommended config.
```bash
# Run linter
npm run lint
# Fix linting issues
npm run lint -- --fix
```
### Pre-commit Hooks
Husky runs lint-staged on commit:
```json
{
"lint-staged": {
"*.{js,jsx,ts,tsx,mjs}": [
"eslint --fix"
]
}
}
```
### TypeScript
Strict mode enabled. Build will fail on type errors (configurable).
```bash
# Type check
npx tsc --noEmit
```
---
## Deployment
### Architecture
- **Frontend:** Vercel (Edge/Serverless)
- **Background Workers:** Railway (Docker containers)
- **Database:** Supabase (Managed Postgres)
- **Cache:** Upstash Redis
- **Workflows:** Inngest Cloud
### Vercel Deployment
1. Connect repository to Vercel
2. Configure environment variables
3. Deploy on push to main
```bash
# Manual deployment
npx vercel deploy --prod
```
### Railway Deployment
1. Connect repository to Railway
2. Configure environment variables
3. Set up Inngest worker service
```bash
# Manual deployment
npx @railway/cli up --detach --project <project-id>
```
### CI/CD Pipeline
GitHub Actions workflow: [`.github/workflows/deploy.yml`](../.github/workflows/deploy.yml)
**Build & Test Job:**
- Runs on every push and PR
- Installs dependencies
- Runs lint
- Runs tests
- Builds application
**Deploy Job:**
- Runs on push to main
- Deploys to Vercel
- Deploys to Railway
---
## Development Workflow
### Branch Strategy
```
main
β”œβ”€β”€ develop
β”‚ β”œβ”€β”€ feature/xyz
β”‚ β”œβ”€β”€ fix/abc
β”‚ └── refactor/def
```
### Commit Convention
```
type(scope): description
Types: feat, fix, refactor, docs, test, chore
```
### Pull Request Process
1. Create feature branch from `develop`
2. Make changes and write tests
3. Run lint and tests locally
4. Create PR to `develop`
5. CI must pass before merge
6. Squash merge to `develop`
7. Periodic releases from `develop` to `main`
---
## Debugging
### Local Debugging
```typescript
// Console logging with context
console.log('[VaultService] Fetching vault for slug:', creatorSlug);
// Sentry capture for investigation
Sentry.captureException(error, {
tags: { component: 'VaultGrid', action: 'fetch' },
extra: { creatorSlug },
});
```
### Inngest Debugging
1. Open Inngest Dev Server UI at `http://localhost:8288`
2. View function runs, events, and errors
3. Replay failed functions
4. Inspect step outputs
### Database Debugging
```bash
# Open Drizzle Studio
npx drizzle-kit studio
# Direct SQL query
psql $DATABASE_URL -c "SELECT * FROM detected_objects LIMIT 10;"
```
---
## Common Tasks
### Add a New Feature Module
1. Create directory in `src/features/`
2. Add subdirectories: `actions/`, `components/`, `services/`, `types/`
3. Create server actions with `'use server'`
4. Create components with `'use client'` where needed
5. Add types in `types/`
6. Write tests in `__tests__/`
### Add a New Inngest Function
1. Create file in `src/inngest/functions/`
2. Define event interface
3. Create function with `inngest.createFunction()`
4. Register in `src/app/api/inngest/route.ts`
5. Trigger from server action or other function
### Add a New API Route
1. Create directory in `src/app/api/`
2. Create `route.ts` file
3. Export HTTP method handlers (GET, POST, etc.)
4. Add authentication/authorization as needed
### Add a New Page
1. Create directory in `src/app/`
2. Create `page.tsx` file
3. For dynamic routes, use `[param]/page.tsx`
4. Add metadata export for SEO
5. Use SSR for public pages, client components for dashboards
---
## Troubleshooting
### Common Issues
**Database Connection Errors**
- Verify DATABASE_URL and DIRECT_URL are correct
- Check Supabase project is not paused
- Ensure SSL is required (`ssl: 'require'`)
**Inngest Functions Not Running**
- Verify INNGEST_EVENT_KEY and INNGEST_SIGNING_KEY
- Check Inngest dev server is running locally
- Verify function is registered in API route
**YouTube API Quota Exceeded**
- Check Google Cloud Console for quota usage
- Request quota increase if needed
- Implement caching to reduce API calls
**Marketplace API Rate Limits**
- Check Redis cache is working
- Implement backoff strategies
- Consider upgrading API tiers
### Getting Help
1. Check existing documentation
2. Search GitHub issues
3. Review Inngest dashboard for errors
4. Check Sentry for error details
---
## Related Documentation
- [Project Overview](./project-overview.md) - Architecture and features
- [Source Tree](./source-tree.md) - File structure
- [Data Models](./data-models.md) - Database schema
- [API Contracts](./api-contracts.md) - Endpoints and actions
- [Inngest Workflows](./inngest-workflows.md) - Background jobs