# ICC Interac Manager — Complete Technical Flow **Every technology, every AI model, every data transformation — fully detailed** **Last updated:** February 2026 --- ## 1. System Architecture — Full Technology Map ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ USER'S BROWSER │ │ │ │ ┌────────────────────────────────────────────────────────────────────┐ │ │ │ VITE 6 + REACT 19 SPA │ │ │ │ TypeScript 5 · Tailwind CSS 4 · shadcn/ui │ │ │ │ │ │ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌────────┐ │ │ │ │ │ Login │ │Dashboard │ │ Scan │ │ Settings │ │Reports │ │ │ │ │ │ Page │ │ Page │ │ Page │ │ Page │ │ Page │ │ │ │ │ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ └───┬────┘ │ │ │ │ │ │ │ │ │ │ │ │ │ ┌────┴─────────────┴────────────┴─────────────┴────────────┴────┐│ │ │ │ │ State Management Layer ││ │ │ │ │ Zustand (global) · TanStack Query v5 (server/cache) ││ │ │ │ │ React Hook Form + Zod (forms) · Socket.io-client (realtime) ││ │ │ │ └───────────────────────────┬────────────────────────────────────┘│ │ │ └──────────────────────────────┼─────────────────────────────────────┘ │ │ │ HTTP + WebSocket │ │ ┌──────────────────┼──────────────────┐ │ │ │ Vite Dev Proxy (port 5173) │ │ │ │ /api/* → localhost:3001 │ │ │ │ /ws → ws://localhost:3001 │ │ │ └──────────────────┼──────────────────┘ │ └─────────────────────────────────┼───────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────────┐ │ EXPRESS.JS 5 BACKEND (port 3001) │ │ Node.js 20 LTS · TypeScript 5 │ │ │ │ ┌──────────────────────────────────────────────────────────────────┐ │ │ │ MIDDLEWARE LAYER │ │ │ │ JWT Auth · CORS · express-rate-limit · Helmet CSP · Pino logger │ │ │ └──────────────────────────────────────────────────────────────────┘ │ │ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │ Auth │ │ Scan │ │ Txns │ │ Receipts │ │ Settings │ │ │ │ Routes │ │ Routes │ │ Routes │ │ Routes │ │ Routes │ │ │ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │ │ │ │ │ │ │ │ │ ┌────┴─────────────┴────────────┴─────────────┴────────────┴────┐ │ │ │ SERVICE LAYER │ │ │ │ │ │ │ │ ┌─────────────┐ ┌──────────────┐ ┌───────────────────────┐│ │ │ │ │ Gmail │ │ Scan Engine │ │ AI Provider Pool ││ │ │ │ │ Service │ │ (pipeline) │ │ (auto-switcher) ││ │ │ │ │ │ │ │ │ ││ │ │ │ │ googleapis │ │ Fetch→Parse │ │ Groq ←→ Mistral ││ │ │ │ │ OR imapflow │ │ →Route→Save │ │ (9 free model slots) ││ │ │ │ └─────────────┘ └──────────────┘ └───────────────────────┘│ │ │ │ │ │ │ │ ┌─────────────┐ ┌──────────────┐ ┌───────────────────────┐│ │ │ │ │ Routing │ │ PDF Service │ │ Export Service ││ │ │ │ │ Service │ │ PDFKit / │ │ SheetJS (xlsx) ││ │ │ │ │ branch map │ │ react-pdf │ │ CSV (built-in) ││ │ │ │ └─────────────┘ └──────────────┘ └───────────────────────┘│ │ │ └────────────────────────────────────────────────────────────────┘ │ │ │ │ ┌──────────────────────────────────────────────────────────────────┐ │ │ │ DATA LAYER │ │ │ │ Drizzle ORM · PostgreSQL 16 · Redis + BullMQ (job queue) │ │ │ └──────────────────────────────────────────────────────────────────┘ │ │ │ │ ┌──────────────────────────────────────────────────────────────────┐ │ │ │ WEBSOCKET LAYER (Socket.io) │ │ │ │ scan:progress · scan:completed · transaction:new · ai:switcher │ │ │ └──────────────────────────────────────────────────────────────────┘ │ └─────────────────────────────────────────────────────────────────────────┘ │ │ │ ▼ ▼ ▼ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │ PostgreSQL 16│ │ Redis 7 │ │ External AI APIs │ │ │ │ │ │ │ │ users │ │ BullMQ jobs │ │ api.groq.com │ │ transactions │ │ scan queue │ │ api.mistral.ai │ │ branch_config│ │ rate limiter │ │ (free tier, $0) │ │ scan_logs │ │ │ │ │ │ ai_settings │ │ │ │ [Optional paid:] │ │ ai_switcher │ │ │ │ api.anthropic.com │ │ _logs │ │ │ │ api.openai.com │ └──────────────┘ └──────────────┘ └──────────────────────┘ ``` --- ## 2. Complete Scan Pipeline — Step by Step This is the exact sequence of events when a user clicks **"Scanner maintenant"**. ### Phase 0: User Initiates Scan ``` USER ACTION TECHNOLOGY INVOLVED ─────────────────────────────────── ────────────────────────────────── 1. User clicks "Scanner aujourd'hui" React onClick handler 2. Frontend sends POST /api/scan/start Axios / fetch (TanStack Query mutation) Body: { preset: "today" } 3. Backend receives request Express.js route handler 4. JWT middleware validates token jsonwebtoken (JWT verify) 5. Creates BullMQ job in Redis BullMQ + Redis 6. Returns { jobId: "scan_abc123" } Express response 7. Frontend opens WebSocket Socket.io-client Listens for scan:progress events ``` ### Phase 1: Email Discovery (Gmail API) ``` STEP ACTION TECHNOLOGY TIME ─────── ─────────────────────────────── ───────────────────── ────── 1.1 Resolve date range: resolveScanDates() <1ms "today" → midnight→now (shared util) 1.2 Build Gmail search query: buildGmailQuery() <1ms "from:notify@payments. (server util) interac.ca after:2026/2/22 before:2026/2/24" 1.3 Call Gmail API: list messages googleapis npm 200ms GET gmail/v1/users/me/messages (google-auth-library ?q={query}&maxResults=500 + googleapis) OAuth token from DB: Drizzle ORM → PG users.access_token (AES-256 crypto.decipher encrypted) → decrypt If token expired: google-auth-library auto-refresh with OAuth2Client users.refresh_token .refreshAccessToken() 1.4 Receive message ID list: Gmail API response — ["msg_001", "msg_002", ...] (just IDs, not full emails) 1.5 Deduplication check: Drizzle ORM → PG 50ms SELECT email_id FROM SQL query transactions WHERE (parameterized, email_id IN (...) indexed) 1.6 Filter: skip existing IDs JavaScript Set <1ms Result: newIds[] (emails to process) 1.7 WebSocket emit: Socket.io <1ms scan:started { jobId, totalEmails, newEmails, skipped, dateRange } PHASE 1 TOTAL: ~300ms for discovery ``` ### Phase 2: Parallel Fetch (Gmail API — 10 concurrent) ``` STEP ACTION TECHNOLOGY TIME ─────── ─────────────────────────────── ───────────────────── ────── 2.1 Create concurrency limiter: p-limit npm <1ms gmailLimit = pLimit(10) (10 concurrent) 2.2 For each newId, launch Promise + p-limit — concurrent fetch: ┌──── gmailLimit(async () => { │ │ 2.3 GET gmail/v1/users/me/ googleapis 100ms │ messages/{id} (full MIME format) each │ ?format=full │ │ 2.4 Extract email body: Custom MIME parser <1ms │ - Find text/html part (base64 decode) │ - Base64 decode │ - Strip HTML tags │ - Extract plain text │ │ 2.5 Extract email metadata: MIME header parse <1ms │ - Date header │ - From header │ - Subject header │ └──── return { emailId, body, metadata } }) With 10 concurrent: 50 emails fetched in ~500ms 200 emails in ~2 seconds 1000 emails in ~10 seconds PHASE 2 TOTAL: ~100ms per email, 10 concurrent = ~10ms effective per email ``` ### Phase 3: AI Parsing (Auto-Switcher — up to 15 concurrent on Groq) This is where the AI models do the work. Every email body goes through the `AIProviderPool`. ``` STEP ACTION TECHNOLOGY TIME ─────── ─────────────────────────────── ───────────────────── ────── 3.1 AIProviderPool receives aiProviderPool.ts <1ms email body text 3.2 Select best available slot: getNextAvailableSlot <1ms Check priority order: () 1. groq:gpt-oss-20b 2. groq:llama4-scout 3. groq:llama31-8b ... 7. mistral:small-3.2 ... 3.3 Enforce rate delay: enforceRateDelay() 0-2000ms Groq: 60s/30RPM = 2s delay (per slot) Mistral: 60s/2RPM = 30s delay 3.4 Build AI request: <1ms ┌────────────────────────────────────────────────┐ │ { │ │ model: "openai/gpt-oss-20b", │ │ messages: [ │ │ { role: "system", │ │ content: EXTRACTION_SYSTEM_PROMPT }, │ │ { role: "user", │ │ content: "Extract transaction │ │ details:\n\n{EMAIL_BODY}" } │ │ ], │ │ temperature: 0.0, │ │ max_tokens: 500, │ │ response_format: { type: "json_object" } │ │ } │ └────────────────────────────────────────────────┘ 3.5 Send to AI provider: fetch() or openai 150-800ms npm package ┌─── IF GROQ: ───────────────────────────────────┐ │ POST https://api.groq.com/openai/v1/ │ │ chat/completions │ │ Headers: │ │ Authorization: Bearer gsk_xxxxx │ │ Content-Type: application/json │ │ │ │ Speed: 500-1000 tokens/sec │ │ Typical response: 150-300ms │ └──────────────────────────────────────────────────┘ ┌─── IF MISTRAL: ────────────────────────────────┐ │ POST https://api.mistral.ai/v1/ │ │ chat/completions │ │ Headers: │ │ Authorization: Bearer xxxxx │ │ Content-Type: application/json │ │ │ │ Speed: 100-300 tokens/sec │ │ Typical response: 300-800ms │ └──────────────────────────────────────────────────┘ 3.6 IF 429 RATE LIMIT: Auto-switcher <1ms - Mark current slot as logic in pool rate_limited - Set cooldownUntil from retry-after header - WebSocket emit ai:switcher - Jump to next priority slot - RETRY from step 3.2 3.7 Receive AI response: JSON.parse() <1ms { "sender": "Jean Tremblay", "amount": 150.00, "currency": "CAD", "reference": "CA1b2c3d4e5f", "message": "Dîme mars 2025", "recipient_email": "montreal.finances@iccameriques.org", "date": "2026-02-23T14:30:00Z", "status": "deposited" } 3.8 Validate with Zod schema: zod npm <1ms InteracTransactionSchema .parse(parsed) If validation fails: - Log warning - Retry with different model - Or mark as needs_review PHASE 3 TOTAL: ~200-800ms per email depending on provider With Groq 15 concurrent: ~15ms effective per email ``` ### Phase 4: Branch Routing (No AI — Pure Logic) ``` STEP ACTION TECHNOLOGY TIME ─────── ─────────────────────────────── ───────────────────── ────── 4.1 Look up recipient_email in BRANCH_MAPPING <1ms branch mapping: (shared constant) "montreal.finances JavaScript object @iccameriques.org" lookup, O(1) → "ICC Montréal" 4.2 If no match found: Fallback logic <1ms → "Non classifié" 4.3 Attach branch to transaction: Object assign <1ms transaction.branch = "ICC Montréal" PHASE 4 TOTAL: <1ms (no network, no AI, pure in-memory lookup) ``` ### Phase 5: Database Save (Batch INSERT) ``` STEP ACTION TECHNOLOGY TIME ─────── ─────────────────────────────── ───────────────────── ────── 5.1 Buffer parsed transaction Array.push <1ms into saveBuffer[] 5.2 When buffer reaches 25: Drizzle ORM 5-15ms Batch INSERT INTO → PostgreSQL transactions ( (parameterized) email_id, user_id, date, sender, amount, currency, reference, message, recipient_email, branch, status, raw_email, parsed_at, reviewed ) VALUES (...), (...), ... ★ Single INSERT for 25 rows is 10-20x faster than 25 individual INSERTs 5.3 WebSocket emit per row: Socket.io <1ms transaction:new { (per transaction) transaction: {...} } Dashboard receives and auto-adds to TanStack Table 5.4 Update scan_logs: Drizzle ORM → PG 2ms UPDATE scan_logs SET emails_parsed = emails_parsed + 25 WHERE id = {scanLogId} PHASE 5 TOTAL: ~15ms per batch of 25 transactions ``` ### Phase 6: Completion ``` STEP ACTION TECHNOLOGY TIME ─────── ─────────────────────────────── ───────────────────── ────── 6.1 Flush remaining buffer Drizzle → PG 5ms (< 25 transactions) 6.2 Write final scan log: Drizzle → PG 2ms UPDATE scan_logs SET finished_at = NOW(), emails_found = {n}, emails_parsed = {n}, errors = {n} 6.3 Write switcher logs: Drizzle → PG 2ms INSERT INTO ai_switcher_logs (batch of all switch events) 6.4 WebSocket emit: Socket.io <1ms scan:completed { jobId, summary: { found, parsed, skipped, errors }, dateRange, duration: "45s" } 6.5 Frontend shows toast: Sonner (toast lib) — "47 nouveaux virements importés en 45 secondes" 6.6 Dashboard auto-refreshes TanStack Query — transaction list invalidateQueries() (already live via WebSocket, (backup full refresh) but also invalidates cache) ``` --- ## 3. AI Model Connection Map Exactly which AI model does what, and where in the code it connects. ``` ┌──────────────────────────────────────────────────────────────────────────┐ │ │ │ WHAT AI DOES IN THIS PROJECT: │ │ │ │ AI has ONE job: Parse raw Interac email text → structured JSON │ │ │ │ AI does NOT do: │ │ ✗ Branch routing (pure lookup table, no AI) │ │ ✗ PDF generation (PDFKit / @react-pdf/renderer, no AI) │ │ ✗ Export CSV/Excel (SheetJS, no AI) │ │ ✗ Dashboard charts (Recharts, no AI) │ │ ✗ Authentication (Google OAuth, no AI) │ │ ✗ Email fetching (Gmail API / IMAP, no AI) │ │ ✗ Database queries (Drizzle ORM / SQL, no AI) │ │ ✗ Real-time updates (WebSocket, no AI) │ │ │ └──────────────────────────────────────────────────────────────────────────┘ ``` ### Where Each Model Connects ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ FILE: packages/server/src/services/aiProviderPool.ts │ │ CLASS: AIProviderPool │ │ METHOD: parse(emailBody: string) → InteracTransaction │ │ │ │ This is the ONLY place AI is called in the entire application. │ │ Everything below is managed by the auto-switcher inside this method. │ │ │ │ SLOT 1 ─── groq:gpt-oss-20b ──────────────────────────────────────────│ │ │ Provider: Groq │ │ │ Model: openai/gpt-oss-20b │ │ │ API: POST https://api.groq.com/openai/v1/chat/completions │ │ │ SDK: openai npm (baseURL override) │ │ │ File: packages/server/src/providers/groq.ts │ │ │ Auth: GROQ_API_KEY env var (free, no credit card) │ │ │ Speed: 1,000 tokens/sec · response in ~150ms │ │ │ Free: 30 RPM · 7,000 req/day · 500K tokens/day │ │ │ Input: ~800 tokens (system prompt + email body) │ │ │ Output: ~200 tokens (JSON transaction object) │ │ │ Quality: ★★★★☆ (good at structured extraction) │ │ │ French: ★★★★☆ │ │ │ │ │ SLOT 2 ─── groq:llama4-scout ─────────────────────────────────────────│ │ │ Provider: Groq │ │ │ Model: meta-llama/llama-4-scout-17b-16e-instruct │ │ │ API: POST https://api.groq.com/openai/v1/chat/completions │ │ │ SDK: openai npm (same Groq adapter) │ │ │ Speed: 594 tokens/sec │ │ │ Free: 30 RPM · 1,000 req/day · 500K tokens/day │ │ │ Quality: ★★★★☆ │ │ │ French: ★★★½☆ │ │ │ │ │ SLOT 3 ─── groq:llama31-8b ──────────────────────────────────────────│ │ │ Provider: Groq │ │ │ Model: llama-3.1-8b-instant │ │ │ API: POST https://api.groq.com/openai/v1/chat/completions │ │ │ Speed: 840 tokens/sec · response in ~100ms │ │ │ Free: 30 RPM · 14,400 req/day · 500K tokens/day │ │ │ Quality: ★★★☆☆ (simpler extraction, may miss edge cases) │ │ │ French: ★★★☆☆ │ │ │ │ │ SLOT 4 ─── groq:qwen3-32b ───────────────────────────────────────────│ │ │ Provider: Groq │ │ │ Model: qwen/qwen3-32b │ │ │ Speed: 662 tokens/sec │ │ │ Free: 30 RPM · 1,000 req/day · 500K tokens/day │ │ │ Quality: ★★★★☆ │ │ │ French: ★★★★☆ (strong multilingual) │ │ │ │ │ SLOT 5 ─── groq:llama4-maverick ──────────────────────────────────────│ │ │ Provider: Groq │ │ │ Model: meta-llama/llama-4-maverick-17b-128e-instruct │ │ │ Speed: 562 tokens/sec │ │ │ Free: 30 RPM · 1,000 req/day · 500K tokens/day │ │ │ Quality: ★★★★☆ │ │ │ French: ★★★★☆ │ │ │ │ │ SLOT 6 ─── groq:gpt-oss-120b ────────────────────────────────────────│ │ │ Provider: Groq │ │ │ Model: openai/gpt-oss-120b │ │ │ Speed: 500 tokens/sec │ │ │ Free: 30 RPM · 1,000 req/day · 100K tokens/day │ │ │ Quality: ★★★★★ (best reasoning on Groq) │ │ │ French: ★★★★☆ │ │ │ │ │ ────── PROVIDER BOUNDARY: Groq → Mistral ──────────────────────────── │ │ │ │ SLOT 7 ─── mistral:small-3.2 ────────────────────────────────────────│ │ │ Provider: Mistral AI │ │ │ Model: mistral-small-3.2-24b-instruct │ │ │ API: POST https://api.mistral.ai/v1/chat/completions │ │ │ SDK: fetch() (OpenAI-compatible) │ │ │ File: packages/server/src/providers/mistral.ts │ │ │ Auth: MISTRAL_API_KEY env var (free, no credit card) │ │ │ Speed: ~200 tokens/sec · response in ~400ms │ │ │ Free: 2 RPM · ~2,880 req/day · 1B tokens/month │ │ │ Quality: ★★★★☆ (excellent structured extraction) │ │ │ French: ★★★★★ (built by French company, native French) │ │ │ │ │ SLOT 8 ─── mistral:medium-3.1 ───────────────────────────────────────│ │ │ Provider: Mistral AI │ │ │ Model: mistral-medium-3.1 │ │ │ Speed: ~150 tokens/sec │ │ │ Free: 2 RPM · ~2,880 req/day · 1B tokens/month │ │ │ Quality: ★★★★★ (highest accuracy) │ │ │ French: ★★★★★ │ │ │ │ │ SLOT 9 ─── mistral:ministral-8b ─────────────────────────────────────│ │ │ Provider: Mistral AI │ │ │ Model: ministral-8b-2512 │ │ │ Speed: ~300 tokens/sec │ │ │ Free: 2 RPM · ~2,880 req/day · 1B tokens/month │ │ │ Quality: ★★★☆☆ │ │ │ French: ★★★★☆ │ │ │ │ └─────────────────────────────────────────────────────────────────────────┘ ``` --- ## 4. Technology → Purpose Map (Every Library) ### Frontend (packages/web) | Technology | Version | What It Does in This Project | Touches AI? | |-----------|---------|------------------------------|-------------| | **Vite** | 6 | Dev server, HMR, production bundler | No | | **React** | 19 | Component rendering, UI framework | No | | **TypeScript** | 5 | Type safety across all code | No | | **Tailwind CSS** | 4 | Utility-first styling for all components | No | | **shadcn/ui** | latest | Pre-built components: Button, Dialog, DatePicker, Table, Toggle, Toast | No | | **React Router** | v7 | Page routing: /login, /dashboard, /scan, /settings, /reports | No | | **Zustand** | latest | Global state: auth user, scan status, provider pool status | No | | **TanStack Query** | v5 | Server state: transactions list, scan history, settings (auto-cache, refetch) | No | | **TanStack Table** | v8 | Dashboard transaction table: sort, filter, paginate, select, expand | No | | **React Hook Form** | latest | Settings forms: AI config, branch config, scan date range | No | | **Zod** | latest | Frontend validation: date ranges, settings input | No | | **Socket.io-client** | latest | Real-time: scan:progress, transaction:new, ai:switcher events | No | | **Recharts** | latest | Reports page: bar charts, line charts, pie charts | No | | **@react-pdf/renderer** | latest | Client-side PDF receipt generation (single transaction) | No | | **SheetJS** | latest | Client-side Excel/CSV export | No | | **date-fns** | latest | Date formatting with fr-CA locale | No | | **Lucide React** | latest | Icons throughout the UI | No | | **Sonner** | latest | Toast notifications ("47 virements importés") | No | | **react-i18next** | latest | French UI translations | No | | **p-limit** | latest | Used in scan progress UI for throttled updates | No | ### Backend (packages/server) | Technology | Version | What It Does in This Project | Touches AI? | |-----------|---------|------------------------------|-------------| | **Express.js** | 5 | HTTP API server, route handling | No | | **TypeScript** | 5 | Type safety across all server code | No | | **googleapis** | latest | Gmail API: fetch message IDs, fetch full MIME emails | No | | **google-auth-library** | latest | OAuth 2.0: token exchange, refresh, consent URL | No | | **imapflow** | latest | IMAP fallback: connect to Gmail via IMAP if API unavailable | No | | **openai** (npm) | latest | **Groq adapter**: same SDK, different baseURL → api.groq.com | **YES** | | **fetch** (built-in) | — | **Mistral adapter**: direct HTTP to api.mistral.ai | **YES** | | **@anthropic-ai/sdk** | latest | **Claude adapter** (optional paid): api.anthropic.com | **YES** | | **Drizzle ORM** | latest | Database queries: all CRUD, batch inserts, migrations | No | | **PostgreSQL driver** (pg) | latest | Database connection pooling | No | | **Redis** (ioredis) | latest | BullMQ backend, rate limit counters, session cache | No | | **BullMQ** | latest | Background job queue: scan jobs run async, not blocking API | No | | **Socket.io** | latest | WebSocket server: push scan progress + provider switches to UI | No | | **jsonwebtoken** | latest | JWT: issue access tokens (15 min) + refresh tokens (7 day) | No | | **PDFKit** | latest | Server-side batch PDF receipt generation | No | | **Pino** | latest | Structured JSON logging | No | | **Helmet** | latest | Security headers (CSP, HSTS, etc.) | No | | **cors** | latest | CORS policy: restrict to known frontend origins | No | | **express-rate-limit** | latest | API rate limiting (prevent abuse of scan endpoints) | No | | **p-limit** | latest | Concurrency control: 10 Gmail fetches, 15 Groq parses | No | | **Zod** | latest | Server-side validation: AI output JSON, API request bodies | No | | **crypto** (built-in) | — | AES-256 encryption: OAuth tokens at rest in PostgreSQL | No | ### Shared (packages/shared) | Technology | What It Does | |-----------|-------------| | **TypeScript types** | InteracTransaction, ScanDateRange, ProviderSlot, SwitcherEvent | | **BRANCH_MAPPING** | 48 email→branch lookup pairs (no AI needed) | | **resolveScanDates()** | Preset→date range helper | | **Zod schemas** | InteracTransactionSchema for validating AI output | --- ## 5. AI Data Flow — Input → Output ### What Goes INTO the AI ``` ╔══════════════════════════════════════════════════════════════════════╗ ║ SYSTEM PROMPT (~350 tokens, same for every email, cached) ║ ║ ║ ║ "You are a financial data extraction assistant. Given the raw ║ ║ text/HTML of an Interac e-Transfer notification email from ║ ║ notify@payments.interac.ca, extract the following fields into ║ ║ a JSON object: ║ ║ ║ ║ - sender: The name of the person who sent the money ║ ║ - amount: The dollar amount (numeric, no $ sign) ║ ║ - currency: Always "CAD" ║ ║ - reference: The Interac reference number ║ ║ - message: The personal message/memo (null if none) ║ ║ - recipient_email: The email the transfer was sent TO ║ ║ - date: The date/time in ISO 8601 format ║ ║ - status: One of deposited, pending, expired, cancelled ║ ║ ║ ║ Return ONLY valid JSON, no markdown, no explanation." ║ ╠══════════════════════════════════════════════════════════════════════╣ ║ USER MESSAGE (~450 tokens, unique per email) ║ ║ ║ ║ "Extract transaction details: ║ ║ ║ ║ INTERAC e-Transfer ║ ║ Bonjour, vous avez reçu un virement Interac de Jean Tremblay. ║ ║ Montant : 150,00 $ (CAD) ║ ║ Numéro de référence: CA1b2c3d4e5f ║ ║ Message de l'expéditeur : Dîme mars 2025 ║ ║ Ce virement a été automatiquement déposé dans le compte de: ║ ║ montreal.finances@iccameriques.org ║ ║ Date: 23 février 2026 14:30 ║ ║ ..." ║ ╚══════════════════════════════════════════════════════════════════════╝ Total input: ~800 tokens ``` ### What Comes OUT of the AI ``` ╔══════════════════════════════════════════════════════════════════════╗ ║ AI RESPONSE (~200 tokens) ║ ║ ║ ║ { ║ ║ "sender": "Jean Tremblay", ║ ║ "amount": 150.00, ║ ║ "currency": "CAD", ║ ║ "reference": "CA1b2c3d4e5f", ║ ║ "message": "Dîme mars 2025", ║ ║ "recipient_email": "montreal.finances@iccameriques.org", ║ ║ "date": "2026-02-23T14:30:00Z", ║ ║ "status": "deposited" ║ ║ } ║ ╚══════════════════════════════════════════════════════════════════════╝ Total output: ~200 tokens Total per email: ~1,000 tokens ``` ### What Happens AFTER AI (No AI Involved) ``` AI Output (JSON) │ ▼ Zod Validation ──── FAIL? → log warning, retry with next model, or flag as needs_review │ ✅ PASS │ ▼ Branch Routing ──── BRANCH_MAPPING["montreal.finances@iccameriques.org"] → "ICC Montréal" │ (pure JavaScript lookup, no AI) │ ▼ PostgreSQL INSERT ── Drizzle ORM → INSERT INTO transactions (...) │ ▼ WebSocket Push ──── Socket.io → transaction:new { ... } │ ▼ React Dashboard ─── TanStack Table auto-adds row to table │ ▼ User Sees New Row ── "Jean Tremblay | 150,00 $ | ICC Montréal | Déposé" ``` --- ## 6. Speed Optimization — Why It's Fast ### Old Architecture (Sequential) ``` Email 1: [Fetch 100ms] → [AI Parse 300ms] → [Save 5ms] = 405ms Email 2: [Fetch] → [Parse] → [Save] Email 3: [Fetch] → ... Total for 50 emails: 50 × 405ms = 20,250ms = ~20 seconds ``` ### New Architecture (Parallel Pipeline) ``` Time → 0ms 100ms 200ms 300ms 400ms 500ms ┌──────┬──────┬──────┬──────┬──────┬──────┐ Fetch: │E1-E10│E11-20│E21-30│E31-40│E41-50│ │ (10 concurrent) Parse: │ │P1-P15│P16-30│P31-45│P46-50│ │ (15 concurrent) Save: │ │ │S1-S25│ │S26-50│ │ (batch 25) └──────┴──────┴──────┴──────┴──────┴──────┘ Total for 50 emails: ~500ms for fetches + ~700ms for parses = ~1.2 seconds (stages overlap, so total < sum) ``` ### Concurrency Budget Per Free Tier ``` GROQ MISTRAL RPM (requests/minute): 30 2 Safe concurrency: 15 concurrent calls 1 sequential call Time per AI call: ~150ms (Groq is fast) ~400ms Emails parsed per minute: ~30 (RPM-limited) ~2 (RPM-limited) Emails parsed per hour: ~1,800 ~120 WITH AUTO-SWITCHER (all 6 Groq + 3 Mistral): Effective RPM: 30 + 30 + 30 + 30 + 30 + 30 + 2 + 2 + 2 = 186 RPM* (*until individual daily limits hit) First hour throughput: ~180 emails/min = ~10,800 emails/hour ``` ### Speed Benchmarks (Revised with Pipeline) | Scan Type | Emails | Time (Pipeline + Auto-Switcher) | |-----------|--------|-------------------------------| | Today | 50 | **~12 seconds** | | 7 Days | 200 | **~35 seconds** | | Custom (1 month) | 1,000 | **~3 minutes** | | Custom (6 months) | 5,000 | **~15 minutes** | | Custom (1 year) | 10,000 | **~30 minutes** | --- ## 7. Non-AI Technology Flows ### PDF Receipt Generation (No AI) ``` User clicks "Générer reçu" on a transaction row │ ▼ CLIENT-SIDE (single receipt): @react-pdf/renderer builds PDF in browser Template: ICC logo + transaction details + branch info + date Downloads immediately as "recu_CA1b2c3d4e5f.pdf" SERVER-SIDE (batch receipts): POST /api/receipts/batch { transactionIds: [...] } │ ▼ PDFKit generates PDF per transaction Merges into single ZIP file Returns ZIP for download ``` ### CSV/Excel Export (No AI) ``` User clicks "Exporter" → selects CSV or Excel │ ▼ CLIENT-SIDE: SheetJS (xlsx npm) builds file from TanStack Table data Columns: Date | Expéditeur | Montant | Référence | Succursale | Statut fr-CA formatting: "1 500,00 $" not "$1,500.00" Downloads immediately ``` ### Reports & Charts (No AI) ``` User navigates to /reports │ ▼ GET /api/transactions/stats?from=2026-01-01&to=2026-02-23 │ ▼ PostgreSQL aggregation queries: - SUM(amount) GROUP BY month → bar chart (Recharts) - COUNT(*) GROUP BY status → pie chart (Recharts) - COUNT(*) GROUP BY branch → top branches table - SUM(amount) over time → trend line chart (Recharts) │ ▼ React renders charts with Recharts No AI involved — pure SQL aggregation + charting library ``` ### Real-Time Dashboard Updates (No AI) ``` Server saves new transaction to PostgreSQL │ ▼ Socket.io server emits: transaction:new { transaction: {...} } │ ▼ Socket.io client receives event │ ▼ Zustand store updates transaction list │ ▼ TanStack Table re-renders with new row (animated row insertion at top of table) │ ▼ SummaryBar recalculates totals (no page refresh, no API call needed) ``` --- ## 8. Security Flow (No AI) ``` 1. User clicks "Se connecter avec Google" 2. Frontend redirects to Google OAuth consent URL (scopes: gmail.readonly, userinfo.email, userinfo.profile) 3. User approves → Google redirects to /api/auth/google/callback?code=xxx 4. Backend exchanges auth code for access_token + refresh_token (google-auth-library) 5. Backend encrypts tokens with AES-256 (crypto module) 6. Backend stores encrypted tokens in PostgreSQL (users table) 7. Backend issues JWT: { userId, email, role, exp: 15min } (jsonwebtoken) 8. Frontend stores JWT in httpOnly secure cookie 9. Every API request includes JWT in Authorization header 10. Backend middleware verifies JWT on every request (jsonwebtoken.verify) 11. If JWT expired → frontend calls POST /api/auth/refresh with refresh token → new JWT issued ``` --- *This document is a companion to the ICC Interac Manager Build Prompt and the FREE AI Models Guide. Together, the three documents provide everything needed to build the complete system.*