Spaces:
Running
Running
ICC Interac Manager — Complete Technical Flow
Every technology, every AI model, every data transformation — fully detailed
Last updated: February 2026
1. System Architecture — Full Technology Map
┌─────────────────────────────────────────────────────────────────────────┐
│ USER'S BROWSER │
│ │
│ ┌────────────────────────────────────────────────────────────────────┐ │
│ │ VITE 6 + REACT 19 SPA │ │
│ │ TypeScript 5 · Tailwind CSS 4 · shadcn/ui │ │
│ │ │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌────────┐ │ │
│ │ │ Login │ │Dashboard │ │ Scan │ │ Settings │ │Reports │ │ │
│ │ │ Page │ │ Page │ │ Page │ │ Page │ │ Page │ │ │
│ │ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ └───┬────┘ │ │
│ │ │ │ │ │ │ │ │
│ │ ┌────┴─────────────┴────────────┴─────────────┴────────────┴────┐│ │
│ │ │ State Management Layer ││ │
│ │ │ Zustand (global) · TanStack Query v5 (server/cache) ││ │
│ │ │ React Hook Form + Zod (forms) · Socket.io-client (realtime) ││ │
│ │ └───────────────────────────┬────────────────────────────────────┘│ │
│ └──────────────────────────────┼─────────────────────────────────────┘ │
│ │ HTTP + WebSocket │
│ ┌──────────────────┼──────────────────┐ │
│ │ Vite Dev Proxy (port 5173) │ │
│ │ /api/* → localhost:3001 │ │
│ │ /ws → ws://localhost:3001 │ │
│ └──────────────────┼──────────────────┘ │
└─────────────────────────────────┼───────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ EXPRESS.JS 5 BACKEND (port 3001) │
│ Node.js 20 LTS · TypeScript 5 │
│ │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ MIDDLEWARE LAYER │ │
│ │ JWT Auth · CORS · express-rate-limit · Helmet CSP · Pino logger │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Auth │ │ Scan │ │ Txns │ │ Receipts │ │ Settings │ │
│ │ Routes │ │ Routes │ │ Routes │ │ Routes │ │ Routes │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ │ │ │ │ │ │
│ ┌────┴─────────────┴────────────┴─────────────┴────────────┴────┐ │
│ │ SERVICE LAYER │ │
│ │ │ │
│ │ ┌─────────────┐ ┌──────────────┐ ┌───────────────────────┐│ │
│ │ │ Gmail │ │ Scan Engine │ │ AI Provider Pool ││ │
│ │ │ Service │ │ (pipeline) │ │ (auto-switcher) ││ │
│ │ │ │ │ │ │ ││ │
│ │ │ googleapis │ │ Fetch→Parse │ │ Groq ←→ Mistral ││ │
│ │ │ OR imapflow │ │ →Route→Save │ │ (9 free model slots) ││ │
│ │ └─────────────┘ └──────────────┘ └───────────────────────┘│ │
│ │ │ │
│ │ ┌─────────────┐ ┌──────────────┐ ┌───────────────────────┐│ │
│ │ │ Routing │ │ PDF Service │ │ Export Service ││ │
│ │ │ Service │ │ PDFKit / │ │ SheetJS (xlsx) ││ │
│ │ │ branch map │ │ react-pdf │ │ CSV (built-in) ││ │
│ │ └─────────────┘ └──────────────┘ └───────────────────────┘│ │
│ └────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ DATA LAYER │ │
│ │ Drizzle ORM · PostgreSQL 16 · Redis + BullMQ (job queue) │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ WEBSOCKET LAYER (Socket.io) │ │
│ │ scan:progress · scan:completed · transaction:new · ai:switcher │ │
│ └──────────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
│ │ │
▼ ▼ ▼
┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐
│ PostgreSQL 16│ │ Redis 7 │ │ External AI APIs │
│ │ │ │ │ │
│ users │ │ BullMQ jobs │ │ api.groq.com │
│ transactions │ │ scan queue │ │ api.mistral.ai │
│ branch_config│ │ rate limiter │ │ (free tier, $0) │
│ scan_logs │ │ │ │ │
│ ai_settings │ │ │ │ [Optional paid:] │
│ ai_switcher │ │ │ │ api.anthropic.com │
│ _logs │ │ │ │ api.openai.com │
└──────────────┘ └──────────────┘ └──────────────────────┘
2. Complete Scan Pipeline — Step by Step
This is the exact sequence of events when a user clicks "Scanner maintenant".
Phase 0: User Initiates Scan
USER ACTION TECHNOLOGY INVOLVED
─────────────────────────────────── ──────────────────────────────────
1. User clicks "Scanner aujourd'hui" React onClick handler
2. Frontend sends POST /api/scan/start Axios / fetch (TanStack Query mutation)
Body: { preset: "today" }
3. Backend receives request Express.js route handler
4. JWT middleware validates token jsonwebtoken (JWT verify)
5. Creates BullMQ job in Redis BullMQ + Redis
6. Returns { jobId: "scan_abc123" } Express response
7. Frontend opens WebSocket Socket.io-client
Listens for scan:progress events
Phase 1: Email Discovery (Gmail API)
STEP ACTION TECHNOLOGY TIME
─────── ─────────────────────────────── ───────────────────── ──────
1.1 Resolve date range: resolveScanDates() <1ms
"today" → midnight→now (shared util)
1.2 Build Gmail search query: buildGmailQuery() <1ms
"from:notify@payments. (server util)
interac.ca after:2026/2/22
before:2026/2/24"
1.3 Call Gmail API: list messages googleapis npm 200ms
GET gmail/v1/users/me/messages (google-auth-library
?q={query}&maxResults=500 + googleapis)
OAuth token from DB: Drizzle ORM → PG
users.access_token (AES-256 crypto.decipher
encrypted) → decrypt
If token expired: google-auth-library
auto-refresh with OAuth2Client
users.refresh_token .refreshAccessToken()
1.4 Receive message ID list: Gmail API response —
["msg_001", "msg_002", ...]
(just IDs, not full emails)
1.5 Deduplication check: Drizzle ORM → PG 50ms
SELECT email_id FROM SQL query
transactions WHERE (parameterized,
email_id IN (...) indexed)
1.6 Filter: skip existing IDs JavaScript Set <1ms
Result: newIds[] (emails
to process)
1.7 WebSocket emit: Socket.io <1ms
scan:started {
jobId, totalEmails,
newEmails, skipped,
dateRange
}
PHASE 1 TOTAL: ~300ms for discovery
Phase 2: Parallel Fetch (Gmail API — 10 concurrent)
STEP ACTION TECHNOLOGY TIME
─────── ─────────────────────────────── ───────────────────── ──────
2.1 Create concurrency limiter: p-limit npm <1ms
gmailLimit = pLimit(10) (10 concurrent)
2.2 For each newId, launch Promise + p-limit —
concurrent fetch:
┌──── gmailLimit(async () => {
│
│ 2.3 GET gmail/v1/users/me/ googleapis 100ms
│ messages/{id} (full MIME format) each
│ ?format=full
│
│ 2.4 Extract email body: Custom MIME parser <1ms
│ - Find text/html part (base64 decode)
│ - Base64 decode
│ - Strip HTML tags
│ - Extract plain text
│
│ 2.5 Extract email metadata: MIME header parse <1ms
│ - Date header
│ - From header
│ - Subject header
│
└──── return { emailId, body, metadata } })
With 10 concurrent: 50 emails fetched in ~500ms
200 emails in ~2 seconds
1000 emails in ~10 seconds
PHASE 2 TOTAL: ~100ms per email, 10 concurrent = ~10ms effective per email
Phase 3: AI Parsing (Auto-Switcher — up to 15 concurrent on Groq)
This is where the AI models do the work. Every email body goes through the AIProviderPool.
STEP ACTION TECHNOLOGY TIME
─────── ─────────────────────────────── ───────────────────── ──────
3.1 AIProviderPool receives aiProviderPool.ts <1ms
email body text
3.2 Select best available slot: getNextAvailableSlot <1ms
Check priority order: ()
1. groq:gpt-oss-20b
2. groq:llama4-scout
3. groq:llama31-8b
...
7. mistral:small-3.2
...
3.3 Enforce rate delay: enforceRateDelay() 0-2000ms
Groq: 60s/30RPM = 2s delay (per slot)
Mistral: 60s/2RPM = 30s delay
3.4 Build AI request: <1ms
┌────────────────────────────────────────────────┐
│ { │
│ model: "openai/gpt-oss-20b", │
│ messages: [ │
│ { role: "system", │
│ content: EXTRACTION_SYSTEM_PROMPT }, │
│ { role: "user", │
│ content: "Extract transaction │
│ details:\n\n{EMAIL_BODY}" } │
│ ], │
│ temperature: 0.0, │
│ max_tokens: 500, │
│ response_format: { type: "json_object" } │
│ } │
└────────────────────────────────────────────────┘
3.5 Send to AI provider: fetch() or openai 150-800ms
npm package
┌─── IF GROQ: ───────────────────────────────────┐
│ POST https://api.groq.com/openai/v1/ │
│ chat/completions │
│ Headers: │
│ Authorization: Bearer gsk_xxxxx │
│ Content-Type: application/json │
│ │
│ Speed: 500-1000 tokens/sec │
│ Typical response: 150-300ms │
└──────────────────────────────────────────────────┘
┌─── IF MISTRAL: ────────────────────────────────┐
│ POST https://api.mistral.ai/v1/ │
│ chat/completions │
│ Headers: │
│ Authorization: Bearer xxxxx │
│ Content-Type: application/json │
│ │
│ Speed: 100-300 tokens/sec │
│ Typical response: 300-800ms │
└──────────────────────────────────────────────────┘
3.6 IF 429 RATE LIMIT: Auto-switcher <1ms
- Mark current slot as logic in pool
rate_limited
- Set cooldownUntil from
retry-after header
- WebSocket emit ai:switcher
- Jump to next priority slot
- RETRY from step 3.2
3.7 Receive AI response: JSON.parse() <1ms
{
"sender": "Jean Tremblay",
"amount": 150.00,
"currency": "CAD",
"reference": "CA1b2c3d4e5f",
"message": "Dîme mars 2025",
"recipient_email": "montreal.finances@iccameriques.org",
"date": "2026-02-23T14:30:00Z",
"status": "deposited"
}
3.8 Validate with Zod schema: zod npm <1ms
InteracTransactionSchema
.parse(parsed)
If validation fails:
- Log warning
- Retry with different model
- Or mark as needs_review
PHASE 3 TOTAL: ~200-800ms per email depending on provider
With Groq 15 concurrent: ~15ms effective per email
Phase 4: Branch Routing (No AI — Pure Logic)
STEP ACTION TECHNOLOGY TIME
─────── ─────────────────────────────── ───────────────────── ──────
4.1 Look up recipient_email in BRANCH_MAPPING <1ms
branch mapping: (shared constant)
"montreal.finances JavaScript object
@iccameriques.org" lookup, O(1)
→ "ICC Montréal"
4.2 If no match found: Fallback logic <1ms
→ "Non classifié"
4.3 Attach branch to transaction: Object assign <1ms
transaction.branch =
"ICC Montréal"
PHASE 4 TOTAL: <1ms (no network, no AI, pure in-memory lookup)
Phase 5: Database Save (Batch INSERT)
STEP ACTION TECHNOLOGY TIME
─────── ─────────────────────────────── ───────────────────── ──────
5.1 Buffer parsed transaction Array.push <1ms
into saveBuffer[]
5.2 When buffer reaches 25: Drizzle ORM 5-15ms
Batch INSERT INTO → PostgreSQL
transactions ( (parameterized)
email_id, user_id, date,
sender, amount, currency,
reference, message,
recipient_email, branch,
status, raw_email,
parsed_at, reviewed
) VALUES (...), (...), ...
★ Single INSERT for 25 rows
is 10-20x faster than 25
individual INSERTs
5.3 WebSocket emit per row: Socket.io <1ms
transaction:new { (per transaction)
transaction: {...}
}
Dashboard receives and
auto-adds to TanStack Table
5.4 Update scan_logs: Drizzle ORM → PG 2ms
UPDATE scan_logs SET
emails_parsed = emails_parsed + 25
WHERE id = {scanLogId}
PHASE 5 TOTAL: ~15ms per batch of 25 transactions
Phase 6: Completion
STEP ACTION TECHNOLOGY TIME
─────── ─────────────────────────────── ───────────────────── ──────
6.1 Flush remaining buffer Drizzle → PG 5ms
(< 25 transactions)
6.2 Write final scan log: Drizzle → PG 2ms
UPDATE scan_logs SET
finished_at = NOW(),
emails_found = {n},
emails_parsed = {n},
errors = {n}
6.3 Write switcher logs: Drizzle → PG 2ms
INSERT INTO ai_switcher_logs
(batch of all switch events)
6.4 WebSocket emit: Socket.io <1ms
scan:completed {
jobId,
summary: { found, parsed,
skipped, errors },
dateRange,
duration: "45s"
}
6.5 Frontend shows toast: Sonner (toast lib) —
"47 nouveaux virements
importés en 45 secondes"
6.6 Dashboard auto-refreshes TanStack Query —
transaction list invalidateQueries()
(already live via WebSocket, (backup full refresh)
but also invalidates cache)
3. AI Model Connection Map
Exactly which AI model does what, and where in the code it connects.
┌──────────────────────────────────────────────────────────────────────────┐
│ │
│ WHAT AI DOES IN THIS PROJECT: │
│ │
│ AI has ONE job: Parse raw Interac email text → structured JSON │
│ │
│ AI does NOT do: │
│ ✗ Branch routing (pure lookup table, no AI) │
│ ✗ PDF generation (PDFKit / @react-pdf/renderer, no AI) │
│ ✗ Export CSV/Excel (SheetJS, no AI) │
│ ✗ Dashboard charts (Recharts, no AI) │
│ ✗ Authentication (Google OAuth, no AI) │
│ ✗ Email fetching (Gmail API / IMAP, no AI) │
│ ✗ Database queries (Drizzle ORM / SQL, no AI) │
│ ✗ Real-time updates (WebSocket, no AI) │
│ │
└──────────────────────────────────────────────────────────────────────────┘
Where Each Model Connects
┌─────────────────────────────────────────────────────────────────────────┐
│ FILE: packages/server/src/services/aiProviderPool.ts │
│ CLASS: AIProviderPool │
│ METHOD: parse(emailBody: string) → InteracTransaction │
│ │
│ This is the ONLY place AI is called in the entire application. │
│ Everything below is managed by the auto-switcher inside this method. │
│ │
│ SLOT 1 ─── groq:gpt-oss-20b ──────────────────────────────────────────│
│ │ Provider: Groq │
│ │ Model: openai/gpt-oss-20b │
│ │ API: POST https://api.groq.com/openai/v1/chat/completions │
│ │ SDK: openai npm (baseURL override) │
│ │ File: packages/server/src/providers/groq.ts │
│ │ Auth: GROQ_API_KEY env var (free, no credit card) │
│ │ Speed: 1,000 tokens/sec · response in ~150ms │
│ │ Free: 30 RPM · 7,000 req/day · 500K tokens/day │
│ │ Input: ~800 tokens (system prompt + email body) │
│ │ Output: ~200 tokens (JSON transaction object) │
│ │ Quality: ★★★★☆ (good at structured extraction) │
│ │ French: ★★★★☆ │
│ │ │
│ SLOT 2 ─── groq:llama4-scout ─────────────────────────────────────────│
│ │ Provider: Groq │
│ │ Model: meta-llama/llama-4-scout-17b-16e-instruct │
│ │ API: POST https://api.groq.com/openai/v1/chat/completions │
│ │ SDK: openai npm (same Groq adapter) │
│ │ Speed: 594 tokens/sec │
│ │ Free: 30 RPM · 1,000 req/day · 500K tokens/day │
│ │ Quality: ★★★★☆ │
│ │ French: ★★★½☆ │
│ │ │
│ SLOT 3 ─── groq:llama31-8b ──────────────────────────────────────────│
│ │ Provider: Groq │
│ │ Model: llama-3.1-8b-instant │
│ │ API: POST https://api.groq.com/openai/v1/chat/completions │
│ │ Speed: 840 tokens/sec · response in ~100ms │
│ │ Free: 30 RPM · 14,400 req/day · 500K tokens/day │
│ │ Quality: ★★★☆☆ (simpler extraction, may miss edge cases) │
│ │ French: ★★★☆☆ │
│ │ │
│ SLOT 4 ─── groq:qwen3-32b ───────────────────────────────────────────│
│ │ Provider: Groq │
│ │ Model: qwen/qwen3-32b │
│ │ Speed: 662 tokens/sec │
│ │ Free: 30 RPM · 1,000 req/day · 500K tokens/day │
│ │ Quality: ★★★★☆ │
│ │ French: ★★★★☆ (strong multilingual) │
│ │ │
│ SLOT 5 ─── groq:llama4-maverick ──────────────────────────────────────│
│ │ Provider: Groq │
│ │ Model: meta-llama/llama-4-maverick-17b-128e-instruct │
│ │ Speed: 562 tokens/sec │
│ │ Free: 30 RPM · 1,000 req/day · 500K tokens/day │
│ │ Quality: ★★★★☆ │
│ │ French: ★★★★☆ │
│ │ │
│ SLOT 6 ─── groq:gpt-oss-120b ────────────────────────────────────────│
│ │ Provider: Groq │
│ │ Model: openai/gpt-oss-120b │
│ │ Speed: 500 tokens/sec │
│ │ Free: 30 RPM · 1,000 req/day · 100K tokens/day │
│ │ Quality: ★★★★★ (best reasoning on Groq) │
│ │ French: ★★★★☆ │
│ │ │
│ ────── PROVIDER BOUNDARY: Groq → Mistral ──────────────────────────── │
│ │
│ SLOT 7 ─── mistral:small-3.2 ────────────────────────────────────────│
│ │ Provider: Mistral AI │
│ │ Model: mistral-small-3.2-24b-instruct │
│ │ API: POST https://api.mistral.ai/v1/chat/completions │
│ │ SDK: fetch() (OpenAI-compatible) │
│ │ File: packages/server/src/providers/mistral.ts │
│ │ Auth: MISTRAL_API_KEY env var (free, no credit card) │
│ │ Speed: ~200 tokens/sec · response in ~400ms │
│ │ Free: 2 RPM · ~2,880 req/day · 1B tokens/month │
│ │ Quality: ★★★★☆ (excellent structured extraction) │
│ │ French: ★★★★★ (built by French company, native French) │
│ │ │
│ SLOT 8 ─── mistral:medium-3.1 ───────────────────────────────────────│
│ │ Provider: Mistral AI │
│ │ Model: mistral-medium-3.1 │
│ │ Speed: ~150 tokens/sec │
│ │ Free: 2 RPM · ~2,880 req/day · 1B tokens/month │
│ │ Quality: ★★★★★ (highest accuracy) │
│ │ French: ★★★★★ │
│ │ │
│ SLOT 9 ─── mistral:ministral-8b ─────────────────────────────────────│
│ │ Provider: Mistral AI │
│ │ Model: ministral-8b-2512 │
│ │ Speed: ~300 tokens/sec │
│ │ Free: 2 RPM · ~2,880 req/day · 1B tokens/month │
│ │ Quality: ★★★☆☆ │
│ │ French: ★★★★☆ │
│ │ │
└─────────────────────────────────────────────────────────────────────────┘
4. Technology → Purpose Map (Every Library)
Frontend (packages/web)
| Technology | Version | What It Does in This Project | Touches AI? |
|---|---|---|---|
| Vite | 6 | Dev server, HMR, production bundler | No |
| React | 19 | Component rendering, UI framework | No |
| TypeScript | 5 | Type safety across all code | No |
| Tailwind CSS | 4 | Utility-first styling for all components | No |
| shadcn/ui | latest | Pre-built components: Button, Dialog, DatePicker, Table, Toggle, Toast | No |
| React Router | v7 | Page routing: /login, /dashboard, /scan, /settings, /reports | No |
| Zustand | latest | Global state: auth user, scan status, provider pool status | No |
| TanStack Query | v5 | Server state: transactions list, scan history, settings (auto-cache, refetch) | No |
| TanStack Table | v8 | Dashboard transaction table: sort, filter, paginate, select, expand | No |
| React Hook Form | latest | Settings forms: AI config, branch config, scan date range | No |
| Zod | latest | Frontend validation: date ranges, settings input | No |
| Socket.io-client | latest | Real-time: scan:progress, transaction:new, ai:switcher events | No |
| Recharts | latest | Reports page: bar charts, line charts, pie charts | No |
| @react-pdf/renderer | latest | Client-side PDF receipt generation (single transaction) | No |
| SheetJS | latest | Client-side Excel/CSV export | No |
| date-fns | latest | Date formatting with fr-CA locale | No |
| Lucide React | latest | Icons throughout the UI | No |
| Sonner | latest | Toast notifications ("47 virements importés") | No |
| react-i18next | latest | French UI translations | No |
| p-limit | latest | Used in scan progress UI for throttled updates | No |
Backend (packages/server)
| Technology | Version | What It Does in This Project | Touches AI? |
|---|---|---|---|
| Express.js | 5 | HTTP API server, route handling | No |
| TypeScript | 5 | Type safety across all server code | No |
| googleapis | latest | Gmail API: fetch message IDs, fetch full MIME emails | No |
| google-auth-library | latest | OAuth 2.0: token exchange, refresh, consent URL | No |
| imapflow | latest | IMAP fallback: connect to Gmail via IMAP if API unavailable | No |
| openai (npm) | latest | Groq adapter: same SDK, different baseURL → api.groq.com | YES |
| fetch (built-in) | — | Mistral adapter: direct HTTP to api.mistral.ai | YES |
| @anthropic-ai/sdk | latest | Claude adapter (optional paid): api.anthropic.com | YES |
| Drizzle ORM | latest | Database queries: all CRUD, batch inserts, migrations | No |
| PostgreSQL driver (pg) | latest | Database connection pooling | No |
| Redis (ioredis) | latest | BullMQ backend, rate limit counters, session cache | No |
| BullMQ | latest | Background job queue: scan jobs run async, not blocking API | No |
| Socket.io | latest | WebSocket server: push scan progress + provider switches to UI | No |
| jsonwebtoken | latest | JWT: issue access tokens (15 min) + refresh tokens (7 day) | No |
| PDFKit | latest | Server-side batch PDF receipt generation | No |
| Pino | latest | Structured JSON logging | No |
| Helmet | latest | Security headers (CSP, HSTS, etc.) | No |
| cors | latest | CORS policy: restrict to known frontend origins | No |
| express-rate-limit | latest | API rate limiting (prevent abuse of scan endpoints) | No |
| p-limit | latest | Concurrency control: 10 Gmail fetches, 15 Groq parses | No |
| Zod | latest | Server-side validation: AI output JSON, API request bodies | No |
| crypto (built-in) | — | AES-256 encryption: OAuth tokens at rest in PostgreSQL | No |
Shared (packages/shared)
| Technology | What It Does |
|---|---|
| TypeScript types | InteracTransaction, ScanDateRange, ProviderSlot, SwitcherEvent |
| BRANCH_MAPPING | 48 email→branch lookup pairs (no AI needed) |
| resolveScanDates() | Preset→date range helper |
| Zod schemas | InteracTransactionSchema for validating AI output |
5. AI Data Flow — Input → Output
What Goes INTO the AI
╔══════════════════════════════════════════════════════════════════════╗
║ SYSTEM PROMPT (~350 tokens, same for every email, cached) ║
║ ║
║ "You are a financial data extraction assistant. Given the raw ║
║ text/HTML of an Interac e-Transfer notification email from ║
║ notify@payments.interac.ca, extract the following fields into ║
║ a JSON object: ║
║ ║
║ - sender: The name of the person who sent the money ║
║ - amount: The dollar amount (numeric, no $ sign) ║
║ - currency: Always "CAD" ║
║ - reference: The Interac reference number ║
║ - message: The personal message/memo (null if none) ║
║ - recipient_email: The email the transfer was sent TO ║
║ - date: The date/time in ISO 8601 format ║
║ - status: One of deposited, pending, expired, cancelled ║
║ ║
║ Return ONLY valid JSON, no markdown, no explanation." ║
╠══════════════════════════════════════════════════════════════════════╣
║ USER MESSAGE (~450 tokens, unique per email) ║
║ ║
║ "Extract transaction details: ║
║ ║
║ INTERAC e-Transfer ║
║ Bonjour, vous avez reçu un virement Interac de Jean Tremblay. ║
║ Montant : 150,00 $ (CAD) ║
║ Numéro de référence: CA1b2c3d4e5f ║
║ Message de l'expéditeur : Dîme mars 2025 ║
║ Ce virement a été automatiquement déposé dans le compte de: ║
║ montreal.finances@iccameriques.org ║
║ Date: 23 février 2026 14:30 ║
║ ..." ║
╚══════════════════════════════════════════════════════════════════════╝
Total input: ~800 tokens
What Comes OUT of the AI
╔══════════════════════════════════════════════════════════════════════╗
║ AI RESPONSE (~200 tokens) ║
║ ║
║ { ║
║ "sender": "Jean Tremblay", ║
║ "amount": 150.00, ║
║ "currency": "CAD", ║
║ "reference": "CA1b2c3d4e5f", ║
║ "message": "Dîme mars 2025", ║
║ "recipient_email": "montreal.finances@iccameriques.org", ║
║ "date": "2026-02-23T14:30:00Z", ║
║ "status": "deposited" ║
║ } ║
╚══════════════════════════════════════════════════════════════════════╝
Total output: ~200 tokens
Total per email: ~1,000 tokens
What Happens AFTER AI (No AI Involved)
AI Output (JSON)
│
▼
Zod Validation ──── FAIL? → log warning, retry with next model, or flag as needs_review
│
✅ PASS
│
▼
Branch Routing ──── BRANCH_MAPPING["montreal.finances@iccameriques.org"] → "ICC Montréal"
│ (pure JavaScript lookup, no AI)
│
▼
PostgreSQL INSERT ── Drizzle ORM → INSERT INTO transactions (...)
│
▼
WebSocket Push ──── Socket.io → transaction:new { ... }
│
▼
React Dashboard ─── TanStack Table auto-adds row to table
│
▼
User Sees New Row ── "Jean Tremblay | 150,00 $ | ICC Montréal | Déposé"
6. Speed Optimization — Why It's Fast
Old Architecture (Sequential)
Email 1: [Fetch 100ms] → [AI Parse 300ms] → [Save 5ms] = 405ms
Email 2: [Fetch] → [Parse] → [Save]
Email 3: [Fetch] → ...
Total for 50 emails: 50 × 405ms = 20,250ms = ~20 seconds
New Architecture (Parallel Pipeline)
Time → 0ms 100ms 200ms 300ms 400ms 500ms
┌──────┬──────┬──────┬──────┬──────┬──────┐
Fetch: │E1-E10│E11-20│E21-30│E31-40│E41-50│ │ (10 concurrent)
Parse: │ │P1-P15│P16-30│P31-45│P46-50│ │ (15 concurrent)
Save: │ │ │S1-S25│ │S26-50│ │ (batch 25)
└──────┴──────┴──────┴──────┴──────┴──────┘
Total for 50 emails: ~500ms for fetches + ~700ms for parses = ~1.2 seconds
(stages overlap, so total < sum)
Concurrency Budget Per Free Tier
GROQ MISTRAL
RPM (requests/minute): 30 2
Safe concurrency: 15 concurrent calls 1 sequential call
Time per AI call: ~150ms (Groq is fast) ~400ms
Emails parsed per minute: ~30 (RPM-limited) ~2 (RPM-limited)
Emails parsed per hour: ~1,800 ~120
WITH AUTO-SWITCHER (all 6 Groq + 3 Mistral):
Effective RPM: 30 + 30 + 30 + 30 + 30 + 30 + 2 + 2 + 2 = 186 RPM*
(*until individual daily limits hit)
First hour throughput: ~180 emails/min = ~10,800 emails/hour
Speed Benchmarks (Revised with Pipeline)
| Scan Type | Emails | Time (Pipeline + Auto-Switcher) |
|---|---|---|
| Today | 50 | ~12 seconds |
| 7 Days | 200 | ~35 seconds |
| Custom (1 month) | 1,000 | ~3 minutes |
| Custom (6 months) | 5,000 | ~15 minutes |
| Custom (1 year) | 10,000 | ~30 minutes |
7. Non-AI Technology Flows
PDF Receipt Generation (No AI)
User clicks "Générer reçu" on a transaction row
│
▼
CLIENT-SIDE (single receipt):
@react-pdf/renderer builds PDF in browser
Template: ICC logo + transaction details + branch info + date
Downloads immediately as "recu_CA1b2c3d4e5f.pdf"
SERVER-SIDE (batch receipts):
POST /api/receipts/batch { transactionIds: [...] }
│
▼
PDFKit generates PDF per transaction
Merges into single ZIP file
Returns ZIP for download
CSV/Excel Export (No AI)
User clicks "Exporter" → selects CSV or Excel
│
▼
CLIENT-SIDE:
SheetJS (xlsx npm) builds file from TanStack Table data
Columns: Date | Expéditeur | Montant | Référence | Succursale | Statut
fr-CA formatting: "1 500,00 $" not "$1,500.00"
Downloads immediately
Reports & Charts (No AI)
User navigates to /reports
│
▼
GET /api/transactions/stats?from=2026-01-01&to=2026-02-23
│
▼
PostgreSQL aggregation queries:
- SUM(amount) GROUP BY month → bar chart (Recharts)
- COUNT(*) GROUP BY status → pie chart (Recharts)
- COUNT(*) GROUP BY branch → top branches table
- SUM(amount) over time → trend line chart (Recharts)
│
▼
React renders charts with Recharts
No AI involved — pure SQL aggregation + charting library
Real-Time Dashboard Updates (No AI)
Server saves new transaction to PostgreSQL
│
▼
Socket.io server emits:
transaction:new { transaction: {...} }
│
▼
Socket.io client receives event
│
▼
Zustand store updates transaction list
│
▼
TanStack Table re-renders with new row
(animated row insertion at top of table)
│
▼
SummaryBar recalculates totals
(no page refresh, no API call needed)
8. Security Flow (No AI)
1. User clicks "Se connecter avec Google"
2. Frontend redirects to Google OAuth consent URL
(scopes: gmail.readonly, userinfo.email, userinfo.profile)
3. User approves → Google redirects to /api/auth/google/callback?code=xxx
4. Backend exchanges auth code for access_token + refresh_token
(google-auth-library)
5. Backend encrypts tokens with AES-256 (crypto module)
6. Backend stores encrypted tokens in PostgreSQL (users table)
7. Backend issues JWT: { userId, email, role, exp: 15min }
(jsonwebtoken)
8. Frontend stores JWT in httpOnly secure cookie
9. Every API request includes JWT in Authorization header
10. Backend middleware verifies JWT on every request
(jsonwebtoken.verify)
11. If JWT expired → frontend calls POST /api/auth/refresh
with refresh token → new JWT issued
This document is a companion to the ICC Interac Manager Build Prompt and the FREE AI Models Guide. Together, the three documents provide everything needed to build the complete system.