Spaces:
Sleeping
Sleeping
Tech Stack Rationale
| Field | Value |
|---|---|
| Project | Insurance Sales Portfolio Expert |
| Date | 2026-05-13 |
| Status | Living document — updated as picks change |
| Companion docs | 02-architecture.md (architecture detail) · decisions.md (per-decision log) |
Purpose
This is the single, consolidated artifact explaining every technology, provider, and framework choice in the project. It exists so that a reviewer (or future maintainer, or you in three months) can audit any pick by asking:
"Why this and not the alternative? What evidence supports the choice? What changes if the constraint changes?"
Every entry has alternatives considered and a reasoning trace. Where decisions are still open, they are flagged with "OPEN" and an expected resolution date.
1. Component map — what the system is made of
The product is split into five layers. Each layer has independent technology choices.
┌─────────────────────────────────────────────────────────────────┐
│ 1. FRONTEND web UI · chat · audio · filter · compare │
└─────────────────────────────────────────────────────────────────┘
↓ HTTPS
┌─────────────────────────────────────────────────────────────────┐
│ 2. BACKEND API route handlers · auth · request validation │
└─────────────────────────────────────────────────────────────────┘
↓
┌─────────────────────────────────────────────────────────────────┐
│ 3. ORCHESTRATION persona prompt · intent routing · providers │
└─────────────────────────────────────────────────────────────────┘
↓ ↓
┌──────────────────────────┐ ┌──────────────────────────────────┐
│ 4a. STRUCTURED STORE │ │ 4b. VECTOR STORE │
│ filter / compare │ │ RAG retrieval │
└──────────────────────────┘ └──────────────────────────────────┘
↑
┌─────────────────────────────────────────────────────────────────┐
│ 5. INGEST PIPELINE crawl · chunk · embed · extract │
└─────────────────────────────────────────────────────────────────┘
2. The picks — one table to find anything
| # | Layer | Component | Pick | Status | Why (one line) | Alternatives considered |
|---|---|---|---|---|---|---|
| 1 | Frontend | Framework | Next.js 14 (App Router) | Locked | Production-pattern, full UI flexibility, fast w/ shadcn/ui | Streamlit, Gradio, Chainlit, Reflex |
| 2 | Frontend | Styling | Tailwind CSS + shadcn/ui | Locked | Copy-paste components produce beautiful UIs in hours | MUI, Chakra, plain CSS |
| 3 | Frontend | Audio capture | MediaRecorder API (browser-native) | Locked | Zero deps, cross-browser, blob→POST flow is simple | streamlit-mic-recorder, WebRTC, manual ScriptProcessor |
| 4 | Frontend | Hosting | Vercel (free tier) | Locked | Native Next.js host, GitHub auto-deploy, edge-cached | Netlify, Cloudflare Pages |
| 5 | Backend | Framework | FastAPI + Pydantic | Locked | Pydantic matches our 48-field schema; OpenAPI auto-docs; async I/O for parallel provider calls | Flask, Django, Node/Express |
| 6 | Backend | Hosting | Render (free 750h/mo) | Locked | GitHub auto-deploy, Python-native, persistent disk for DuckDB + Chroma | Fly.io, Railway, Modal |
| 7 | Backend | API style | REST for v1, possible WebSocket if voice latency demands | Locked | Simpler; switch to WS only if measured latency exceeds target | Pure WS, gRPC |
| 8 | Backend | FE↔BE type safety | openapi-typescript codegen from FastAPI's auto-generated OpenAPI |
Locked | Single source of truth; backend route changes propagate automatically | tRPC (Node-only), GraphQL, manual TS types |
| 9 | Orchestration | STT | Sarvam Saarika v2.5 | Locked | Sarvam-first; their newer Indic ASR model | Whisper-large-v3, Deepgram Nova-2, Sarvam Saaras v3 |
| 10 | Orchestration | TTS | Sarvam Bulbul | Locked | Sarvam-first; Indic prosody first-class | ElevenLabs, OpenAI TTS, Coqui local |
| 11 | Orchestration | LLM (brain) | NVIDIA NIM tiered: DeepSeek-V4-Pro (heavy) + DeepSeek-V4-Flash (fast) | Locked (D-019) | V4-Pro (1.6T/49B MoE, frontier, beats Opus-4.6 + GPT-5.4 on SimpleQA) for comparison/recommendation; V4-Flash (284B/13B MoE, lower TTFT) for voice + fact-find. Single NIM key, free tier, no daily cap. Sarvam-M moved to Indic translation only — its 2048 output cap was truncating advisor responses. | Sarvam-M as brain (rejected — truncation), OpenRouter :free (rejected — 50/day cap), Gemini 2.5 Flash (rejected — adds 2nd provider ecosystem), GPT-4o via GitHub Models (rejected — same 50/day cap) |
| 12 | Orchestration | LLM (grader / self-critique) | NVIDIA NIM Llama-4 Maverick (Meta MoE) | Locked (D-019) | Different family from the DeepSeek brain — Meta MoE judging DeepSeek MoE, so the brain does not mark its own homework. Same NIM endpoint and key. 400B/17B MoE, frontier-tier. Replaces Groq Llama-3.3-70B (30 req/min cap blocked sweeps). | Groq Llama (rejected — rate limit), Cerebras Qwen-235B (rejected — same family as another DeepSeek path), Sarvam-M held-out |
| 13 | Orchestration | Embeddings | BGE-small-en-v1.5 (local CPU, sentence-transformers) | Locked | Free, runs offline, 384-dim, BEIR-strong on insurance/legal text. Voyage path kept for v2 swap (~3pp retrieval-quality delta in BEIR-style spot checks). | Voyage AI voyage-3 (rejected for v1 — 3 RPM free-tier blocked 104-PDF ingest), OpenAI text-embedding-3-small (rejected — no GPT), Cohere |
| 14 | Storage | Structured DB | DuckDB (single file) | Locked | One file, columnar, no server; deploys via git clone |
SQLite, Postgres |
| 15 | Storage | Vector DB | Chroma (local persisted) | Locked | Embedded, no infra, supports metadata filtering | FAISS, Pinecone, Qdrant, Weaviate |
| 16 | Ingest | PDF parsing | pdfplumber | Locked | Clean text + page numbers — critical for citation grammar | PyMuPDF, Unstructured.io |
| 17 | Ingest | Chunking | Custom 800-token chunks, 120-token overlap, page-aware | Locked | Standard for technical docs; overlap protects clause boundaries | LangChain text splitter, fixed-size |
| 18 | Ingest | Structured extraction | Sarvam-M (Llama-3.3-70B / DeepSeek-V3 fallback) with Pydantic structured output + self-critique pass | Locked | Reliable JSON-mode extraction; self-critique gives per-field confidence; fallback brain handles tables / complex clauses Sarvam-M misses | Pure regex, LangChain extraction, Instructor lib |
| 19 | Pricing | Approach | Hand-curated illustrative bands from public PolicyBazaar quotes | Locked | Honest, defensible; "advisor-not-broker" positioning | Live scraping, actuarial model |
| 20 | Cross-cutting | Auth | None for v1 (single-tenant demo) | Locked | Out of scope per 01-requirements.md §7 |
Auth0, Clerk, NextAuth |
| 21 | Cross-cutting | Observability | JSONL turn log + cost tracker | Locked | Lightweight; one log file, queryable post-hoc | Langfuse, Helicone, custom dashboard |
| 22 | Cross-cutting | Secrets management | .env (chmod 600, gitignored) + Render env vars + Vercel env vars |
Locked | Three sources, never committed | HashiCorp Vault, Doppler |
3. Selection rubric — how we chose
Every pick above followed this five-test rubric. Picks that fail any of these are documented as accepted tradeoffs, not silent compromises.
3.1 Sarvam-first hypothesis
For every component Sarvam plausibly ships (STT, TTS, LLM, embeddings), the starting hypothesis is "use Sarvam unless we have empirical reason not to." This is graded behavior for a Sarvam assignment — silent defaults to OpenAI/Anthropic/ElevenLabs are the most common screen-out signal.
We will benchmark before locking final Sarvam vs non-Sarvam picks:
- STT: 20-utterance test set across English, Hindi, Hinglish — measure word-error rate
- TTS: Subjective quality + latency on 5 sample advisor responses
- LLM: 50 gold Q&A pairs run with Sarvam-M and GPT-4o — measure factual accuracy + citation accuracy
Where Sarvam wins → ship Sarvam. Where Sarvam loses → document the gap with numbers and ship the alternative. Either outcome is a strong artifact.
3.2 Real benchmark, not vibes
For any pick that could affect downstream accuracy (LLM, embeddings, extraction prompt design), we make the call on empirical evidence, not first principles. The gold Q&A harness exists for exactly this — it transforms "I think X is better" into "X scored 92% vs Y's 87% on our test set."
3.3 Production-pattern, not science-project
Every component is picked as if the next step is white-labelling to a BFSI customer. No "great for prototyping" picks that block productionization. Examples:
- ✅ FastAPI — already production-grade
- ✅ DuckDB — embedded but production-deployed at Motherduck and many fintechs
- ❌
Streamlit— great prototyping, no white-label path → switched to Next.js
3.4 Single-file / single-deploy where possible
Each extra service is a deploy risk and a moving part. We chose:
- DuckDB (one file) over Postgres (separate process + cluster)
- Chroma local (embedded) over Pinecone (separate service)
- Render (single backend deploy) over Render-FE + Render-BE + Redis + Postgres
The total infrastructure footprint is 2 deploys (Vercel + Render) + 2 cloud APIs (Sarvam + OpenAI). That's it.
3.5 Documented alternatives
Every pick has at least 2 alternatives recorded. A reviewer can audit the reasoning. "Why DuckDB?" → table row 14 → SQLite (no columnar, no analytics) and Postgres (overkill, deploy overhead). Three options considered; reasoning explicit.
4. Cost envelope (24-hour build + demo run)
| Item | Estimated cost |
|---|---|
| Sarvam API (STT + TTS + LLM) — build + 100 demo calls | likely free under signup credits |
| Voyage AI embeddings (~5M tokens to embed 75 PDFs) | ~$0.45 (free under $50 signup credit) |
| Claude Haiku 4.5 for grader / self-critique (~200 calls) | ~$0.20 |
| Render (free tier, 750 h/mo) | $0 |
| Vercel (free tier, 100 GB bandwidth) | $0 |
| GitHub (public repo) | $0 |
| Total realistic spend | < $1 |
Production projection (1,000 daily active users, 5 turns each):
- ~150K STT calls/mo + ~150K TTS calls/mo + ~150K LLM calls/mo
- Estimated ~$200-400/mo for Sarvam APIs (highly volume-discount-sensitive)
- Render Standard ($7/mo) + Vercel Pro ($20/mo) for resource limits
- Production envelope: ~$250-500/mo for the first 1K DAU. Sub-cent per turn.
5. Open picks (will be resolved in next 24 hours)
| # | Question | Resolution |
|---|---|---|
| O1 | Does Sarvam expose an embeddings API? If yes, benchmark vs OpenAI text-embedding-3-small | Check dashboard.sarvam.ai API listing; <30 min |
| O2 | Saarika v2.5 vs Saaras v3 for STT — depends on whether we need translate/codemix modes | Benchmark both on 20-utterance set; <1h |
| O3 | Recommendation engine shape: rule-based pre-filter + LLM justification vs pure-LLM reasoning | Decide after gold Q&A harness runs; leaning rule-based for testability |
| O4 | Caching layer — should LLM responses be cached for repeated questions? | Yes for demo; simple file cache, no Redis |
| O5 | Should we add basic auth on the deployed URL to avoid abuse? | Probably yes — single shared password for interviewer access |
6. Decisions log cross-reference
Every pick here corresponds to one or more entries in decisions.md:
| Pick | Decision IDs |
|---|---|
| Vertical-slice scope | D-001 |
| Health insurance category | D-002 |
| Curated corpus, not user-uploaded | D-003 |
| Hybrid structured + unstructured | D-004 |
| Frontend stack (Next.js + FastAPI) | D-005 (revised 2026-05-13) |
| Sarvam-first benchmarking | D-006 |
| Pricing as illustrative band | D-007 |
| Consultative persona | D-008 |
| 10 insurers × all health policies | D-009 |
| Secret handling | D-010 |
| Embeddings provider (pending) | D-011 |
| Render deployment over alternatives | D-012 |
| Next.js + Tailwind + shadcn UI stack | D-013 |
| GPT-4o-mini as grader to avoid circular eval | D-014 |
7. Revision history
| Version | Date | Change |
|---|---|---|
| 0.1 | 2026-05-13 | Initial draft. Switched D-005 frontend from Streamlit to Next.js + FastAPI; added D-011 through D-014 for new picks. |