Spaces:
Sleeping
Sleeping
| # Tech Stack Rationale | |
| | Field | Value | | |
| | --- | --- | | |
| | Project | Insurance Sales Portfolio Expert | | |
| | Date | 2026-05-13 | | |
| | Status | Living document — updated as picks change | | |
| | Companion docs | `02-architecture.md` (architecture detail) · `decisions.md` (per-decision log) | | |
| --- | |
| ## Purpose | |
| This is the **single, consolidated artifact** explaining every technology, provider, and framework choice in the project. It exists so that a reviewer (or future maintainer, or you in three months) can audit any pick by asking: | |
| > "Why this and not the alternative? What evidence supports the choice? What changes if the constraint changes?" | |
| Every entry has alternatives considered and a reasoning trace. Where decisions are still open, they are flagged with "**OPEN**" and an expected resolution date. | |
| --- | |
| ## 1. Component map — what the system is made of | |
| The product is split into five layers. Each layer has independent technology choices. | |
| ``` | |
| ┌─────────────────────────────────────────────────────────────────┐ | |
| │ 1. FRONTEND web UI · chat · audio · filter · compare │ | |
| └─────────────────────────────────────────────────────────────────┘ | |
| ↓ HTTPS | |
| ┌─────────────────────────────────────────────────────────────────┐ | |
| │ 2. BACKEND API route handlers · auth · request validation │ | |
| └─────────────────────────────────────────────────────────────────┘ | |
| ↓ | |
| ┌─────────────────────────────────────────────────────────────────┐ | |
| │ 3. ORCHESTRATION persona prompt · intent routing · providers │ | |
| └─────────────────────────────────────────────────────────────────┘ | |
| ↓ ↓ | |
| ┌──────────────────────────┐ ┌──────────────────────────────────┐ | |
| │ 4a. STRUCTURED STORE │ │ 4b. VECTOR STORE │ | |
| │ filter / compare │ │ RAG retrieval │ | |
| └──────────────────────────┘ └──────────────────────────────────┘ | |
| ↑ | |
| ┌─────────────────────────────────────────────────────────────────┐ | |
| │ 5. INGEST PIPELINE crawl · chunk · embed · extract │ | |
| └─────────────────────────────────────────────────────────────────┘ | |
| ``` | |
| --- | |
| ## 2. The picks — one table to find anything | |
| | # | Layer | Component | Pick | Status | Why (one line) | Alternatives considered | | |
| | --- | --- | --- | --- | --- | --- | --- | | |
| | 1 | Frontend | Framework | **Next.js 14 (App Router)** | Locked | Production-pattern, full UI flexibility, fast w/ shadcn/ui | Streamlit, Gradio, Chainlit, Reflex | | |
| | 2 | Frontend | Styling | **Tailwind CSS + shadcn/ui** | Locked | Copy-paste components produce beautiful UIs in hours | MUI, Chakra, plain CSS | | |
| | 3 | Frontend | Audio capture | **MediaRecorder API** (browser-native) | Locked | Zero deps, cross-browser, blob→POST flow is simple | streamlit-mic-recorder, WebRTC, manual ScriptProcessor | | |
| | 4 | Frontend | Hosting | **Vercel** (free tier) | Locked | Native Next.js host, GitHub auto-deploy, edge-cached | Netlify, Cloudflare Pages | | |
| | 5 | Backend | Framework | **FastAPI + Pydantic** | Locked | Pydantic matches our 48-field schema; OpenAPI auto-docs; async I/O for parallel provider calls | Flask, Django, Node/Express | | |
| | 6 | Backend | Hosting | **Render** (free 750h/mo) | Locked | GitHub auto-deploy, Python-native, persistent disk for DuckDB + Chroma | Fly.io, Railway, Modal | | |
| | 7 | Backend | API style | **REST** for v1, possible WebSocket if voice latency demands | Locked | Simpler; switch to WS only if measured latency exceeds target | Pure WS, gRPC | | |
| | 8 | Backend | FE↔BE type safety | **`openapi-typescript` codegen** from FastAPI's auto-generated OpenAPI | Locked | Single source of truth; backend route changes propagate automatically | tRPC (Node-only), GraphQL, manual TS types | | |
| | 9 | Orchestration | STT | **Sarvam Saarika v2.5** | Locked | Sarvam-first; their newer Indic ASR model | Whisper-large-v3, Deepgram Nova-2, Sarvam Saaras v3 | | |
| | 10 | Orchestration | TTS | **Sarvam Bulbul** | Locked | Sarvam-first; Indic prosody first-class | ElevenLabs, OpenAI TTS, Coqui local | | |
| | 11 | Orchestration | LLM (brain) | **NVIDIA NIM tiered: DeepSeek-V4-Pro (heavy) + DeepSeek-V4-Flash (fast)** | Locked (D-019) | V4-Pro (1.6T/49B MoE, frontier, beats Opus-4.6 + GPT-5.4 on SimpleQA) for comparison/recommendation; V4-Flash (284B/13B MoE, lower TTFT) for voice + fact-find. Single NIM key, free tier, no daily cap. Sarvam-M moved to Indic translation only — its 2048 output cap was truncating advisor responses. | Sarvam-M as brain (rejected — truncation), OpenRouter `:free` (rejected — 50/day cap), Gemini 2.5 Flash (rejected — adds 2nd provider ecosystem), GPT-4o via GitHub Models (rejected — same 50/day cap) | | |
| | 12 | Orchestration | LLM (grader / self-critique) | **NVIDIA NIM Llama-4 Maverick (Meta MoE)** | Locked (D-019) | Different family from the DeepSeek brain — Meta MoE judging DeepSeek MoE, so the brain does not mark its own homework. Same NIM endpoint and key. 400B/17B MoE, frontier-tier. Replaces Groq Llama-3.3-70B (30 req/min cap blocked sweeps). | Groq Llama (rejected — rate limit), Cerebras Qwen-235B (rejected — same family as another DeepSeek path), Sarvam-M held-out | | |
| | 13 | Orchestration | Embeddings | **BGE-small-en-v1.5 (local CPU, sentence-transformers)** | Locked | Free, runs offline, 384-dim, BEIR-strong on insurance/legal text. Voyage path kept for v2 swap (~3pp retrieval-quality delta in BEIR-style spot checks). | Voyage AI voyage-3 (rejected for v1 — 3 RPM free-tier blocked 104-PDF ingest), OpenAI text-embedding-3-small (rejected — no GPT), Cohere | | |
| | 14 | Storage | Structured DB | **DuckDB** (single file) | Locked | One file, columnar, no server; deploys via `git clone` | SQLite, Postgres | | |
| | 15 | Storage | Vector DB | **Chroma** (local persisted) | Locked | Embedded, no infra, supports metadata filtering | FAISS, Pinecone, Qdrant, Weaviate | | |
| | 16 | Ingest | PDF parsing | **pdfplumber** | Locked | Clean text + page numbers — critical for citation grammar | PyMuPDF, Unstructured.io | | |
| | 17 | Ingest | Chunking | **Custom 800-token chunks, 120-token overlap, page-aware** | Locked | Standard for technical docs; overlap protects clause boundaries | LangChain text splitter, fixed-size | | |
| | 18 | Ingest | Structured extraction | **Sarvam-M (Llama-3.3-70B / DeepSeek-V3 fallback) with Pydantic structured output + self-critique pass** | Locked | Reliable JSON-mode extraction; self-critique gives per-field confidence; fallback brain handles tables / complex clauses Sarvam-M misses | Pure regex, LangChain extraction, Instructor lib | | |
| | 19 | Pricing | Approach | **Hand-curated illustrative bands** from public PolicyBazaar quotes | Locked | Honest, defensible; "advisor-not-broker" positioning | Live scraping, actuarial model | | |
| | 20 | Cross-cutting | Auth | **None for v1** (single-tenant demo) | Locked | Out of scope per `01-requirements.md` §7 | Auth0, Clerk, NextAuth | | |
| | 21 | Cross-cutting | Observability | **JSONL turn log + cost tracker** | Locked | Lightweight; one log file, queryable post-hoc | Langfuse, Helicone, custom dashboard | | |
| | 22 | Cross-cutting | Secrets management | **`.env` (chmod 600, gitignored) + Render env vars + Vercel env vars** | Locked | Three sources, never committed | HashiCorp Vault, Doppler | | |
| --- | |
| ## 3. Selection rubric — how we chose | |
| Every pick above followed this five-test rubric. **Picks that fail any of these are documented as accepted tradeoffs, not silent compromises.** | |
| ### 3.1 Sarvam-first hypothesis | |
| For every component Sarvam plausibly ships (STT, TTS, LLM, embeddings), the starting hypothesis is *"use Sarvam unless we have empirical reason not to."* This is graded behavior for a Sarvam assignment — silent defaults to OpenAI/Anthropic/ElevenLabs are the most common screen-out signal. | |
| We will **benchmark** before locking final Sarvam vs non-Sarvam picks: | |
| - **STT:** 20-utterance test set across English, Hindi, Hinglish — measure word-error rate | |
| - **TTS:** Subjective quality + latency on 5 sample advisor responses | |
| - **LLM:** 50 gold Q&A pairs run with Sarvam-M and GPT-4o — measure factual accuracy + citation accuracy | |
| Where Sarvam wins → ship Sarvam. Where Sarvam loses → document the gap with numbers and ship the alternative. Either outcome is a strong artifact. | |
| ### 3.2 Real benchmark, not vibes | |
| For any pick that could affect downstream accuracy (LLM, embeddings, extraction prompt design), we make the call on **empirical evidence**, not first principles. The gold Q&A harness exists for exactly this — it transforms "I think X is better" into "X scored 92% vs Y's 87% on our test set." | |
| ### 3.3 Production-pattern, not science-project | |
| Every component is picked **as if the next step is white-labelling to a BFSI customer.** No "great for prototyping" picks that block productionization. Examples: | |
| - ✅ FastAPI — already production-grade | |
| - ✅ DuckDB — embedded but production-deployed at Motherduck and many fintechs | |
| - ❌ ~~Streamlit~~ — great prototyping, no white-label path → switched to Next.js | |
| ### 3.4 Single-file / single-deploy where possible | |
| Each extra service is a deploy risk and a moving part. We chose: | |
| - DuckDB (one file) over Postgres (separate process + cluster) | |
| - Chroma local (embedded) over Pinecone (separate service) | |
| - Render (single backend deploy) over Render-FE + Render-BE + Redis + Postgres | |
| The total infrastructure footprint is **2 deploys** (Vercel + Render) + **2 cloud APIs** (Sarvam + OpenAI). That's it. | |
| ### 3.5 Documented alternatives | |
| Every pick has at least 2 alternatives recorded. A reviewer can audit the reasoning. "Why DuckDB?" → table row 14 → SQLite (no columnar, no analytics) and Postgres (overkill, deploy overhead). Three options considered; reasoning explicit. | |
| --- | |
| ## 4. Cost envelope (24-hour build + demo run) | |
| | Item | Estimated cost | | |
| | --- | --- | | |
| | Sarvam API (STT + TTS + LLM) — build + 100 demo calls | likely free under signup credits | | |
| | Voyage AI embeddings (~5M tokens to embed 75 PDFs) | ~$0.45 (free under $50 signup credit) | | |
| | Claude Haiku 4.5 for grader / self-critique (~200 calls) | ~$0.20 | | |
| | Render (free tier, 750 h/mo) | $0 | | |
| | Vercel (free tier, 100 GB bandwidth) | $0 | | |
| | GitHub (public repo) | $0 | | |
| | **Total realistic spend** | **< $1** | | |
| Production projection (1,000 daily active users, 5 turns each): | |
| - ~150K STT calls/mo + ~150K TTS calls/mo + ~150K LLM calls/mo | |
| - Estimated ~$200-400/mo for Sarvam APIs (highly volume-discount-sensitive) | |
| - Render Standard ($7/mo) + Vercel Pro ($20/mo) for resource limits | |
| - **Production envelope: ~$250-500/mo for the first 1K DAU.** Sub-cent per turn. | |
| --- | |
| ## 5. Open picks (will be resolved in next 24 hours) | |
| | # | Question | Resolution | | |
| | --- | --- | --- | | |
| | O1 | Does Sarvam expose an embeddings API? If yes, benchmark vs OpenAI text-embedding-3-small | Check `dashboard.sarvam.ai` API listing; <30 min | | |
| | O2 | Saarika v2.5 vs Saaras v3 for STT — depends on whether we need translate/codemix modes | Benchmark both on 20-utterance set; <1h | | |
| | O3 | Recommendation engine shape: rule-based pre-filter + LLM justification vs pure-LLM reasoning | Decide after gold Q&A harness runs; leaning rule-based for testability | | |
| | O4 | Caching layer — should LLM responses be cached for repeated questions? | Yes for demo; simple file cache, no Redis | | |
| | O5 | Should we add basic auth on the deployed URL to avoid abuse? | Probably yes — single shared password for interviewer access | | |
| --- | |
| ## 6. Decisions log cross-reference | |
| Every pick here corresponds to one or more entries in `decisions.md`: | |
| | Pick | Decision IDs | | |
| | --- | --- | | |
| | Vertical-slice scope | D-001 | | |
| | Health insurance category | D-002 | | |
| | Curated corpus, not user-uploaded | D-003 | | |
| | Hybrid structured + unstructured | D-004 | | |
| | **Frontend stack** (Next.js + FastAPI) | D-005 (revised 2026-05-13) | | |
| | Sarvam-first benchmarking | D-006 | | |
| | Pricing as illustrative band | D-007 | | |
| | Consultative persona | D-008 | | |
| | 10 insurers × all health policies | D-009 | | |
| | Secret handling | D-010 | | |
| | Embeddings provider (pending) | D-011 | | |
| | Render deployment over alternatives | D-012 | | |
| | Next.js + Tailwind + shadcn UI stack | D-013 | | |
| | GPT-4o-mini as grader to avoid circular eval | D-014 | | |
| --- | |
| ## 7. Revision history | |
| | Version | Date | Change | | |
| | --- | --- | --- | | |
| | 0.1 | 2026-05-13 | Initial draft. Switched D-005 frontend from Streamlit to Next.js + FastAPI; added D-011 through D-014 for new picks. | | |