SYSTEM_DESIGN.md

SYSTEM DESIGN — Sidewalk FM

A voice/text bookkeeping ledger + advisor for one named informal street trader. Build Small Hackathon 2026 · Backyard AI track. Design principle (from the winning-pattern research): the demo must never break — deterministic core, models enhance.

---

1. The user (named, specific)

Mama Aisha — a morning-market produce trader. Low-literacy-friendly, speaks in plain language ("bought 50 mangoes at 200 shillings each"), often offline / limited connectivity, needs to know am I making a profit? — not formal accounting.

2. High-level flow

  ┌─────────────┐   text/voice    ┌──────────────────────┐
  │  Trader     │ ──────────────► │  EXTRACTION          │
  │ (Gradio UI) │                 │  1) deterministic     │
  └─────────────┘                 │     regex/rules (1st) │
        ▲                         │  2) Nemotron LLM      │
        │ advice + ledger         │     backup (messy in) │
        │                         └──────────┬───────────┘
        │                                    │ structured txn {action,item,qty,price,ccy}
        │                         ┌──────────▼───────────┐
        │                         │  LEDGER STORE         │  JSON/SQLite (local, private)
        │                         └──────────┬───────────┘
        │                         ┌──────────▼───────────┐
        │                         │  MATH ENGINE (Python) │  revenue/cost/profit/margin
        │                         │  — never the LLM      │  (deterministic, exact)
        │                         └──────────┬───────────┘
        │            ┌───────────────────────▼───────────┐
        └────────────┤  ADVISORY                          │
                     │  Nemotron-Mini-4B on MODAL (vLLM)  │
                     │  fallback: deterministic canned    │
                     │  advice from the numbers           │
                     └────────────────────────────────────┘

3. Components

• Input (Gradio): text box now; voice (Nemotron 3 ASR) + TTS (Kokoro 82M) in Phase 2.
• Extraction: deterministic parser first (regex for buy/sell/return qty item at price ccy), grounded — every field traceable to the input. Small Nemotron LLM only as a backup when regex fails. (NOT nemotron-parse — that's image/OCR and 400s on text.)
• Ledger store: append-only transactions, local (privacy — "a trader's books are private"). JSON file or SQLite.
• Math engine: pure Python — totals, per-item margin, daily profit. Deterministic so numbers are always correct and the demo never breaks.
• Advisory: nvidia/Nemotron-Mini-4B-Instruct on Modal (vLLM 0.19.0, L4) produces short, actionable advice from the computed numbers. If the endpoint is unavailable, a rule-based advisor returns canned guidance from the same numbers.

4. Transaction schema

{ "action": "buy|sell|return|refund", "item": "string", "quantity": number,
  "unit_price": number, "currency": "string", "vendor": "string|null",
  "notes": "string|null", "ts": "ISO-8601" }

5. Model stack (all ≤4B → Tiny Titan; all NVIDIA → Nemotron prize)

Component	Model	Params	Provider	Status
Extraction backup	Nemotron (small instruct)	≤4B	NIM/Modal	optional
Advisory	nvidia/Nemotron-Mini-4B-Instruct	4B	Modal (L4, vLLM 0.19.0)	core
(opt) ASR	Nemotron 3 ASR	0.6B	—	Phase 2
(opt) TTS	Kokoro	82M	—	Phase 2
(opt) Receipt OCR	MiniCPM-V 4.6	~1.3B	OpenBMB	Phase 2 (OpenBMB prize)

6. Serving (Modal — also the 10k-credit prize)

• App sidewalk-fm-advisor; endpoint https://joshua-abok--sidewalk-fm-advisor-serve.modal.run/v1.
• vLLM 0.19.0 OpenAI-compatible, L4 single GPU, min_containers=1 (stays warm for the demo), max-model-len 8192.
• README must note Modal-at-runtime to qualify for the Modal prize.

7. Reliability / graceful degradation (the differentiator)

Every model call has a deterministic fallback:

• Extraction: regex first → LLM only if regex fails → if LLM down, ask the user to rephrase.
• Math: always Python (no dependency).
• Advisory: Modal LLM → if down, rule-based advice from the numbers. Result: the core ledger + math + basic advice work with zero external calls → "Off the Grid"-leaning, demo-safe.

8. Prize → feature mapping

Backyard AI (named trader) · Modal (advisory runtime) · NVIDIA (Nemotron) · Tiny Titan (≤4B) · Best Agent (parse→ledger→analyze→advise) · Off Brand (custom UI, P2) · Best Demo (video+social, P2) · Bonus Quest Champion (stack) · OpenBMB (MiniCPM receipt, P2) · OpenAI Codex (Codex commits, P2).

9. Out of scope (post-hackathon)

Fine-tuned classifier, published vendor-dialogue dataset, revenue "garden" visualization, multi-agent "market council", Android/GGUF edge build.