SYSTEM_DESIGN.md

# SYSTEM DESIGN — Sidewalk FM

**A voice/text bookkeeping ledger + advisor for one named informal street trader.**
Build Small Hackathon 2026 · Backyard AI track. Design principle (from the winning-pattern research): **the demo must never break** — deterministic core, models enhance.

---

## 1. The user (named, specific)
**Mama Aisha** — a morning-market produce trader. Low-literacy-friendly, speaks in plain language ("bought 50 mangoes at 200 shillings each"), often offline / limited connectivity, needs to know *am I making a profit?* — not formal accounting.

## 2. High-level flow
```
  ┌─────────────┐   text/voice    ┌──────────────────────┐
  │  Trader     │ ──────────────► │  EXTRACTION          │
  │ (Gradio UI) │                 │  1) deterministic     │
  └─────────────┘                 │     regex/rules (1st) │
        ▲                         │  2) Nemotron LLM      │
        │ advice + ledger         │     backup (messy in) │
        │                         └──────────┬───────────┘
        │                                    │ structured txn {action,item,qty,price,ccy}
        │                         ┌──────────▼───────────┐
        │                         │  LEDGER STORE         │  JSON/SQLite (local, private)
        │                         └──────────┬───────────┘
        │                         ┌──────────▼───────────┐
        │                         │  MATH ENGINE (Python) │  revenue/cost/profit/margin
        │                         │  — never the LLM      │  (deterministic, exact)
        │                         └──────────┬───────────┘
        │            ┌───────────────────────▼───────────┐
        └────────────┤  ADVISORY                          │
                     │  Nemotron-Mini-4B on MODAL (vLLM)  │
                     │  fallback: deterministic canned    │
                     │  advice from the numbers           │
                     └────────────────────────────────────┘
```

## 3. Components
- **Input (Gradio):** text box now; voice (Nemotron 3 ASR) + TTS (Kokoro 82M) in Phase 2.
- **Extraction:** deterministic parser first (regex for `buy/sell/return qty item at price ccy`), grounded — every field traceable to the input. Small **Nemotron** LLM only as a backup when regex fails. (NOT `nemotron-parse` — that's image/OCR and 400s on text.)
- **Ledger store:** append-only transactions, local (privacy — "a trader's books are private"). JSON file or SQLite.
- **Math engine:** pure Python — totals, per-item margin, daily profit. Deterministic so numbers are always correct and the demo never breaks.
- **Advisory:** `nvidia/Nemotron-Mini-4B-Instruct` on Modal (vLLM 0.19.0, L4) produces short, actionable advice from the computed numbers. If the endpoint is unavailable, a rule-based advisor returns canned guidance from the same numbers.

## 4. Transaction schema
```json
{ "action": "buy|sell|return|refund", "item": "string", "quantity": number,
  "unit_price": number, "currency": "string", "vendor": "string|null",
  "notes": "string|null", "ts": "ISO-8601" }
```

## 5. Model stack (all ≤4B → Tiny Titan; all NVIDIA → Nemotron prize)
| Component | Model | Params | Provider | Status |
|---|---|---|---|---|
| Extraction backup | Nemotron (small instruct) | ≤4B | NIM/Modal | optional |
| **Advisory** | **nvidia/Nemotron-Mini-4B-Instruct** | 4B | **Modal (L4, vLLM 0.19.0)** | core |
| *(opt)* ASR | Nemotron 3 ASR | 0.6B | — | Phase 2 |
| *(opt)* TTS | Kokoro | 82M | — | Phase 2 |
| *(opt)* Receipt OCR | MiniCPM-V 4.6 | ~1.3B | OpenBMB | Phase 2 (OpenBMB prize) |

## 6. Serving (Modal — also the 10k-credit prize)
- App `sidewalk-fm-advisor`; endpoint `https://joshua-abok--sidewalk-fm-advisor-serve.modal.run/v1`.
- vLLM 0.19.0 OpenAI-compatible, L4 single GPU, `min_containers=1` (stays warm for the demo), `max-model-len 8192`.
- README must note Modal-at-runtime to qualify for the Modal prize.

## 7. Reliability / graceful degradation (the differentiator)
Every model call has a deterministic fallback:
- Extraction: regex first → LLM only if regex fails → if LLM down, ask the user to rephrase.
- Math: always Python (no dependency).
- Advisory: Modal LLM → if down, rule-based advice from the numbers.
Result: **the core ledger + math + basic advice work with zero external calls** → "Off the Grid"-leaning, demo-safe.

## 8. Prize → feature mapping
Backyard AI (named trader) · Modal (advisory runtime) · NVIDIA (Nemotron) · Tiny Titan (≤4B) · Best Agent (parse→ledger→analyze→advise) · Off Brand (custom UI, P2) · Best Demo (video+social, P2) · Bonus Quest Champion (stack) · OpenBMB (MiniCPM receipt, P2) · OpenAI Codex (Codex commits, P2).

## 9. Out of scope (post-hackathon)
Fine-tuned classifier, published vendor-dialogue dataset, revenue "garden" visualization, multi-agent "market council", Android/GGUF edge build.