README.md

---
title: Sidewalk FM
sdk: gradio
tags:
  - sidewalk-fm
  - gradio
  - hackathon
  - build-small
  - tiny-titan
  - backyard-ai
  - best-agent
  - off-brand
  - best-demo
  - nvidia-nim
  - modal
  - well-tuned
  - swahili
  - multilingual
license: mit
---

**Hackathon Entry**  
HuggingFace Build Small Hackathon 2026 — Backyard AI track  
🏆 Tiny Titan + Best Agent + Off Brand + Best Demo + Well-Tuned + NVIDIA Sponsor + Modal

**For a real person:** Mama Aisha, a morning-market produce trader who speaks Kiswahili and
English. She logs sales by *voice* in her own language and hears whether she's making a profit —
no typing, no accountant.

> "What's your biggest expense today?"  
> "I spent 500 shillings on mangoes, but I don't know if I'm making a profit."

**Sidewalk FM** helps small traders track purchases, sales, and margins in plain English.  
It's for the person who runs a shop on the street — not the accountant.  

---

## 🧠 Models Used

| Role | Model | Size | Provider | Badge Eligible |
|------|-------|------|----------|----------------|
| **Extraction** | `nvidia/nemotron-mini-4b-instruct` | 4B | NVIDIA NIM | ✅ Tiny Titan |
| **Advisory** | `nvidia/nemotron-mini-4b-instruct` | 4B | **Modal vLLM (T4)** → deterministic fallback | ✅ Tiny Titan + Modal |
| **Voice — Kiswahili** | [`Joshua-Abok/finetuning-wav2vec-large-swahili-asr-model_v12`](https://huggingface.co/Joshua-Abok/finetuning-wav2vec-large-swahili-asr-model_v12) | ~1B (our **fine-tune**, WER 14.36%) | HF Inference | ✅ Tiny Titan + **Well-Tuned** |
| **Voice — English / code-switch** | `openai/whisper-large-v3` | — | HF Inference | multilingual fallback |

Every model ≤4B → **Tiny Titan**. We **fine-tuned and published** the Kiswahili ASR model →
**Well-Tuned**. Advisory served on **Modal** → **Modal** prize. NVIDIA Nemotron throughout →
**NVIDIA** prize.

## 🌍 Why this wins for a real trader (Backyard AI)
Mama Aisha can't type fast and isn't an accountant — but she speaks. Sidewalk FM is **voice-first
and bilingual (English + Kiswahili)**, using a Swahili model **we fine-tuned ourselves** so it
actually understands a Nairobi market trader, not generic English. The ledger and profit math are
**deterministic Python** (never the LLM), so the numbers are always right and the demo never breaks.

---

## 🧩 Features

- **Natural Language Input:** "bought 50 mangoes at 200 shillings each"  
- **Bilingual Voice Input:** Click the mic 🎤 and speak in **English or Kiswahili** — Swahili uses our **fine-tuned** wav2vec2 model, English/code-switch uses Whisper, gated so it never breaks  
- **Structured Ledger:** Transactions parsed and stored with full metadata  
- **Deterministic Math:** Revenue, costs, profit calculated with pure Python — never the LLM  
- **AI Advisory:** Get actionable advice on inventory and pricing via NVIDIA NIM or Modal vLLM  
- **Custom HTML Ledger UI:** Styled tables with color-coded margins (Off Brand badge)  
- **Structured Agent Traces:** Every extraction, validation, and advisory step is logged for audit 📡
- **Voice-to-Ledger:** Full text flow — log a sale → see margin → get advice — all in one demo  

---

## 📦 Deployment

### HF Space (Gradio)

The Gradio app (`app.py`) uses NVIDIA NIM for extraction and advisory.

```bash
pip install -r requirements.txt
python app.py
```

### Deploy to HF Spaces

1. Push to a repo (this one)  
2. Create Space with `gradio` runtime  
3. Add API keys to Secrets: `NVIDIA_NIM_API_KEY`  

---

## 🖥 Modal vLLM (Optional — On-Prem Advisory)

For a hosted advisory model on Modal GPU, use the Modal deployment:

```bash
modal deploy modal_serve_qwen3b.py
```

### Hard Deployment Constraints

1. **GPU: L4** — Nemotron 4B with comfortable VRAM headroom
2. **Only 1 GPU** — no multi-GPU setups
3. **min_containers=1** — endpoint stays warm, never scales to 0

### API Endpoints

- `GET /health` — Model health check
- `GET /v1/models` — Available models
- `POST /v1/chat/completions` — OpenAI-compatible chat completions

---

## 📊 Architecture

```
User Input (natural language)
        │
        ▼
┌──────────────────┐
│  Deterministic   │ ← Regex/rules: fast, reliable, no hallucination
│  Extraction      │ ← "bought 50 mangoes at 200 shillings"
└────────┬─────────┘
         │
    [regex failed?]
         │
         ▼
┌──────────────────┐
│  NIM Fallback    │ ← nemotron-mini-4b-instruct for messy input
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│   Validation     │ ← Hard rules: qty > 0, action in {buy,sell,return}
│   & Ledger       │ ← Pure Python struct, deterministic
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│  Deterministic   │ ← calculate_margins() — never the LLM
│  Math Engine     │ ← revenue, costs, profit, margin%
└────────┬─────────┘
         │
    ┌────┴────┐
    │         │
    ▼         ▼
┌──────┐  ┌──────────┐
│ Rule │  │  Modal   │
│ Based│  │  vLLM    │
│Advice│  │ Advisory │
└──────┘  └──────────┘
```

---

## 🎯 Badges Targeted

| Badge | Status |
|-------|--------|
| **Backyard AI** | ✅ Track real-world problems (street traders) |
| **Tiny Titan** | ✅ FULL — all models ≤4B (nemotron-mini-4b-instruct) |
| **Best Agent** | ✅ Multi-step tool use (parse → validate → ledger → analyze → advise) |
| **Off Brand** | ✅ Custom ledger UI (not a chatbot) |
| **Best Demo** | ✅ Story + video + social post |
| **Well-Tuned** | ✅ Our **fine-tuned & published** Kiswahili wav2vec2 model (WER 14.36%) |
| **NVIDIA Sponsor** | ✅ Full NVIDIA stack (Nemotron Mini 4B on NIM + Modal) |
| **Modal — Best Use** | ✅ Advisory served on Modal (vLLM) at runtime |

---

## 📡 Structured Agent Traces

Every action in the pipeline is logged with structured traces:

| Step | Fields |
|------|--------|
| extraction | method (regex/llm), success, fields |
| validation | success, error |
| advisory | method (rule/nim/modal), success, error |

This provides full auditability — judges can trace exactly how every answer was reached.

---

## links
https://www.linkedin.com/posts/joshuaabok_buildsmallhackathon-buildsmall-huggingface-share-7472437382152179712-Q3jX/?utm_source=share&utm_medium=member_desktop&rcm=ACoAAF92zkABF9X9x72P-GywndHGRVJCSFAjFL8


https://youtu.be/uzu377Yq_EM

## 📝 License

MIT