README.md

metadata

title: Sidewalk FM
sdk: gradio
tags:
  - sidewalk-fm
  - gradio
  - hackathon
  - build-small
  - tiny-titan
  - backyard-ai
  - best-agent
  - off-brand
  - best-demo
  - nvidia-nim
  - modal
  - well-tuned
  - swahili
  - multilingual
license: mit

Hackathon Entry
HuggingFace Build Small Hackathon 2026 — Backyard AI track
🏆 Tiny Titan + Best Agent + Off Brand + Best Demo + Well-Tuned + NVIDIA Sponsor + Modal

For a real person: Mama Aisha, a morning-market produce trader who speaks Kiswahili and English. She logs sales by voice in her own language and hears whether she's making a profit — no typing, no accountant.

"What's your biggest expense today?"
"I spent 500 shillings on mangoes, but I don't know if I'm making a profit."

Sidewalk FM helps small traders track purchases, sales, and margins in plain English.
It's for the person who runs a shop on the street — not the accountant.

---

🧠 Models Used

Role	Model	Size	Provider	Badge Eligible
Extraction	nvidia/nemotron-mini-4b-instruct	4B	NVIDIA NIM	✅ Tiny Titan
Advisory	nvidia/nemotron-mini-4b-instruct	4B	Modal vLLM (T4) → deterministic fallback	✅ Tiny Titan + Modal
Voice — Kiswahili	Joshua-Abok/finetuning-wav2vec-large-swahili-asr-model_v12	~1B (our fine-tune, WER 14.36%)	HF Inference	✅ Tiny Titan + Well-Tuned
Voice — English / code-switch	openai/whisper-large-v3	—	HF Inference	multilingual fallback

Every model ≤4B → Tiny Titan. We fine-tuned and published the Kiswahili ASR model → Well-Tuned. Advisory served on Modal → Modal prize. NVIDIA Nemotron throughout → NVIDIA prize.

🌍 Why this wins for a real trader (Backyard AI)

Mama Aisha can't type fast and isn't an accountant — but she speaks. Sidewalk FM is voice-first and bilingual (English + Kiswahili), using a Swahili model we fine-tuned ourselves so it actually understands a Nairobi market trader, not generic English. The ledger and profit math are deterministic Python (never the LLM), so the numbers are always right and the demo never breaks.

---

🧩 Features

• Natural Language Input: "bought 50 mangoes at 200 shillings each"
• Bilingual Voice Input: Click the mic 🎤 and speak in English or Kiswahili — Swahili uses our fine-tuned wav2vec2 model, English/code-switch uses Whisper, gated so it never breaks
• Structured Ledger: Transactions parsed and stored with full metadata
• Deterministic Math: Revenue, costs, profit calculated with pure Python — never the LLM
• AI Advisory: Get actionable advice on inventory and pricing via NVIDIA NIM or Modal vLLM
• Custom HTML Ledger UI: Styled tables with color-coded margins (Off Brand badge)
• Structured Agent Traces: Every extraction, validation, and advisory step is logged for audit 📡
• Voice-to-Ledger: Full text flow — log a sale → see margin → get advice — all in one demo

---

📦 Deployment

HF Space (Gradio)

The Gradio app (app.py) uses NVIDIA NIM for extraction and advisory.

pip install -r requirements.txt
python app.py

Deploy to HF Spaces

1. Push to a repo (this one)
2. Create Space with gradio runtime
3. Add API keys to Secrets: NVIDIA_NIM_API_KEY

---

🖥 Modal vLLM (Optional — On-Prem Advisory)

For a hosted advisory model on Modal GPU, use the Modal deployment:

modal deploy modal_serve_qwen3b.py

Hard Deployment Constraints

1. GPU: L4 — Nemotron 4B with comfortable VRAM headroom
2. Only 1 GPU — no multi-GPU setups
3. min_containers=1 — endpoint stays warm, never scales to 0

API Endpoints

• GET /health — Model health check
• GET /v1/models — Available models
• POST /v1/chat/completions — OpenAI-compatible chat completions

---

📊 Architecture

User Input (natural language)
        │
        ▼
┌──────────────────┐
│  Deterministic   │ ← Regex/rules: fast, reliable, no hallucination
│  Extraction      │ ← "bought 50 mangoes at 200 shillings"
└────────┬─────────┘
         │
    [regex failed?]
         │
         ▼
┌──────────────────┐
│  NIM Fallback    │ ← nemotron-mini-4b-instruct for messy input
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│   Validation     │ ← Hard rules: qty > 0, action in {buy,sell,return}
│   & Ledger       │ ← Pure Python struct, deterministic
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│  Deterministic   │ ← calculate_margins() — never the LLM
│  Math Engine     │ ← revenue, costs, profit, margin%
└────────┬─────────┘
         │
    ┌────┴────┐
    │         │
    ▼         ▼
┌──────┐  ┌──────────┐
│ Rule │  │  Modal   │
│ Based│  │  vLLM    │
│Advice│  │ Advisory │
└──────┘  └──────────┘

---

🎯 Badges Targeted

Badge	Status
Backyard AI	✅ Track real-world problems (street traders)
Tiny Titan	✅ FULL — all models ≤4B (nemotron-mini-4b-instruct)
Best Agent	✅ Multi-step tool use (parse → validate → ledger → analyze → advise)
Off Brand	✅ Custom ledger UI (not a chatbot)
Best Demo	✅ Story + video + social post
Well-Tuned	✅ Our fine-tuned & published Kiswahili wav2vec2 model (WER 14.36%)
NVIDIA Sponsor	✅ Full NVIDIA stack (Nemotron Mini 4B on NIM + Modal)
Modal — Best Use	✅ Advisory served on Modal (vLLM) at runtime

---

📡 Structured Agent Traces

Every action in the pipeline is logged with structured traces:

Step	Fields
extraction	method (regex/llm), success, fields
validation	success, error
advisory	method (rule/nim/modal), success, error

This provides full auditability — judges can trace exactly how every answer was reached.

---

links

https://www.linkedin.com/posts/joshuaabok_buildsmallhackathon-buildsmall-huggingface-share-7472437382152179712-Q3jX/?utm_source=share&utm_medium=member_desktop&rcm=ACoAAF92zkABF9X9x72P-GywndHGRVJCSFAjFL8

https://youtu.be/uzu377Yq_EM

📝 License

MIT