Deploy BudgetBuddy update

FIELD_NOTES.md

@@ -1,94 +1,119 @@
-# Field Notes — Building BudgetBuddy on small models + Modal
+# Field Notes — building BudgetBuddy
 
-*Build-story / report for the Build Small hackathon "Field Notes" badge. Edit the
-voice to your own, add a couple of screenshots/GIFs, and (ideally) publish it on
-the Hugging Face blog or your own site, then link it from the Space.*
+*A build log for the Build Small hackathon. This is the honest version — the wrong
+turns included. Publish-ready; edit the voice to your own and add a screenshot or
+two before posting.*
 
 ---
 
-## The itch
-
-My family doesn't use a budgeting app. Not because they don't care about money —
-because every app wants you to either link your bank, type each expense like a
-data-entry clerk, or hand your receipts to some cloud. For a homemaker running a
-household on cash, UPI, and a pile of crumpled bills, none of that fits.
-
-So for the **Build Small** hackathon (Backyard AI track) I built **BudgetBuddy**:
-snap a bill, it reads it, and you can literally *ask it* where your money went.
-It runs on two **MiniCPM** models and no third-party AI API.
-
-## The shape of it
-
-- **MiniCPM-V-4.6 (1.3B, vision)** reads a receipt *or* a UPI/card screenshot
-  into structured JSON: vendor, date, line items, tax/service/tip/discount,
-  round-off, total. Runs on the Space's **ZeroGPU**.
-- **MiniCPM4.1-8B (8B, reasoning)** does two jobs: a *reason-and-repair* pass that
-  cleans up the extraction and categorises it, and a *tool-using agent* that
-  answers questions about your spending. Runs on **Modal**.
-- A custom dark dashboard (no stock Gradio) shows it all.
-
-~9.3B params total, comfortably under the hackathon's 32B cap.
-
-## Five things that bit me (so they don't bite you)
-
-**1. "4.6" is a version, not a parameter count.** I assumed MiniCPM-V-4.6 was an
-~8B model like the older 2.6/4.5 line and warned about GPU memory. It's actually
-**1.3B**. Always check `safetensors` metadata, not the name.
-
-**2. The model dropped the API I coded against.** MiniCPM-V's older `.chat()`
-method is gone in 4.6 — it's now a native `transformers` model
-(`AutoModelForImageTextToText` + `AutoProcessor` + `apply_chat_template`). I only
-found this by reading the repo's `config.json` and chat template, not from
-memory. Lesson: when wiring a model, read its *actual* files first.
-
-**3. Two models, one impossible dependency.** The vision model needs a modern
-`transformers` (≈5.7+); MiniCPM4.1-8B's `trust_remote_code` is written for
-`transformers` ≈4.56 and *RuntimeErrors* on 5.x. You cannot satisfy both in one
-environment. I burned hours trying to shim the 8B onto new `transformers` before
-admitting defeat. **The fix was Modal:** run the 8B in its *own* container with
-its own pinned `transformers`, expose a `generate()` method, and call it from the
-Space over the Modal SDK. One memory snapshot for fast cold starts, a warm pool
-so it stays hot, and the version war just… disappears. Isolating the awkward
-dependency on its own compute was the single highest-leverage decision.
-
-**4. Real dates aren't ISO, and literal OCR isn't "understanding."** My analytics
-silently dropped transactions because a bill said `01/07/17` and my parser only
-accepted `YYYY-MM-DD`. Separately, the vision model would extract "Subtotal",
-"Total" and "Amount Payable" *as line items*, so nothing reconciled. The fix was a
-tolerant date parser used everywhere, plus a deterministic post-processor that
-understands a literal extraction — dropping summary/total rows, moving tax/fee
-rows into charges (while keeping utility "energy charges" as items), and
-recovering the total. Instant, no model, far less manual fixing.
-
-**5. Make the agent *plan*, but never let it do the math.** For the assistant I
-gave the 8B a ReAct loop over **13 analytics tools** (`total_spend`,
-`category_spend`, `vendor_spend`, `biggest_expense` (rank-aware),
-`average_monthly`, `budget_status`, …). The model decides *which* tool to call and
-chains another when needed; the tools are deterministic Python, so the numbers
-can't be hallucinated. Crucially, an answer is only returned once it's **grounded
-by a real tool call** — if the model free-associates a number, it's rejected and a
-deterministic router answers instead. That split — *AI plans, code computes* — is
-what makes a small-model agent both genuinely agentic and trustworthy. (Open
-traces: [`AGENT_TRACES.md`](AGENT_TRACES.md).)
-
-## The "Off-Brand" bit: a custom frontend on `gradio.Server`
-
-The hackathon rewards going past the default Gradio look. Gradio 6's
-**`gradio.Server`** is perfect for it: you write Python API endpoints
-(`@app.api`, still ZeroGPU-enabled) and serve *your own* HTML/CSS/JS on top. I
-validated it with a tiny ping/`cuda_available` build on the Space first — the
-single best de-risking decision, because it proved a major version bump before I
-wrote a line of frontend. The result is a dark single-page dashboard (Chart.js
-donut + line, a budget ring, a calendar heatmap, editable line items, and an
-agent chat that shows the tools it used) that talks to the models through the
-Gradio JS client.
-
-## What "building small" taught me
-
-Constraints made the product *better*. Because the models are small and the data
-is mine, it can stay private (per-user, hashed PIN, a private dataset). Because
-the models are imperfect, I had to build the reconciliation, the reason-and-repair
-pass, and the grounded agent — which is exactly the "intelligence" that makes it
-trustworthy. Small didn't mean toy. It meant *honest*.
-
-*BudgetBuddy is open on its Space — snap a bill and ask it something.*
+## Why I built it
+
+My mum keeps a household running on cash, UPI, and a drawer full of crumpled
+bills. She writes the day's spends in a notebook and tries to hold a budget — and
+by night she genuinely can't remember where ₹350 went. Every "real" budgeting app
+wants her to link a bank account or type every line like a clerk, or it ships her
+receipts off to some cloud. None of that was ever going to happen in my house.
+
+So I built **BudgetBuddy** for the Build Small hackathon (Backyard AI track): take
+a photo of a bill or a payment screenshot, a small model reads it, and you can
+just *ask* where your money went. Two MiniCPM models do everything; no third-party
+AI API is ever called.
+
+It works now. Getting here was not a straight line.
+
+## I got the very first fact wrong
+
+I started by assuming MiniCPM-V-4.6 was an ~8B model like the older 2.6/4.5 line,
+and spent an evening worrying about GPU memory I didn't need to worry about. It's
+**1.3B**. I'd anchored on the version number as if it were a size. Then I tried to
+call it the way I "knew" the model worked — `model.chat(...)` — and it didn't
+exist anymore. In 4.6 it's a plain `transformers` model you drive with
+`AutoModelForImageTextToText` + `apply_chat_template`. I only sorted both out by
+actually opening the repo's `config.json` and chat template instead of trusting my
+memory. Lesson I keep re-learning: read the model's *actual* files first.
+
+## Then the dependencies tried to kill it
+
+Putting a current stack on a Hugging Face Space turned into a cascade. An older
+Gradio imported `HfFolder`, which `huggingface_hub` 1.0 had removed, so the Space
+500'd on boot. Fix that and the `gradio_client` schema walker choked on a boolean
+in my API signature. Fix that and Starlette and Jinja disagreed about something
+else. I kept patching symptoms until I gave up and moved onto a single,
+self-consistent modern Gradio instead of pinning a museum piece. Hours, gone, on
+nothing to do with the actual product.
+
+## A 401 that turned into an auth system
+
+First time I tried to save a parsed bill: **"Save failed: 401 Unauthorized."** Of
+course — writing to a Dataset needs a token. I wired that up, felt clever, and
+then it hit me: if every tester and judge writes to the *same* dataset, they'd all
+see each other's receipts. That's not a budgeting app, that's a group chat. So a
+"quick" capture demo grew a real spine: username + PIN (PBKDF2-hashed, salted),
+signed session tokens, and one private file per user. Not bank-grade, and I say so
+in the README — but honest about what it is.
+
+## Two models that refused to share a room
+
+This one cost me the most. The vision model wants a modern `transformers` (≈5.7+).
+MiniCPM4.1-8B — the reasoning model I wanted for understanding and the agent — has
+`trust_remote_code` written for `transformers` ≈4.56 and throws `RuntimeError`s on
+5.x. You cannot have both in one environment. I tried to shim the gaps
+(`is_torch_greater_or_equal_than_1_13`, `ALL_LAYERNORM_LAYERS`,
+`is_torch_fx_available` …) and got it to *load*, but it was a house of cards.
+
+The fix that actually worked was to stop fighting: run the 8B on **Modal**, in its
+own container with its own pinned `transformers`, behind a `generate()` method the
+Space calls over the SDK. A memory snapshot makes the cold start bearable and a
+warm window keeps it hot. The moment I stopped trying to make one environment do
+two contradictory things, the whole problem evaporated. That's also what made the
+8B affordable to use at all.
+
+## It worked, and it looked like everything else
+
+At one point the pipeline ran end-to-end and I was almost proud — until I looked
+at it. It was the default Gradio look, indistinguishable from a thousand other
+demos. So I rebuilt the front of it on Gradio 6's **`gradio.Server`**: Python API
+endpoints (still ZeroGPU-enabled) with my own HTML/CSS/JS served on top — a dark
+dashboard with Chart.js, a budget ring, a calendar heatmap, editable line items.
+I de-risked the version jump with a throwaway ping build before writing any UI,
+which saved me from discovering a breaking change at the worst possible time.
+
+## The assistant confidently lied to me
+
+The chat was the part I was proudest of and the part that embarrassed me most. I
+asked "how much did I spend in July 2026?" and it answered **₹0.00** — while a ₹419
+July bill sat right there in the data. The model wasn't hallucinating; my *tools*
+only understood `this_month` / `last_month` / `all`, so "July 2026" silently
+collapsed to the current month. I taught the tools real periods (a specific
+`YYYY-MM`, a year, this/last year) and started handing the agent today's date.
+
+Then I asked for my "second most expensive" spend and it cheerfully returned the
+*first* — my regex had matched "most expensive" inside "second", and there was no
+notion of rank at all. Little failures like that taught me the real rule for a
+small-model agent: let it *plan* which tool to call, but never let it do the
+arithmetic. The 13 tools are deterministic Python; the model chooses and chains
+them; an answer is only returned once it's grounded in an actual tool result, and
+ungrounded output is thrown away. *AI plans, code computes.* (Open traces:
+[`AGENT_TRACES.md`](AGENT_TRACES.md).)
+
+## Bills are messier than any schema
+
+Real bills don't respect your data model. Dates showed up as `01/07/17` and my
+ISO-only parser dropped them, so the charts looked empty while the data was fine.
+The vision model would list "Subtotal", "Total" and "Amount Payable" as if they
+were *items*, so nothing reconciled. An electricity bill's "energy charges" got
+mixed in with its taxes. None of that is solved by a bigger prompt — it's solved
+by a small, boring, deterministic pass that knows a summary row from a line item
+and a tax from a charge, runs in microseconds, and means my mum almost never has
+to fix anything by hand.
+
+## What building small actually taught me
+
+I expected "small models" to mean "compromise." It didn't. Because the models are
+small and local, the data can stay private and the thing is cheap to run. Because
+the models are imperfect, I *had* to build the reconciliation, the repair pass,
+and the grounded agent — and that work is exactly what makes the app trustworthy
+instead of a toy. The constraints didn't shrink the product. They made it honest.
+
+*BudgetBuddy is live — snap a bill and ask it something:*
+*https://huggingface.co/spaces/build-small-hackathon/BudgetBuddy*