I built my mom a budget app on two tiny MiniCPM models — here's what bit me
Live app: https://huggingface.co/spaces/build-small-hackathon/BudgetBuddy
90-second demo: https://youtu.be/QbgY6HDbrxE
Why I built it
My mum keeps a household running on cash, UPI, and a drawer full of crumpled bills. She writes the day's spends in a notebook and tries to hold a budget — and by night she genuinely can't remember where ₹350 went. Every "real" budgeting app wants her to link a bank account or type every line like a clerk, or it ships her receipts off to some cloud. None of that was ever going to happen in my house.
So I built BudgetBuddy for the Build Small hackathon (Backyard AI track): take a photo of a bill or a payment screenshot, a small model reads it, and you can just ask where your money went. Two MiniCPM models do everything; no third-party AI API is ever called.
It works now. Getting here was not a straight line.
I got the very first fact wrong
I started by assuming MiniCPM-V-4.6 was an ~8B model like the older 2.6/4.5 line, and spent an evening worrying about GPU memory I didn't need to worry about. It's 1.3B. I'd anchored on the version number as if it were a size. Then I tried to call it the way I "knew" the model worked — model.chat(...) — and it didn't exist anymore. In 4.6 it's a plain transformers model you drive with AutoModelForImageTextToText + apply_chat_template. I only sorted both out by actually opening the repo's config.json and chat template instead of trusting my memory. Lesson I keep re-learning: read the model's actual files first.
Then the dependencies tried to kill it
Putting a current stack on a Hugging Face Space turned into a cascade. An older Gradio imported HfFolder, which huggingface_hub 1.0 had removed, so the Space 500'd on boot. Fix that and the gradio_client schema walker choked on a boolean in my API signature. Fix that and Starlette and Jinja disagreed about something else. I kept patching symptoms until I gave up and moved onto a single, self-consistent modern Gradio instead of pinning a museum piece. Hours, gone, on nothing to do with the actual product.
A 401 that turned into an auth system
First time I tried to save a parsed bill: "Save failed: 401 Unauthorized." Of course — writing to a Dataset needs a token. I wired that up, felt clever, and then it hit me: if every tester and judge writes to the same dataset, they'd all see each other's receipts. That's not a budgeting app, that's a group chat. So a "quick" capture demo grew a real spine: username + PIN (PBKDF2-hashed, salted), signed session tokens, and one private file per user. Not bank-grade, and I say so in the README — but honest about what it is.
Two models that refused to share a room
This one cost me the most. The vision model wants a modern transformers (≈5.7+). MiniCPM4.1-8B — the reasoning model I wanted for understanding and the agent — has trust_remote_code written for transformers ≈4.56 and throws RuntimeErrors on 5.x. You cannot have both in one environment. I tried to shim the gaps (is_torch_greater_or_equal_than_1_13, ALL_LAYERNORM_LAYERS, is_torch_fx_available …) and got it to load, but it was a house of cards.
The fix that actually worked was to stop fighting: run the 8B on Modal, in its own container with its own pinned transformers, behind a generate() method the Space calls over the SDK. A memory snapshot makes the cold start bearable and a warm window keeps it hot. The moment I stopped trying to make one environment do two contradictory things, the whole problem evaporated. That's also what made the 8B affordable to use at all.
It worked, and it looked like everything else
At one point the pipeline ran end-to-end and I was almost proud — until I looked at it. It was the default Gradio look, indistinguishable from a thousand other demos. So I rebuilt the front of it on Gradio 6's gradio.Server: Python API endpoints (still ZeroGPU-enabled) with my own HTML/CSS/JS served on top — a dark dashboard with Chart.js, a budget ring, a calendar heatmap, editable line items. I de-risked the version jump with a throwaway ping build before writing any UI, which saved me from discovering a breaking change at the worst possible time.
The assistant confidently lied to me
The chat was the part I was proudest of and the part that embarrassed me most. I asked "how much did I spend in July 2026?" and it answered ₹0.00 — while a ₹419 July bill sat right there in the data. The model wasn't hallucinating; my tools only understood this_month / last_month / all, so "July 2026" silently collapsed to the current month. I taught the tools real periods (a specific YYYY-MM, a year, this/last year) and started handing the agent today's date.
Then I asked for my "second most expensive" spend and it cheerfully returned the first — my regex had matched "most expensive" inside "second", and there was no notion of rank at all. Little failures like that taught me the real rule for a small-model agent: let it plan which tool to call, but never let it do the arithmetic. The 13 tools are deterministic Python; the model chooses and chains them; an answer is only returned once it's grounded in an actual tool result, and ungrounded output is thrown away. AI plans, code computes. (Open traces: AGENT_TRACES.md.)
Bills are messier than any schema
Real bills don't respect your data model. Dates showed up as 01/07/17 and my ISO-only parser dropped them, so the charts looked empty while the data was fine. The vision model would list "Subtotal", "Total" and "Amount Payable" as if they were items, so nothing reconciled. An electricity bill's "energy charges" got mixed in with its taxes. None of that is solved by a bigger prompt — it's solved by a small, boring, deterministic pass that knows a summary row from a line item and a tax from a charge, runs in microseconds, and means my mum almost never has to fix anything by hand.
What building small actually taught me
I expected "small models" to mean "compromise." It didn't. Because the models are small and local, the data can stay private and the thing is cheap to run. Because the models are imperfect, I had to build the reconciliation, the repair pass, and the grounded agent — and that work is exactly what makes the app trustworthy instead of a toy. The constraints didn't shrink the product. They made it honest.
BudgetBuddy is live — snap a bill and ask it something: https://huggingface.co/spaces/build-small-hackathon/BudgetBuddy