Spaces:
Running
Running
| title: Pinch | |
| emoji: 🤏 | |
| colorFrom: yellow | |
| colorTo: green | |
| sdk: gradio | |
| sdk_version: 6.18.0 | |
| app_file: app.py | |
| pinned: false | |
| license: apache-2.0 | |
| tags: | |
| - build-small-hackathon | |
| - track:backyard | |
| - sponsor:openbmb | |
| - sponsor:modal | |
| - off-brand | |
| - best-agent | |
| - best-demo | |
| - tiny-titan | |
| # 🤏 Pinch — photograph your ingredients, get a grounded plan | |
| **Build Small Hackathon 2026 · 🏡 Backyard AI track** | |
| **Snap a photo of what's in your kitchen — Pinch works out what to cook, plans | |
| the seasoning grounded in real flavour science, and computes the amounts.** A | |
| vision model reads the ingredients from your photo; an agent reasons over the | |
| Epicure flavour model (1,790 ingredients distilled from ~4M recipes) to decide | |
| *what* to add, *what to substitute* for what you lack, and *in what order*; and | |
| it runs Python in a sandbox for the quantitative side (servings, ratios, salt %, | |
| timing). Optionally, FLUX turns your ingredients photo into a picture of the | |
| finished dish. | |
| Small models, each load-bearing, **≈24B total — under the 32B cap**: | |
| [MiniCPM-V](https://huggingface.co/openbmb/MiniCPM-V-4.6) (OpenBMB) sees the | |
| ingredients, [Mellum 2](https://huggingface.co/JetBrains/Mellum2-12B-A2.5B-Instruct) | |
| (12B, JetBrains) reasons about flavour, and | |
| [FLUX.2 klein](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B) (4B) | |
| renders the dish. | |
| ### Why this needs an agent | |
| The hard parts aren't a lookup. **Flavour** — *given the messy set of things in | |
| my kitchen and this dish, what's worth adding, and in what order?* — is | |
| combinatorial and grounded in the Epicure pairing model, not the LLM's opinion. | |
| **Arithmetic** — scaling to servings, ratios, salt % — is something LLMs get | |
| wrong in their heads, so the agent offloads it to real code in a sandbox. The | |
| model reasons and sequences; the tools provide the facts. | |
| **🎥 Demo video:** https://www.youtube.com/watch?v=gpXQS35QWW0 | |
| **📣 Social post:** https://x.com/ssaacar/status/2066645089942708604 | |
| ## Architecture | |
| ``` | |
| You: photo (or typed ingredients) + optional constraints | |
| │ MiniCPM-V reads the photo → ingredient list (you edit it) | |
| ▼ suggest_dish() + build_plan() | |
| ┌────────────────────────────┐ pairing_score ┌────────────────────────────┐ | |
| │ JetBrains Mellum 2 (12B) │ ──────────────► │ Epicure MCP (Kaikaku) │ | |
| │ strict-JSON planner loop │ ◄────────────── │ pairing scores, neighbours │ | |
| │ │ run_python ┌────────────────────────────┐ | |
| │ │ ──────────────► │ sandbox (kitchen math: │ | |
| │ │ ◄────────────── │ amounts, ratios, timing) │ | |
| └────────────┬───────────────┘ stdout └────────────────────────────┘ | |
| │ | |
| ▼ staged plan + amounts ──► (optional) FLUX.2 klein → dish photo | |
| ``` | |
| - **Models:** [Mellum 2](https://huggingface.co/JetBrains/Mellum2-12B-A2.5B-Instruct) reasons + writes the math code (served on Modal); [MiniCPM-V](https://huggingface.co/openbmb/MiniCPM-V-4.6) reads the photo (OpenBMB's free hosted API); [FLUX.2 klein](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B) renders the dish (HF Inference / fal-ai). All remote — **the Space itself needs no GPU.** | |
| - **Flavour grounding:** [Epicure MCP](https://github.com/KAIKAKU-AI/epicure-mcp) — public, anonymous flavour server (`find_pairings`, `pairing_score`, `neighbors`, …) via a hand-rolled MCP client (`epicure_client.py`) with 404/429 retry + a disk cache. Each step's evidence is re-derived from a **real** pairing score, never the model's guess. | |
| - **Sandbox:** the quantitative math runs as real Python (`sandbox.py`), tailored to the dish type (a salad gets no simmer time; a dessert gets no savoury salt). `SANDBOX_BACKEND=local|modal|mock`. | |
| - **Robustness:** the model emits one JSON object per turn — an Epicure tool call, a `run_python`, or the finished plan (`agent.py`). If it doesn't converge, a grounded scripted planner (real Epicure scores + sandbox math, no LLM) takes over. | |
| - **Honesty note:** pairing *scores* are grounded facts; the substitution hint uses Epicure `neighbors` (co-occurrence, not a functional swap), so it's surfaced as "closest in flavour space," not "use X for the sourness of Y." | |
| ## Run it | |
| **In the browser:** add a photo (or type ingredients) → **Detect ingredients** → | |
| tidy the list to what you actually have → **Decide a dish & plan** → optionally | |
| **✨ See the dish**. | |
| **Local dev:** | |
| ```bash | |
| pip install -r requirements.txt | |
| cp .env.example .env # fill in MODAL_REASON_URL, HF_TOKEN, etc. | |
| python app.py | |
| # fully offline (no models, scripted planner + sample detection): | |
| MOCK_LLM=1 python app.py | |
| ``` | |
| ## Where the models run (inference backend) | |
| Each model is configured independently via env vars (see `.env.example`). | |
| **Vision (MiniCPM-V) — `VISION_BACKEND`:** has a free hosted API, so it runs real | |
| with **no GPU**. | |
| | `VISION_BACKEND` | runs on | cost | | |
| |---|---|---| | |
| | `openbmb` | OpenBMB's free hosted API (`api.modelbest.cn`, OpenAI-compatible) | free | | |
| | `zerogpu` | in-Space ZeroGPU | free, quota | | |
| | `modal` | your Modal endpoint | credits | | |
| **Reasoning (Mellum 2) — `INFERENCE_BACKEND`:** Mellum has **no hosted API** | |
| anywhere, so it runs on a GPU you control. | |
| | `INFERENCE_BACKEND` | runs on | cost | | |
| |---|---|---| | |
| | `modal` | your Modal endpoint (no quota) | credits | | |
| | `zerogpu` | in-Space ZeroGPU (40 min/day) | free | | |
| | `mock` (`MOCK_LLM=1`) | scripted planner, no model | free (dev) | | |
| **Dish image — FLUX.2 klein:** the "✨ See the dish" button renders the finished | |
| plate via HF Inference (`provider="fal-ai"`), billed to your account. Set | |
| `HF_TOKEN`; without it the button just shows a hint. | |
| **Deploy the Modal reasoning backend** (one-time; `pip install modal` + `modal token new`): | |
| ```bash | |
| modal deploy modal_app.py # serves Mellum 2 on A100-40GB; weights cached in a Volume | |
| ``` | |
| It prints a URL. Set on the Space (Settings → Variables & secrets): | |
| ``` | |
| INFERENCE_BACKEND=modal | |
| MODAL_REASON_URL=<the printed url> | |
| VISION_BACKEND=openbmb | |
| ``` | |
| `min_containers=0` → idle costs nothing; cold start ~30–60s (weights cached). | |