---
title: Pinch
emoji: 🤏
colorFrom: yellow
colorTo: green
sdk: gradio
sdk_version: 6.18.0
app_file: app.py
pinned: false
license: apache-2.0
tags:
  - build-small-hackathon
  - track:backyard
  - sponsor:openbmb
  - sponsor:modal
  - off-brand
  - best-agent
  - best-demo
  - tiny-titan
---

# 🤏 Pinch — photograph your ingredients, get a grounded plan

**Build Small Hackathon 2026 · 🏡 Backyard AI track**

**Snap a photo of what's in your kitchen — Pinch works out what to cook, plans
the seasoning grounded in real flavour science, and computes the amounts.** A
vision model reads the ingredients from your photo; an agent reasons over the
Epicure flavour model (1,790 ingredients distilled from ~4M recipes) to decide
*what* to add, *what to substitute* for what you lack, and *in what order*; and
it runs Python in a sandbox for the quantitative side (servings, ratios, salt %,
timing). Optionally, FLUX turns your ingredients photo into a picture of the
finished dish.

Small models, each load-bearing, **≈24B total — under the 32B cap**:
[MiniCPM-V](https://huggingface.co/openbmb/MiniCPM-V-4.6) (OpenBMB) sees the
ingredients, [Mellum 2](https://huggingface.co/JetBrains/Mellum2-12B-A2.5B-Instruct)
(12B, JetBrains) reasons about flavour, and
[FLUX.2 klein](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B) (4B)
renders the dish.

### Why this needs an agent

The hard parts aren't a lookup. **Flavour** — *given the messy set of things in
my kitchen and this dish, what's worth adding, and in what order?* — is
combinatorial and grounded in the Epicure pairing model, not the LLM's opinion.
**Arithmetic** — scaling to servings, ratios, salt % — is something LLMs get
wrong in their heads, so the agent offloads it to real code in a sandbox. The
model reasons and sequences; the tools provide the facts.

**🎥 Demo video:** https://www.youtube.com/watch?v=gpXQS35QWW0

**📣 Social post:** https://x.com/ssaacar/status/2066645089942708604

## Architecture

```
  You: photo (or typed ingredients) + optional constraints
            │  MiniCPM-V reads the photo → ingredient list (you edit it)
            ▼   suggest_dish()  +  build_plan()
   ┌────────────────────────────┐  pairing_score  ┌────────────────────────────┐
   │  JetBrains Mellum 2 (12B)  │ ──────────────► │  Epicure MCP (Kaikaku)     │
   │  strict-JSON planner loop  │ ◄────────────── │  pairing scores, neighbours │
   │                            │   run_python    ┌────────────────────────────┐
   │                            │ ──────────────► │  sandbox (kitchen math:     │
   │                            │ ◄────────────── │  amounts, ratios, timing)   │
   └────────────┬───────────────┘   stdout        └────────────────────────────┘
                │
                ▼  staged plan + amounts  ──►  (optional) FLUX.2 klein → dish photo
```

- **Models:** [Mellum 2](https://huggingface.co/JetBrains/Mellum2-12B-A2.5B-Instruct) reasons + writes the math code (served on Modal); [MiniCPM-V](https://huggingface.co/openbmb/MiniCPM-V-4.6) reads the photo (OpenBMB's free hosted API); [FLUX.2 klein](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B) renders the dish (HF Inference / fal-ai). All remote — **the Space itself needs no GPU.**
- **Flavour grounding:** [Epicure MCP](https://github.com/KAIKAKU-AI/epicure-mcp) — public, anonymous flavour server (`find_pairings`, `pairing_score`, `neighbors`, …) via a hand-rolled MCP client (`epicure_client.py`) with 404/429 retry + a disk cache. Each step's evidence is re-derived from a **real** pairing score, never the model's guess.
- **Sandbox:** the quantitative math runs as real Python (`sandbox.py`), tailored to the dish type (a salad gets no simmer time; a dessert gets no savoury salt). `SANDBOX_BACKEND=local|modal|mock`.
- **Robustness:** the model emits one JSON object per turn — an Epicure tool call, a `run_python`, or the finished plan (`agent.py`). If it doesn't converge, a grounded scripted planner (real Epicure scores + sandbox math, no LLM) takes over.
- **Honesty note:** pairing *scores* are grounded facts; the substitution hint uses Epicure `neighbors` (co-occurrence, not a functional swap), so it's surfaced as "closest in flavour space," not "use X for the sourness of Y."

## Run it

**In the browser:** add a photo (or type ingredients) → **Detect ingredients** →
tidy the list to what you actually have → **Decide a dish & plan** → optionally
**✨ See the dish**.

**Local dev:**
```bash
pip install -r requirements.txt
cp .env.example .env     # fill in MODAL_REASON_URL, HF_TOKEN, etc.
python app.py
# fully offline (no models, scripted planner + sample detection):
MOCK_LLM=1 python app.py
```

## Where the models run (inference backend)

Each model is configured independently via env vars (see `.env.example`).

**Vision (MiniCPM-V) — `VISION_BACKEND`:** has a free hosted API, so it runs real
with **no GPU**.

| `VISION_BACKEND` | runs on | cost |
|---|---|---|
| `openbmb` | OpenBMB's free hosted API (`api.modelbest.cn`, OpenAI-compatible) | free |
| `zerogpu` | in-Space ZeroGPU | free, quota |
| `modal` | your Modal endpoint | credits |

**Reasoning (Mellum 2) — `INFERENCE_BACKEND`:** Mellum has **no hosted API**
anywhere, so it runs on a GPU you control.

| `INFERENCE_BACKEND` | runs on | cost |
|---|---|---|
| `modal` | your Modal endpoint (no quota) | credits |
| `zerogpu` | in-Space ZeroGPU (40 min/day) | free |
| `mock` (`MOCK_LLM=1`) | scripted planner, no model | free (dev) |

**Dish image — FLUX.2 klein:** the "✨ See the dish" button renders the finished
plate via HF Inference (`provider="fal-ai"`), billed to your account. Set
`HF_TOKEN`; without it the button just shows a hint.

**Deploy the Modal reasoning backend** (one-time; `pip install modal` + `modal token new`):
```bash
modal deploy modal_app.py     # serves Mellum 2 on A100-40GB; weights cached in a Volume
```
It prints a URL. Set on the Space (Settings → Variables & secrets):
```
INFERENCE_BACKEND=modal
MODAL_REASON_URL=<the printed url>
VISION_BACKEND=openbmb
```
`min_containers=0` → idle costs nothing; cold start ~30–60s (weights cached).