Pour Judgement: Teaching a Small Model to Judge Latte Art on a CPU

I recently fell down the latte art rabbit hole. It started innocently — a few YouTube videos on milk steaming technique, then an hour on r/latteart, then somehow three hours later I was watching slow-motion pour compilations and reading arguments about whether a tulip needs three layers or five to be "real." I do not own an espresso machine. I have not yet poured a single cup. But I have opinions now, apparently, and that felt like enough to build something.

The hackathon brief said: build small, run on a laptop, make it delightful. I had a judge in mind immediately — a fictional foam-obsessed champion who scores your pour, names what the pattern looks like (or what it accidentally resembles), and gives you exactly one tip. One screen, one button, no instructions needed.

The Scoring Problem

The first version was embarrassingly easy to fool. A plain white circle — a foam dump, no art at all — scored 72 out of 100. The metrics were measuring the right things in the wrong way.

The culprit was definition. I was measuring edge sharpness along the foam boundary using a Sobel gradient. A perfect circle has a beautifully sharp edge. So does a rosetta. The metric couldn't tell them apart.

The fix was boundary complexity — the isoperimetric ratio, which compares a shape's perimeter squared to its area. A circle has the minimum ratio for its size. Latte art has lobes, indentations, fine lines, multiple regions — all of which increase the ratio dramatically. I made both definition and flow scores a product of sharpness and complexity, so a smooth blob can no longer ride a clean edge to a high score.

I also replaced the symmetry metric entirely. Mirror IoU punished swans (asymmetric by design), misread stacked tulips photographed at an angle (perspective compression collapses the layers), and rewarded blobs (which self-mirror fine). The new flow metric uses gradient orientation histograms inside the foam — measuring whether the pattern has strong preferred directions and multiple distinct directional clusters. A tulip has strong vertical dominance. A rosetta has both a stem direction and a leaf zigzag. A blob has diffuse, uniform orientations. No mirror needed.

The Model

MiniCPM-V 2.6 (OpenBMB) in GGUF format, quantized to Q4, running via llama-cpp-python on CPU. The model sees the photo plus the objective CV scores — so the personality is grounded in real measurements, not vibes. The biggest speed win was resizing images to 448px before sending them to the model. Phone photos are 3000px+. MiniCPM-V tiles large images into multiple patches and runs the vision encoder on each one. Dropping to 448px cut the dominant cost dramatically without affecting verdict quality — a latte pattern is fully readable at that resolution.

The app never crashes. If the model fails to load, template verdicts grounded in the same CV scores take over silently.

What I Learned

Classical CV and a small VLM are a better team than either alone. The numbers give the judge something real to say. The judge gives the numbers a voice. A foam dump scores 51 now. A rosetta scores 86. That feels about right.

I still haven't poured anything art worthy, and I know exactly what Esme (fictional character) would say if I did.