# The face was always there: building PAREIDOLIA with two small OpenBMB models

*A Build Small Hackathon field report — Thousand Token Wood.*
*Live: [build-small-hackathon/pareidolia](https://huggingface.co/spaces/build-small-hackathon/pareidolia)*

---

Pareidolia is the thing your brain does when it sees a face in a power outlet,
in the front of a car, in two windows and a door. Everyone has it. No app
celebrates it.

So we built one. Point your camera at any object. A small vision model studies
the photo the way you study clouds — and the face that was *already hiding* in
your fire hydrant opens its eyes, blinks at you, and tells you, in its own
voice, exactly what it has been putting up with.

> *"Forty years on this corner. Not one dog has shown me respect."*

Two models do all of it, and together they're about **11B parameters** — well
under the hackathon's 32B ceiling, and every parameter earns its place:

- **[openbmb/MiniCPM-V-4_5](https://huggingface.co/openbmb/MiniCPM-V-4_5)** (8.7B) — finds the face and writes the soul.
- **[openbmb/VoxCPM2](https://huggingface.co/openbmb/VoxCPM2)** (2.29B) — gives each object a distinct designed voice, from a text description alone, no reference audio.

## The one rule: taste from the model, facts from the code

The failure mode that kills a project like this is obvious once you hit it: no
vision model is *trained* to point at the two bolts on a hydrant and call them
eyes. Ask it for pixel coordinates and it will confidently place an eye in the
sky. A misplaced eye doesn't read as a quirky soul — it reads as a broken
sticker.

So we never let the model touch a coordinate. The séance is one structured
generation: the model **names visible features** ("the two bonnet bolts," "the
front outlet cap"), reads the object's condition, and writes its grudge — then
deterministic OpenCV snaps those names onto the strongest real circles, blobs,
and corners nearby. The model brings taste; the code owns the facts. It's the
pattern this community already celebrates, and it's the whole reason the eyes
land on things that are really there.

The writing is governed the same way — not vibes, but a bank of grudge
archetypes keyed to an object's *visible condition* (the veteran/rusted, the
martyr/worn, the perfectionist/pristine, the diva/displayed), with an iron rule
that every line must reference something actually in the photo. A coffee mug
with a violin-shaped handle, unprompted: *"Why a violin? Why not a guitar?"*

## The shared wall, alive with zero GPU

Every awakened object joins a public **Menagerie** — one wall, shared by every
visitor. Because each record is deterministic, the overlays (the blinking, the
saccades, the mouth riding the audio) replay entirely client-side from JSON.
The landing page is fully alive even when the GPU is asleep, and every object
gets its own shareable page with a social preview card showing it *with its
eyes*. The GPU only wakes when you awaken something new.

Privacy is structural, not bolted on: the same generation gates real human
faces and NSFW before anything else runs, and nothing reaches the public wall
that hasn't passed that gate. No visitor identity is ever stored.

## What we tried, measured, and threw away

The honest part. We had two "obvious" upgrades and let live measurement veto
both:

- **A punch-up pass** that rewrote each grudge to be funnier. Tested live on six
  real awakenings: zero rewrites better than the original, two grammatically
  broken. Our acceptance check validated *form* (length, banned openers, cites a
  specific) but couldn't see *coherence*. Reverted within the hour. A flat
  original beats a broken rewrite.
- **A set-of-marks grounding flip** — let OpenCV draw numbered badges and have
  the model pick numbers instead of inventing coordinates. It sounded purer. The
  bench said no: **3/14** plausible faces versus **8/14** for the coarse-then-snap
  flow. The model picked badges sitting on the pavement while confabulating names
  for them. The real lesson: a vision model's coarse spatial *intuition*,
  corrected by deterministic snapping, beats its discrete *selection* over
  badly-curated choices.

Both are written up, with their data, in the repo's
[FIELD_NOTES.md](https://huggingface.co/spaces/build-small-hackathon/pareidolia/blob/main/FIELD_NOTES.md).
Every awakening's full trace — the model's JSON, the snap deltas, the chosen
voice — is published to a [public dataset](https://huggingface.co/datasets/AndresCarreon/pareidolia-menagerie).

## What shipped to make it survive a judging crowd

- A face-**geometry gate** that measures each placed face (eyes level? cramped?
  mouth below? on the object?) and gives the model one in-context do-over when it
  fails.
- A **health endpoint and an external watchdog** that auto-restarts the Space if
  the ZeroGPU context ever wedges.
- Honest, retryable messaging when the free GPU quota is busy — *"the spirits are
  overwhelmed; rest a moment and try again, or sign in for your own candle"* —
  instead of a generic error.

It's small models, used carefully, doing something that genuinely couldn't exist
without them. Go wake something up. It was in there all along. 👁️