PawMap: Field Notes
The spark
My friend Camila has been feeding stray animals in Brasília for years. She knows every cat in her colony by name — which ones have been neutered, which one got hit by a car last winter and recovered, which one hasn't shown up this week. She holds all of this in her head, in WhatsApp messages, in a notes app on her phone.
When I told her I was building a tool to map stray animals with AI, she was immediately skeptical. "Another app that's hard to use and needs you to create an account." That sentence shaped every design decision I made.
By the end of the hackathon, she had registered 11 animals near her block in Asa Norte. Her feedback was blunt and useful: the map worked, the camera flow worked, but the first time two similar-looking cats got merged into one profile she said "your app thinks Pretinha and Fumaça are the same cat. They are not."
She was right. And that's what this post is about.
What PawMap does
PawMap is a collaborative map for stray animals. Anyone takes a photo on their phone, the AI identifies species, breed and color, GPS pins the location, and the app checks whether that animal has been seen before — using cosine similarity on semantic embeddings to link sightings to a profile.
Over time each animal builds up a trail: where it appears, who helped, whether its condition is improving.
The stack is small: Llama-3.2-11B-Vision via HF Serverless for visual identification, all-MiniLM-L6-v2 (22M parameters) running locally for embeddings, SQLite for storage, and a fully custom SPA frontend built with gradio.Server + Leaflet.js — no default Gradio components in sight.
The hard part: making the matcher not lie
The core idea sounds clean: generate a semantic embedding from the AI's description of the animal, compare it against existing profiles with cosine similarity, and if the score is ≥ 0.80 it's the same animal.
In practice, this broke in interesting ways.
Problem 1: The AI describes animals similarly
When I asked Llama-3.2-11B-Vision to describe a random sample of street cats in Brasília, a depressing number of them came back as "medium grey tabby cat, thin, no distinctive marks." That's an accurate description of roughly 40% of the cats in my test set. Their embeddings clustered so close together that cosine similarity at 0.80 merged them into one mega-profile.
Pretinha and Fumaça were both small black cats with no distinguishing features. Score: 0.94. Merged. Camila was not impressed.
Problem 2: The same animal looks different across photos
A dog photographed at noon in direct sunlight versus the same dog at dusk looks like two different animals to the vision model. The color descriptions diverge ("golden" vs "brown"), the condition assessment changes ("healthy" vs "thin"), and the embedding shifts enough to fall below the threshold — creating a duplicate profile for the same animal.
What I did to patch it
I raised the threshold to 0.85 for cats (they're harder to distinguish) and kept 0.80 for dogs. I also made the confirmation screen show the top 3 matches so the user can manually merge — Camila used this constantly. It's not elegant, but it works.
What I'd do differently
The embedding is generated from a text description, which loses a lot of visual information. A proper solution would be a dual-encoder: one branch for the image embedding (something like CLIP), another for the semantic description, and a learned fusion layer that combines both. The matching would be far more robust.
For a hackathon project running on a free CPU Space, that was out of scope. But it's the first thing I'd build if this were a real product.
The thing that actually worked well: the help flow
I added a feature late in the build: when someone helps an animal — feeds it, treats a wound, takes it to a vet — they can submit a help proof photo. The AI does two things with it:
- Verifies the photo is of the same registered animal (cosine similarity again, now used as a trust signal rather than a registration gate)
- Detects condition improvement by comparing the new description against the profile's history — if the animal went from
"thin"to"healthy", that gets flagged
Camila photographed one of her cats after a vet visit and the app correctly flagged a condition improvement: "was: thin → now: healthy." That moment felt like the app was actually doing something useful.
The frontend detour
I wanted the map to be the first thing you see, not a Gradio chat interface. That meant using gradio.Server and building the whole frontend myself: a SPA in vanilla JS with Leaflet.js for the map, Lucide Icons, and a custom CSS theme.
This took longer than expected. gradio.Server is powerful but the documentation for running it as a pure backend (with your own HTML/JS frontend) is thin. I spent a full day just getting the static file mounting right and understanding how @app.api() routes interact with Gradio's queue.
The result was worth it — the app feels like a real mobile tool, not a demo. But if I did it again I'd budget twice the time for the frontend.
Numbers
- 6 animals in the seed dataset, spread across Asa Norte and Asa Sul in Brasília
- 11 animals registered by Camila during real-world testing
- 3 false merges caught and manually corrected (all cats)
- 1 condition improvement correctly detected
- ~22M parameters running locally (MiniLM only)
- 0 cloud APIs with persistent data — everything stored in SQLite on the Space
What's next
PawMap is a working prototype, not a finished product. The things I'd prioritize if I kept building:
- Image embeddings alongside text embeddings for better matching
- Neighborhood-aware matching — a dog seen at -15.756, -47.892 and then at -15.900, -47.800 (4 km away) is probably not the same dog, regardless of similarity score
- Push notifications — Camila wants to be alerted when an animal she's been tracking hasn't been spotted in 30 days
- Multilingual interface — Brasília is the capital, but the stray animal problem is everywhere in Brazil
Built in Brasília, DF — for Camila, Pretinha, Fumaça, and the eleven animals that now have a profile on the map. 🐾