| --- |
| title: Paint Match |
| emoji: 🎨 |
| colorFrom: yellow |
| colorTo: red |
| sdk: gradio |
| sdk_version: 6.16.0 |
| app_file: server.py |
| license: apache-2.0 |
| short_description: Paint codes from kit photos — finetuned 1B VLM, no cloud |
| pinned: false |
| models: |
| - build-small-hackathon/paint-match-minicpm |
| tags: |
| - vision |
| - llama-cpp |
| - scale-modelling |
| - track:backyard |
| - sponsor:openbmb |
| - sponsor:modal |
| - achievement:offgrid |
| - achievement:welltuned |
| - achievement:offbrand |
| - achievement:llama |
| - achievement:fieldnotes |
| --- |
| |
| # Paint Match |
|
|
| **A finetuned 1-billion-parameter vision model running on a $50 ARM board identifies paint codes from scale model instruction sheets — no cloud API, no GPU.** |
|
|
| Upload a photo of an instruction sheet or a shop screenshot. Paint Match extracts the paint codes, converts Humbrol references to Tamiya equivalents, and shows shop links. Paints you already own are flagged automatically. |
|
|
| ## Who it's for |
|
|
| I build plastic scale models. Every kit ships an instruction sheet listing paint codes in a manufacturer's own system (Humbrol, Tamiya, Meng), and matching those to what's actually on the shelf — and to what I already own — is tedious manual cross-referencing before every build. Paint Match is the tool I use on my own kits: photograph the sheet, get a Tamiya shopping list, with paints already in my Google Sheets inventory flagged so I don't re-buy them. |
|
|
| The UI is styled after Airfix Dogfighter — a 2000 PC game I played as a kid. The aesthetic felt right for a tool that's part of the same hobby. |
|
|
| ## How it works |
|
|
| - Photo is resized to 960px (A/B tested: 640px drops codes on small print; 1280px adds 30s with no recall gain) |
| - [MiniCPM-V-4.6](https://huggingface.co/openbmb/MiniCPM-V-4.6) finetuned on paint instruction data, runs via llama.cpp on a Radxa Dragon Q6A — 4 CPU cores, no GPU |
| - Fine-tuned on Modal.com H100 (16 min, ~391 training examples); training data scraped from Airfix instruction sheets and paper photos |
| - JSON schema constraint forces structured output; no regex fragility |
| - Humbrol→Tamiya conversion from a hand-curated CSV; inventory check via Google Sheets |
| - Inference: ~64s average (down from ~190s with the base InternVL model) |
|
|
| ## Benchmark (MiniCPM-V-4.6 finetuned, running on Radxa CPU) |
|
|
| | Metric | Score | |
| |--------|-------| |
| | F1 — benchmark (10 images) | **0.935** | |
| | F1 — shop holdout (53 images) | **0.927** | |
| | F1 — paper holdout (7 images) | **0.928** | |
| | Avg latency | ~64s | |
|
|
| Base model comparison (InternVL3.5-2B): F1=0.873, ~190s latency. Gemma 4B timed out on every instruction sheet image on this hardware. |
|
|
| ## Architecture |
|
|
| All inference is 100% local, on hardware I own. This HF Space is a thin proxy UI only — it holds no model and calls no hosted LLM API. Photos are forwarded over a Cloudflare Tunnel to a private Radxa SBC, where the finetuned 1B model runs on CPU via llama.cpp; the Space just renders the result. Off-grid where it counts: the AI never leaves the board. |
|
|
| ## Demo & social |
|
|
| - Demo video - https://youtu.be/mNMjyIMAUuo |
| - Social post - https://x.com/szwendacz/status/2064679708877291630 |
| - Field notes - [blog post](https://llmops.pl/posts/paint_match_first_hackathon_first_vlm_on_a_board/) |
|
|
