curbcheck: can a small VLM tell you if you can legally park in San Francisco?

I came to SF for a week in April and left with two parking tickets. Both times I stood in front of a pole with four stacked signs (a 2-hour limit, except with a permit, except street cleaning on Tuesdays, also tow-away at evening rush) and could not work out whether I was allowed to leave my car there. Every fact you need is printed right there on the pole. It is pure perception, plus rule-logic, plus a clock.

So I built curbcheck: a small vision-language model that does the thing my brain failed to do.

Read, then reason

The whole design is one idea. The model does not guess a verdict. It only reads each sign on the pole into structured rules, and then a tiny deterministic resolver applies those rules to the current time.

photo  ->  VLM (reads each sign to JSON)  ->  deterministic resolver  ->  verdict + reason

The neural net only perceives. The logic is exact Python. And both halves are shown to you, so a misread is visible instead of buried inside a confident sentence. That split is also why stacked signs stop being hard: the resolver never fumbles the logic, no matter how many signs are on the pole.

The model

A QLoRA fine-tune (rank 16, language layers only, vision encoder frozen) of Qwen2.5-VL-3B-Instruct, trained on synthetic sign renders generated from public Caltrans sign specs, plus real SF photos teacher-labeled by a frontier model and human-verified.

Results

A stock 3B scores 0.16 on "can I park right now", below random. One QLoRA run takes it to 0.82 reasoning and 0.98 read accuracy on the synthetic benchmark.

The honest part: doubling the real training data and adding human labels and renderer realism moved real-photo reading by exactly one point, from 0.33 to 0.34. That tells me the remaining gap is model capacity, not data volume. The vision encoder is frozen, so unfreezing it (a vision-encoder LoRA) is the next experiment, not more data. Meanwhile the deterministic resolver keeps pipeline reasoning at 0.89 even when reading is hard, which is exactly why the read-then-reason split is worth it.

Try it

Upload a photo of an SF sign pole, pick a day and time, and see both what the model read and the verdict. Runs on ZeroGPU.

• Demo (Space): https://huggingface.co/spaces/build-small-hackathon/curbcheck
• Model (QLoRA adapter): https://huggingface.co/shubhamgoel27/curbcheck-qwen25vl3b-v4-lora
• Code and benchmark: https://github.com/shubhamgoel27/curbcheck

Built for the Build Small hackathon.