README.md

metadata

title: Facade
emoji: 🏛
colorFrom: blue
colorTo: red
sdk: gradio
app_file: app.py
pinned: false
license: mit
short_description: Photograph a building, find its architectural style

Facade

Point a camera at a building. Find out what you are looking at.

Upload a photograph of a facade and the app returns the three closest architectural styles from a 1,000-plate synthetic reference corpus, a reading written for your photograph at query time, the measurements behind the match, and real named buildings near you in the same style.

• Dataset: Jonathandav/facade-styles
• Corpus: 1,000 generated plates · 20 styles · 50 each
• Retrieval model: laion/CLIP-ViT-B-32-laion2B-s34B-b79K
• Reading model: Qwen/Qwen2.5-1.5B-Instruct

---

The idea

Architectural style is a family resemblance rather than a fact about a building. That makes it a good target for synthetic reference data: a generated Bauhaus facade is a Bauhaus facade, because there is no external specimen it can contradict.

It also makes style unusually measurable. Brutalism is low-saturation and heavy-massed; Gothic Revival is vertical-edge dominant; Bauhaus is horizontal; Ottoman Revival is curve-rich. Every one of those has a pixel-level correlate, so "did the generator do what it was told?" is a computed quantity rather than an opinion.

The whole project is built on that: a corpus specified attribute by attribute, audited by measurement, and finally tested against real photographs.

---

How it works

 style_seed.csv          20 styles, each with massing, material, window
       │                 rhythm, roofline, palette, and two EXPECTED
       │                 measurements (saturation band, edge orientation)
       ▼
 sample_plates()         50 specs per style; view / crop / light / condition
       │                 CYCLED, not sampled — so they cannot correlate
       ▼                 with style
 SDXL-Turbo              style name + period + key features lead the prompt
       │                 acceptance gate measures every render and retries
       ▼                 with a fresh seed until it passes
 1,000 plates ──► LAION-CLIP embeddings ──► index_embeddings.npy
       │
       ▼
 Qwen2.5-1.5B            style readings, barred from naming architects,
                         asserting dates, or claiming heritage status

At query time: embed the photograph → cosine similarity against the index → optionally rerank with an OpenStreetMap prior over what actually stands nearby → generate a reading from the matched style, the runner-up, and the measurements taken off the photograph.

---

What the corpus can identify

Bauhaus International · Brutalist · Art Deco · Ottoman Revival · Mandate Eclectic · Neoclassical · Gothic Revival · Venetian Gothic · Georgian · Victorian Terrace · Mid-century Modern · Postmodern · Deconstructivist · Contemporary Curtain Wall · Mediterranean Vernacular · Adobe Pueblo · Soviet Constructivist · Scandinavian Functionalist · Industrial Warehouse · Japanese Metabolist

Anything outside these twenty is still forced onto the nearest of them. The app says so in the interface, because a user should know that before trusting a result.

---

Did the generator obey?

Each style declares an expected saturation level. That is an ordinal claim — brutalism less saturated than Art Deco — not a value on a calibrated scale.

Spearman ρ = 0.67, p ≈ 1.5 × 10⁻¹³², n = 1000. Declared rank predicts measured saturation strongly.

The first version of this test scored 8% compliance, because the bands were absolute thresholds chosen before any image existed: measured saturation ran 0.30–0.85 while "medium" had been guessed at 0.22–0.50. Recalibrating the bands to corpus quantiles and reporting rank agreement is the correct test, and it is a better methodological story than thresholds that happened to work first time.

---

Where will it get confused?

Plotting each style's centroid in saturation × edge-orientation space produces a prediction, before any embedding model runs, of which pairs will be conflated:

pair	centroid distance
Neoclassical ↔ Postmodern	0.41
Japanese Metabolist ↔ Scandinavian Functionalist	0.62
Mandate Eclectic ↔ Neoclassical	0.65
Brutalist ↔ Japanese Metabolist	0.65
Georgian ↔ Victorian Terrace	0.75

These are architecturally meaningful, not arbitrary. Postmodernism is defined by ironic quotation of classical form. Mandate Eclectic borrowed classical proportion directly. Brutalism and Metabolism are contemporaneous raw-concrete movements. A purely pixel-level metric recovered relationships an architectural historian would recognise.

Georgian ↔ Victorian Terrace was confirmed by the CLIP confusion matrix at 0.16 — the top confusion by a wide margin. The others were not, and CLIP's actual confusions cluster differently: Bauhaus International acts as an attractor for the modernist styles, with Brutalist, Soviet Constructivist and Japanese Metabolist all leaking toward it. That is a semantic effect the pixel metric cannot see.

One prediction confirmed and four refuted with a coherent explanation is a better result than five vague hits.

---

Does style dominate the nuisance factors?

Style explains 43–66% of variance in every measured feature. Crop, view and condition sit at essentially zero — which validates the sampler: those axes are cycled rather than drawn independently, so they cannot correlate with style. Had they done so, an embedding model could have scored well by learning framing instead of architecture.

Light is the only meaningful contaminant, and only on the photometric features, which is physically unsurprising.

---

Retrieval

Image-to-image over the corpus, style as the label, chance ≈ 0.049.

model	Recall@1	Recall@3	MRR	lift
SigLIP-B/16	0.972	0.994	0.983	19.8×
LAION-CLIP-B/32	0.970	0.991	0.981	19.8×
CLIP-B/32	0.955	0.988	0.972	19.5×

These are at ceiling and the comparison cannot resolve. Plates of the same style share nearly identical prompts, differing only in angle, crop, light and condition, so they are near-siblings. Every slice falls between 0.975 and 1.000. The differences between models are inside noise.

---

Sim-to-real — the result that matters

17 hand-labelled photographs of real buildings in Tel Aviv, Jaffa, Paris and the Greek islands, retrieved against the wholly synthetic index.

model	real R@3	synthetic R@3	gap
LAION-CLIP-B/32	0.94 (16/17)	0.991	−0.05
SigLIP-B/16	0.88 (15/17)	0.994	−0.11
CLIP-B/32	0.76 (13/17)	0.988	−0.22

The synthetic benchmark ranked SigLIP first by 0.002 on Recall@1. On real photographs LAION wins and has less than half the domain gap. Selecting on the saturated benchmark would have shipped the weaker model. A benchmark at ceiling does not merely fail to discriminate — it discriminates wrongly.

Both failure cases are architecturally sensible: a Bauhaus building retrieved as Soviet Constructivist (contemporaneous interwar modernism, strip windows, white render), and a night photograph of a Mandate building retrieved as Mediterranean Vernacular (both cream render with shutters).

---

Generator failures found along the way

Four, plus one measurement bug. Three of the four were silent — no exception, no warning, and plausible-looking output.

#	failure	how it was caught	silent
1	SDXL's pooled embedding overrode every style	Bauhaus, Brutalist and Gothic all rendered as the same beige office block	yes
2	detail crops destroyed the class signal	style unrecoverable from a cropped window grid	yes
3	key_features never reached the image prompt	code review after failure 1	no
4	prompts truncated at 77 tokens	validator token-budget check	yes

The fifth was a metric bug: the orientation classifier could only emit three of the five declared labels, so every style declaring curved or diagonal scored zero and looked like a total generator failure. Fixing the measurement raised orientation compliance from 60% to 83%.

A broken metric can invent a failure as easily as a broken generator can hide one. Both argue for the same thing: instrumented acceptance criteria, not visual inspection.

---

Limitations

• Visual-similarity search, not authoritative identification.
• No plate depicts a real building; none is attributable to any architect.
• The real-photograph evaluation is small (n = 17) and unevenly distributed across styles.
• Style boundaries blur where visual grammars genuinely overlap.
• Location reranking depends on OpenStreetMap tag density, which varies enormously by city — Paris returns far more than Tel Aviv.
• The app makes no claim about any building's date, architect, or heritage status, and both models are explicitly prompted not to.

Using it

Include the whole building where you can. Style lives in massing, roofline and silhouette, and a cropped window grid discards all three.

For the location features, type a place with its country — "Rothschild Boulevard, Tel Aviv, Israel" — and check the resolved name before running. Geocoders will happily match a street name on the wrong continent.