README.md

---
title: Facade
emoji: 🏛
colorFrom: blue
colorTo: red
sdk: gradio
app_file: app.py
pinned: false
license: mit
short_description: Photograph a building, find its architectural style
---

# Facade

**Point a camera at a building. Find out what you are looking at.**

Upload a photograph of a facade and the app returns the three closest
architectural styles from a 1,000-plate synthetic reference corpus, a reading
written for *your* photograph at query time, the measurements behind the match,
and real named buildings near you in the same style.

- **Dataset:** [Jonathandav/facade-styles](https://huggingface.co/datasets/Jonathandav/facade-styles)
- **Corpus:** 1,000 generated plates · 20 styles · 50 each
- **Retrieval model:** `laion/CLIP-ViT-B-32-laion2B-s34B-b79K`
- **Reading model:** `Qwen/Qwen2.5-1.5B-Instruct`

---

## The idea

Architectural style is a family resemblance rather than a fact about a
building. That makes it a good target for *synthetic* reference data: a
generated Bauhaus facade **is** a Bauhaus facade, because there is no external
specimen it can contradict.

It also makes style unusually measurable. Brutalism is low-saturation and
heavy-massed; Gothic Revival is vertical-edge dominant; Bauhaus is horizontal;
Ottoman Revival is curve-rich. Every one of those has a pixel-level correlate,
so "did the generator do what it was told?" is a computed quantity rather than
an opinion.

The whole project is built on that: a corpus specified attribute by attribute,
audited by measurement, and finally tested against real photographs.

---

## How it works

```
 style_seed.csv          20 styles, each with massing, material, window
       │                 rhythm, roofline, palette, and two EXPECTED
       │                 measurements (saturation band, edge orientation)
       ▼
 sample_plates()         50 specs per style; view / crop / light / condition
       │                 CYCLED, not sampled — so they cannot correlate
       ▼                 with style
 SDXL-Turbo              style name + period + key features lead the prompt
       │                 acceptance gate measures every render and retries
       ▼                 with a fresh seed until it passes
 1,000 plates ──► LAION-CLIP embeddings ──► index_embeddings.npy
       │
       ▼
 Qwen2.5-1.5B            style readings, barred from naming architects,
                         asserting dates, or claiming heritage status
```

At query time: embed the photograph → cosine similarity against the index →
optionally rerank with an OpenStreetMap prior over what actually stands nearby
→ generate a reading from the matched style, the runner-up, and the
measurements taken off the photograph.

---

## What the corpus can identify

Bauhaus International · Brutalist · Art Deco · Ottoman Revival · Mandate
Eclectic · Neoclassical · Gothic Revival · Venetian Gothic · Georgian ·
Victorian Terrace · Mid-century Modern · Postmodern · Deconstructivist ·
Contemporary Curtain Wall · Mediterranean Vernacular · Adobe Pueblo · Soviet
Constructivist · Scandinavian Functionalist · Industrial Warehouse · Japanese
Metabolist

Anything outside these twenty is still forced onto the nearest of them. The app
says so in the interface, because a user should know that before trusting a
result.

---

## Did the generator obey?

![Declared vs measured saturation, and compliance by style](EDA1.png)

Each style declares an expected saturation level. That is an **ordinal** claim —
brutalism less saturated than Art Deco — not a value on a calibrated scale.

**Spearman ρ = 0.67, p ≈ 1.5 × 10⁻¹³², n = 1000.** Declared rank predicts
measured saturation strongly.

The first version of this test scored **8%** compliance, because the bands were
absolute thresholds chosen before any image existed: measured saturation ran
0.30–0.85 while "medium" had been guessed at 0.22–0.50. Recalibrating the bands
to corpus quantiles and reporting rank agreement is the correct test, and it is
a better methodological story than thresholds that happened to work first time.

---

## Where will it get confused?

![Style centroids in measured feature space, and centroid distance matrix](EDA2.png)

Plotting each style's centroid in saturation × edge-orientation space produces
a **prediction, before any embedding model runs**, of which pairs will be
conflated:

| pair | centroid distance |
|---|---|
| Neoclassical ↔ Postmodern | 0.41 |
| Japanese Metabolist ↔ Scandinavian Functionalist | 0.62 |
| Mandate Eclectic ↔ Neoclassical | 0.65 |
| Brutalist ↔ Japanese Metabolist | 0.65 |
| Georgian ↔ Victorian Terrace | 0.75 |

These are architecturally meaningful, not arbitrary. Postmodernism is *defined*
by ironic quotation of classical form. Mandate Eclectic borrowed classical
proportion directly. Brutalism and Metabolism are contemporaneous raw-concrete
movements. A purely pixel-level metric recovered relationships an architectural
historian would recognise.

**Georgian ↔ Victorian Terrace was confirmed** by the CLIP confusion matrix at
0.16 — the top confusion by a wide margin. The others were not, and CLIP's
actual confusions cluster differently: Bauhaus International acts as an
*attractor* for the modernist styles, with Brutalist, Soviet Constructivist and
Japanese Metabolist all leaking toward it. That is a semantic effect the pixel
metric cannot see.

One prediction confirmed and four refuted with a coherent explanation is a
better result than five vague hits.

---

## Does style dominate the nuisance factors?

![Variance explained by each factor, and style's margin over the strongest nuisance](EDA3.png)

Style explains **43–66%** of variance in every measured feature. Crop, view and
condition sit at essentially **zero** — which validates the sampler: those axes
are cycled rather than drawn independently, so they cannot correlate with
style. Had they done so, an embedding model could have scored well by learning
framing instead of architecture.

Light is the only meaningful contaminant, and only on the photometric features,
which is physically unsurprising.

---

## Retrieval

Image-to-image over the corpus, style as the label, chance ≈ 0.049.

| model | Recall@1 | Recall@3 | MRR | lift |
|---|---|---|---|---|
| SigLIP-B/16 | 0.972 | 0.994 | 0.983 | 19.8× |
| LAION-CLIP-B/32 | 0.970 | 0.991 | 0.981 | 19.8× |
| CLIP-B/32 | 0.955 | 0.988 | 0.972 | 19.5× |

**These are at ceiling and the comparison cannot resolve.** Plates of the same
style share nearly identical prompts, differing only in angle, crop, light and
condition, so they are near-siblings. Every slice falls between 0.975 and
1.000. The differences between models are inside noise.

---

## Sim-to-real — the result that matters

17 hand-labelled photographs of real buildings in Tel Aviv, Jaffa, Paris and
the Greek islands, retrieved against the wholly synthetic index.

| model | real R@3 | synthetic R@3 | gap |
|---|---|---|---|
| **LAION-CLIP-B/32** | **0.94** (16/17) | 0.991 | **−0.05** |
| SigLIP-B/16 | 0.88 (15/17) | 0.994 | −0.11 |
| CLIP-B/32 | 0.76 (13/17) | 0.988 | −0.22 |

The synthetic benchmark ranked SigLIP first by 0.002 on Recall@1. On real
photographs LAION wins and has less than half the domain gap. **Selecting on
the saturated benchmark would have shipped the weaker model.** A benchmark at
ceiling does not merely fail to discriminate — it discriminates wrongly.

Both failure cases are architecturally sensible: a Bauhaus building retrieved
as Soviet Constructivist (contemporaneous interwar modernism, strip windows,
white render), and a night photograph of a Mandate building retrieved as
Mediterranean Vernacular (both cream render with shutters).

---

## Generator failures found along the way

Four, plus one measurement bug. **Three of the four were silent** — no
exception, no warning, and plausible-looking output.

| # | failure | how it was caught | silent |
|---|---|---|---|
| 1 | SDXL's pooled embedding overrode every style | Bauhaus, Brutalist and Gothic all rendered as the same beige office block | yes |
| 2 | detail crops destroyed the class signal | style unrecoverable from a cropped window grid | yes |
| 3 | `key_features` never reached the image prompt | code review after failure 1 | no |
| 4 | prompts truncated at 77 tokens | validator token-budget check | yes |

The fifth was a **metric** bug: the orientation classifier could only emit three
of the five declared labels, so every style declaring `curved` or `diagonal`
scored zero and looked like a total generator failure. Fixing the measurement
raised orientation compliance from 60% to 83%.

A broken metric can invent a failure as easily as a broken generator can hide
one. Both argue for the same thing: instrumented acceptance criteria, not
visual inspection.

---

## Limitations

- Visual-similarity search, **not** authoritative identification.
- No plate depicts a real building; none is attributable to any architect.
- The real-photograph evaluation is small (n = 17) and unevenly distributed
  across styles.
- Style boundaries blur where visual grammars genuinely overlap.
- Location reranking depends on OpenStreetMap tag density, which varies
  enormously by city — Paris returns far more than Tel Aviv.
- The app makes no claim about any building's date, architect, or heritage
  status, and both models are explicitly prompted not to.

## Using it

Include the whole building where you can. Style lives in massing, roofline and
silhouette, and a cropped window grid discards all three.

For the location features, type a place with its country — "Rothschild
Boulevard, Tel Aviv, Israel" — and check the resolved name before running.
Geocoders will happily match a street name on the wrong continent.