--- title: Facade emoji: 🏛 colorFrom: blue colorTo: red sdk: gradio app_file: app.py pinned: false license: mit short_description: Photograph a building, find its architectural style --- # Facade **Point a camera at a building. Find out what you are looking at.** Upload a photograph of a facade and the app returns the three closest architectural styles from a 1,000-plate synthetic reference corpus, a reading written for *your* photograph at query time, the measurements behind the match, and real named buildings near you in the same style. - **Dataset:** [Jonathandav/facade-styles](https://huggingface.co/datasets/Jonathandav/facade-styles) - **Corpus:** 1,000 generated plates · 20 styles · 50 each - **Retrieval model:** `laion/CLIP-ViT-B-32-laion2B-s34B-b79K` - **Reading model:** `Qwen/Qwen2.5-1.5B-Instruct` --- ## The idea Architectural style is a family resemblance rather than a fact about a building. That makes it a good target for *synthetic* reference data: a generated Bauhaus facade **is** a Bauhaus facade, because there is no external specimen it can contradict. It also makes style unusually measurable. Brutalism is low-saturation and heavy-massed; Gothic Revival is vertical-edge dominant; Bauhaus is horizontal; Ottoman Revival is curve-rich. Every one of those has a pixel-level correlate, so "did the generator do what it was told?" is a computed quantity rather than an opinion. The whole project is built on that: a corpus specified attribute by attribute, audited by measurement, and finally tested against real photographs. --- ## How it works ``` style_seed.csv 20 styles, each with massing, material, window │ rhythm, roofline, palette, and two EXPECTED │ measurements (saturation band, edge orientation) ▼ sample_plates() 50 specs per style; view / crop / light / condition │ CYCLED, not sampled — so they cannot correlate ▼ with style SDXL-Turbo style name + period + key features lead the prompt │ acceptance gate measures every render and retries ▼ with a fresh seed until it passes 1,000 plates ──► LAION-CLIP embeddings ──► index_embeddings.npy │ ▼ Qwen2.5-1.5B style readings, barred from naming architects, asserting dates, or claiming heritage status ``` At query time: embed the photograph → cosine similarity against the index → optionally rerank with an OpenStreetMap prior over what actually stands nearby → generate a reading from the matched style, the runner-up, and the measurements taken off the photograph. --- ## What the corpus can identify Bauhaus International · Brutalist · Art Deco · Ottoman Revival · Mandate Eclectic · Neoclassical · Gothic Revival · Venetian Gothic · Georgian · Victorian Terrace · Mid-century Modern · Postmodern · Deconstructivist · Contemporary Curtain Wall · Mediterranean Vernacular · Adobe Pueblo · Soviet Constructivist · Scandinavian Functionalist · Industrial Warehouse · Japanese Metabolist Anything outside these twenty is still forced onto the nearest of them. The app says so in the interface, because a user should know that before trusting a result. --- ## Did the generator obey? ![Declared vs measured saturation, and compliance by style](EDA1.png) Each style declares an expected saturation level. That is an **ordinal** claim — brutalism less saturated than Art Deco — not a value on a calibrated scale. **Spearman ρ = 0.67, p ≈ 1.5 × 10⁻¹³², n = 1000.** Declared rank predicts measured saturation strongly. The first version of this test scored **8%** compliance, because the bands were absolute thresholds chosen before any image existed: measured saturation ran 0.30–0.85 while "medium" had been guessed at 0.22–0.50. Recalibrating the bands to corpus quantiles and reporting rank agreement is the correct test, and it is a better methodological story than thresholds that happened to work first time. --- ## Where will it get confused? ![Style centroids in measured feature space, and centroid distance matrix](EDA2.png) Plotting each style's centroid in saturation × edge-orientation space produces a **prediction, before any embedding model runs**, of which pairs will be conflated: | pair | centroid distance | |---|---| | Neoclassical ↔ Postmodern | 0.41 | | Japanese Metabolist ↔ Scandinavian Functionalist | 0.62 | | Mandate Eclectic ↔ Neoclassical | 0.65 | | Brutalist ↔ Japanese Metabolist | 0.65 | | Georgian ↔ Victorian Terrace | 0.75 | These are architecturally meaningful, not arbitrary. Postmodernism is *defined* by ironic quotation of classical form. Mandate Eclectic borrowed classical proportion directly. Brutalism and Metabolism are contemporaneous raw-concrete movements. A purely pixel-level metric recovered relationships an architectural historian would recognise. **Georgian ↔ Victorian Terrace was confirmed** by the CLIP confusion matrix at 0.16 — the top confusion by a wide margin. The others were not, and CLIP's actual confusions cluster differently: Bauhaus International acts as an *attractor* for the modernist styles, with Brutalist, Soviet Constructivist and Japanese Metabolist all leaking toward it. That is a semantic effect the pixel metric cannot see. One prediction confirmed and four refuted with a coherent explanation is a better result than five vague hits. --- ## Does style dominate the nuisance factors? ![Variance explained by each factor, and style's margin over the strongest nuisance](EDA3.png) Style explains **43–66%** of variance in every measured feature. Crop, view and condition sit at essentially **zero** — which validates the sampler: those axes are cycled rather than drawn independently, so they cannot correlate with style. Had they done so, an embedding model could have scored well by learning framing instead of architecture. Light is the only meaningful contaminant, and only on the photometric features, which is physically unsurprising. --- ## Retrieval Image-to-image over the corpus, style as the label, chance ≈ 0.049. | model | Recall@1 | Recall@3 | MRR | lift | |---|---|---|---|---| | SigLIP-B/16 | 0.972 | 0.994 | 0.983 | 19.8× | | LAION-CLIP-B/32 | 0.970 | 0.991 | 0.981 | 19.8× | | CLIP-B/32 | 0.955 | 0.988 | 0.972 | 19.5× | **These are at ceiling and the comparison cannot resolve.** Plates of the same style share nearly identical prompts, differing only in angle, crop, light and condition, so they are near-siblings. Every slice falls between 0.975 and 1.000. The differences between models are inside noise. --- ## Sim-to-real — the result that matters 17 hand-labelled photographs of real buildings in Tel Aviv, Jaffa, Paris and the Greek islands, retrieved against the wholly synthetic index. | model | real R@3 | synthetic R@3 | gap | |---|---|---|---| | **LAION-CLIP-B/32** | **0.94** (16/17) | 0.991 | **−0.05** | | SigLIP-B/16 | 0.88 (15/17) | 0.994 | −0.11 | | CLIP-B/32 | 0.76 (13/17) | 0.988 | −0.22 | The synthetic benchmark ranked SigLIP first by 0.002 on Recall@1. On real photographs LAION wins and has less than half the domain gap. **Selecting on the saturated benchmark would have shipped the weaker model.** A benchmark at ceiling does not merely fail to discriminate — it discriminates wrongly. Both failure cases are architecturally sensible: a Bauhaus building retrieved as Soviet Constructivist (contemporaneous interwar modernism, strip windows, white render), and a night photograph of a Mandate building retrieved as Mediterranean Vernacular (both cream render with shutters). --- ## Generator failures found along the way Four, plus one measurement bug. **Three of the four were silent** — no exception, no warning, and plausible-looking output. | # | failure | how it was caught | silent | |---|---|---|---| | 1 | SDXL's pooled embedding overrode every style | Bauhaus, Brutalist and Gothic all rendered as the same beige office block | yes | | 2 | detail crops destroyed the class signal | style unrecoverable from a cropped window grid | yes | | 3 | `key_features` never reached the image prompt | code review after failure 1 | no | | 4 | prompts truncated at 77 tokens | validator token-budget check | yes | The fifth was a **metric** bug: the orientation classifier could only emit three of the five declared labels, so every style declaring `curved` or `diagonal` scored zero and looked like a total generator failure. Fixing the measurement raised orientation compliance from 60% to 83%. A broken metric can invent a failure as easily as a broken generator can hide one. Both argue for the same thing: instrumented acceptance criteria, not visual inspection. --- ## Limitations - Visual-similarity search, **not** authoritative identification. - No plate depicts a real building; none is attributable to any architect. - The real-photograph evaluation is small (n = 17) and unevenly distributed across styles. - Style boundaries blur where visual grammars genuinely overlap. - Location reranking depends on OpenStreetMap tag density, which varies enormously by city — Paris returns far more than Tel Aviv. - The app makes no claim about any building's date, architect, or heritage status, and both models are explicitly prompted not to. ## Using it Include the whole building where you can. Style lives in massing, roofline and silhouette, and a cropped window grid discards all three. For the location features, type a place with its country — "Rothschild Boulevard, Tel Aviv, Israel" — and check the resolved name before running. Geocoders will happily match a street name on the wrong continent.