Spaces:
Sleeping
Sleeping
Update README.md
Browse files
README.md
CHANGED
|
@@ -11,70 +11,77 @@ pinned: false
|
|
| 11 |
license: mit
|
| 12 |
---
|
| 13 |
|
| 14 |
-
|
| 15 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
> An advanced, multi-modal recommendation system leveraging Deep Learning (CLIP), Unsupervised Clustering (K-Means), and Generative AI to understand stylistic intent and retrieve context-aware interior design architecture.
|
| 20 |
|
| 21 |
---
|
| 22 |
|
| 23 |
-
## 📖
|
| 24 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
---
|
| 27 |
|
| 28 |
## 🗄️ 1. Dataset Architecture
|
| 29 |
|
| 30 |
-
For the foundation of this engine,
|
| 31 |
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
|
|
|
| 38 |
|
| 39 |
---
|
| 40 |
|
| 41 |
## 📊 2. Exploratory Data Analysis (EDA)
|
| 42 |
|
| 43 |
-
Before feeding data into deep neural networks,
|
| 44 |
|
| 45 |
### 2.1 Initial Sanity Checks & Data Integrity
|
| 46 |
-
* **Missing Value Verification:**
|
| 47 |
-
* **Dimension Uniformity:** Batch tensor operations require exact spatial dimensions.
|
| 48 |
|
| 49 |
### 2.2 Class Distribution Analysis
|
| 50 |
Understanding the distribution of our labels is critical to identifying potential model biases.
|
| 51 |
-
* **Room Types:**
|
| 52 |
* **Interior Styles:** The stylistic distribution is more balanced, gently sloping from ~4,000 samples down to ~2,100 in the least frequent class.
|
| 53 |
* **Takeaway:** While the model has ample data to generate robust embeddings across all vectors, the recommendation engine might naturally perform with higher confidence on Living Rooms due to broader exposure during the embedding phase.
|
| 54 |
|
| 55 |
<div align="center">
|
| 56 |
-
<img src="
|
| 57 |
<br>
|
| 58 |
<em>Figure 1: Distribution of Room Types and Interior Styles across the dataset.</em>
|
| 59 |
</div>
|
| 60 |
|
| 61 |
### 2.3 RGB Channel Distribution
|
| 62 |
Unlike tabular data, images are composed of pixel intensities across Red, Green, and Blue channels.
|
| 63 |
-
* **Insights:** The generated histogram reveals a dataset that is predominantly bright and well-lit. Notably, the **Red channel dominates** the higher intensity ranges. This perfectly aligns with real-world interior photography, which heavily features warm ambient lighting, wooden textures, and earth-toned furniture.
|
| 64 |
|
| 65 |
<div align="center">
|
| 66 |
-
<img src="
|
| 67 |
<br>
|
| 68 |
<em>Figure 2: RGB Pixel Intensity Distribution highlighting warm-tone dominance.</em>
|
| 69 |
</div>
|
| 70 |
|
| 71 |
### 2.4 Spatial Variance & The "Mean Image"
|
| 72 |
-
To identify structural patterns,
|
| 73 |
* **Insights:** The resulting images were highly amorphous and blurry, lacking distinct structural boundaries like beds or counters. This proves a high degree of spatial variance—rooms are photographed from vastly different angles and depths.
|
| 74 |
-
* **Conclusion:** This strictly validates
|
| 75 |
|
| 76 |
<div align="center">
|
| 77 |
-
<img src="
|
| 78 |
<br>
|
| 79 |
<em>Figure 3: Mean Image representation showing high spatial variance.</em>
|
| 80 |
</div>
|
|
@@ -83,13 +90,13 @@ To identify structural patterns, we computed the mathematical "Mean Image" by av
|
|
| 83 |
|
| 84 |
## 🧠 3. Semantic Embeddings & Clustering Pipeline
|
| 85 |
|
| 86 |
-
|
| 87 |
|
| 88 |
### 3.1 Feature Extraction (CLIP)
|
| 89 |
-
|
| 90 |
|
| 91 |
### 3.2 Unsupervised K-Means Clustering
|
| 92 |
-
To test the quality of these embeddings,
|
| 93 |
|
| 94 |
### 3.3 Dimensionality Reduction (t-SNE) & Coherence
|
| 95 |
Projecting 512 dimensions down to a 2D visual scatter plot via t-SNE yielded outstanding results. Rather than a chaotic cloud, the data formed highly cohesive "islands."
|
|
@@ -101,33 +108,27 @@ By cross-tabulating the generated clusters against actual room labels, distinct
|
|
| 101 |
* **Clusters 2 & 3 (Soft-Furnishings):** A logical blending of Bedrooms and Living Rooms, driven by shared semantic textures (rugs, beds, upholstery).
|
| 102 |
|
| 103 |
<div align="center">
|
| 104 |
-
<img src="
|
| 105 |
<br>
|
| 106 |
<em>Figure 4: t-SNE 2D Projection showing distinct semantic clustering of 512D embeddings.</em>
|
| 107 |
</div>
|
| 108 |
|
| 109 |
-
<div align="center">
|
| 110 |
-
<img src="REPLACE_THIS_WITH_IMAGE_LINK.png" alt="Cross-Tabulation Matrix" width="600"/>
|
| 111 |
-
<br>
|
| 112 |
-
<em>Figure 5: Cross-tabulation proving alignment between unsupervised clusters and ground-truth labels.</em>
|
| 113 |
-
</div>
|
| 114 |
-
|
| 115 |
---
|
| 116 |
|
| 117 |
## 🚀 4. Recommendation Engine & Final Application
|
| 118 |
|
| 119 |
-
The highly coherent 512D matrix was exported as an optimized `.parquet` file to serve as the
|
| 120 |
|
| 121 |
### System Architecture:
|
| 122 |
-
1. **Input Processing:** When a user interacts with the Gradio UI—either by uploading an inspiration photo or typing a conceptual text prompt—the system passes the input through the CLIP processor, translating it into the exact mathematical vector space as
|
| 123 |
-
2. **Cosine Similarity:** The engine measures the geometric angle between the user's vector and the 5,000 vectors in
|
| 124 |
3. **Top-K Retrieval:** The algorithm ranks the closest spatial matches and retrieves the top 3 results alongside their exact confidence scores.
|
| 125 |
|
| 126 |
<div align="center">
|
| 127 |
-
<img src="
|
| 128 |
<br>
|
| 129 |
-
<em>Figure
|
| 130 |
</div>
|
| 131 |
|
| 132 |
-
### 🎯
|
| 133 |
-
Testing the complete pipeline confirmed that the system
|
|
|
|
| 11 |
license: mit
|
| 12 |
---
|
| 13 |
|
| 14 |
+
<h1 align="center">**VISIONARY: AI-Powered Interior Design Engine** 🏡</h1>
|
| 15 |
|
| 16 |
+
<div align="center">
|
| 17 |
+
|
| 18 |
+
**Matan Zigelman**
|
| 19 |
+
*Reichman University*
|
| 20 |
|
| 21 |
+
</div>
|
|
|
|
|
|
|
| 22 |
|
| 23 |
---
|
| 24 |
|
| 25 |
+
## 📖 Project Overview
|
| 26 |
+
Traditional visual search engines often rely on basic pixel-matching or color histograms, which struggle to grasp the structural and semantic nuances of architectural spaces. **Visionary** was developed as a comprehensive academic and technical showcase to solve this limitation.
|
| 27 |
+
|
| 28 |
+
By utilizing the `openai/clip-vit-base-patch32` vision-language model, this project projects raw images into a 512-dimensional semantic space. As a result, the system autonomously understands abstract concepts—such as "minimalist," "industrial," or "warm lighting"—and retrieves highly accurate room designs based on user inspiration photos or textual descriptions.
|
| 29 |
+
|
| 30 |
+
**Tech Stack:** `Python` | `PyTorch` | `Hugging Face (Transformers, Datasets)` | `Scikit-Learn (K-Means, t-SNE)` | `Pandas` | `Gradio`
|
| 31 |
|
| 32 |
---
|
| 33 |
|
| 34 |
## 🗄️ 1. Dataset Architecture
|
| 35 |
|
| 36 |
+
For the foundation of this engine, I selected the `my_interior_design_dataset` from Hugging Face. This dataset is exceptionally well-suited for high-level visual recommendation tasks.
|
| 37 |
|
| 38 |
+
| Feature | Details |
|
| 39 |
+
| :--- | :--- |
|
| 40 |
+
| **Total Volume** | 33,453 images (~33,500 in the training split). |
|
| 41 |
+
| **Storage Size** | 535 MB. |
|
| 42 |
+
| **Standardization**| All image matrices are perfectly uniform at a `256x256` pixel resolution. |
|
| 43 |
+
| **Style Labels** | 11 distinct design classes (e.g., *Rustic, Bohemian, Transitional*). |
|
| 44 |
+
| **Room Labels** | 5 functional architectural classes (e.g., *Bedroom, Kitchen, Living Room*). |
|
| 45 |
|
| 46 |
---
|
| 47 |
|
| 48 |
## 📊 2. Exploratory Data Analysis (EDA)
|
| 49 |
|
| 50 |
+
Before feeding data into deep neural networks, I conducted rigorous Exploratory Data Analysis to validate structural integrity and uncover hidden visual biases.
|
| 51 |
|
| 52 |
### 2.1 Initial Sanity Checks & Data Integrity
|
| 53 |
+
* **Missing Value Verification:** I inspected the metadata arrays and confirmed **zero missing values (NaNs)**. Every visual sample is explicitly paired with valid ground-truth labels.
|
| 54 |
+
* **Dimension Uniformity:** Batch tensor operations require exact spatial dimensions. The checks passed flawlessly—100% of the tested samples maintain the strict 256x256 boundary, eliminating the need for dynamic resizing during training.
|
| 55 |
|
| 56 |
### 2.2 Class Distribution Analysis
|
| 57 |
Understanding the distribution of our labels is critical to identifying potential model biases.
|
| 58 |
+
* **Room Types:** There is a moderate class imbalance. *Class 4 (Living Room)* is the dominant category with >10,000 samples, whereas *Class 3* contains ~4,000.
|
| 59 |
* **Interior Styles:** The stylistic distribution is more balanced, gently sloping from ~4,000 samples down to ~2,100 in the least frequent class.
|
| 60 |
* **Takeaway:** While the model has ample data to generate robust embeddings across all vectors, the recommendation engine might naturally perform with higher confidence on Living Rooms due to broader exposure during the embedding phase.
|
| 61 |
|
| 62 |
<div align="center">
|
| 63 |
+
<img src="https://huggingface.co/spaces/matanzig/Interior-Design-GenAI/resolve/main/plot.png" alt="Class Distribution Bar Charts" />
|
| 64 |
<br>
|
| 65 |
<em>Figure 1: Distribution of Room Types and Interior Styles across the dataset.</em>
|
| 66 |
</div>
|
| 67 |
|
| 68 |
### 2.3 RGB Channel Distribution
|
| 69 |
Unlike tabular data, images are composed of pixel intensities across Red, Green, and Blue channels.
|
| 70 |
+
* **Insights:** The generated histogram reveals a dataset that is predominantly bright and well-lit. Notably, the **Red channel dominates** the higher intensity ranges. This perfectly aligns with real-world interior photography, which heavily features warm ambient lighting, wooden textures, and earth-toned furniture. The CLIP model utilizes these specific warm-light biases as strong features for calculating room similarities.
|
| 71 |
|
| 72 |
<div align="center">
|
| 73 |
+
<img src="https://huggingface.co/spaces/matanzig/Interior-Design-GenAI/resolve/main/rgb.png" alt="RGB Histogram" />
|
| 74 |
<br>
|
| 75 |
<em>Figure 2: RGB Pixel Intensity Distribution highlighting warm-tone dominance.</em>
|
| 76 |
</div>
|
| 77 |
|
| 78 |
### 2.4 Spatial Variance & The "Mean Image"
|
| 79 |
+
To identify structural patterns, I computed the mathematical "Mean Image" by averaging pixel arrays within specific classes (e.g., Bedrooms).
|
| 80 |
* **Insights:** The resulting images were highly amorphous and blurry, lacking distinct structural boundaries like beds or counters. This proves a high degree of spatial variance—rooms are photographed from vastly different angles and depths.
|
| 81 |
+
* **Conclusion:** This strictly validates the decision to use Deep Learning embeddings over traditional computer vision techniques; the system requires a model that detects *semantic objects*, regardless of where they are spatially located in the frame.
|
| 82 |
|
| 83 |
<div align="center">
|
| 84 |
+
<img src="https://huggingface.co/spaces/matanzig/Interior-Design-GenAI/resolve/main/mid.png" alt="Mean Images by Room Type" />
|
| 85 |
<br>
|
| 86 |
<em>Figure 3: Mean Image representation showing high spatial variance.</em>
|
| 87 |
</div>
|
|
|
|
| 90 |
|
| 91 |
## 🧠 3. Semantic Embeddings & Clustering Pipeline
|
| 92 |
|
| 93 |
+
I extracted a randomized, reproducible subset of 5,000 images (`seed=42`) to optimize computational efficiency while fully capturing the dataset's architectural diversity.
|
| 94 |
|
| 95 |
### 3.1 Feature Extraction (CLIP)
|
| 96 |
+
I deployed Hugging Face's `openai/clip-vit-base-patch32`. Leveraging PyTorch (`torch.no_grad()`) and GPU acceleration, the visual data was processed through the transformer, extracting a highly dense **512-dimensional vector** for each image. These vectors capture complex stylistic signatures.
|
| 97 |
|
| 98 |
### 3.2 Unsupervised K-Means Clustering
|
| 99 |
+
To test the quality of these embeddings, I applied a K-Means clustering algorithm, explicitly instructing it to find 5 clusters (`n_clusters=5`) to mirror the 5 ground-truth room types.
|
| 100 |
|
| 101 |
### 3.3 Dimensionality Reduction (t-SNE) & Coherence
|
| 102 |
Projecting 512 dimensions down to a 2D visual scatter plot via t-SNE yielded outstanding results. Rather than a chaotic cloud, the data formed highly cohesive "islands."
|
|
|
|
| 108 |
* **Clusters 2 & 3 (Soft-Furnishings):** A logical blending of Bedrooms and Living Rooms, driven by shared semantic textures (rugs, beds, upholstery).
|
| 109 |
|
| 110 |
<div align="center">
|
| 111 |
+
<img src="https://huggingface.co/spaces/matanzig/Interior-Design-GenAI/resolve/main/tsne.png" alt="t-SNE Scatter Plot" />
|
| 112 |
<br>
|
| 113 |
<em>Figure 4: t-SNE 2D Projection showing distinct semantic clustering of 512D embeddings.</em>
|
| 114 |
</div>
|
| 115 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 116 |
---
|
| 117 |
|
| 118 |
## 🚀 4. Recommendation Engine & Final Application
|
| 119 |
|
| 120 |
+
The highly coherent 512D matrix was exported as an optimized `.parquet` file to serve as the core engine of the production application.
|
| 121 |
|
| 122 |
### System Architecture:
|
| 123 |
+
1. **Input Processing:** When a user interacts with the Gradio UI—either by uploading an inspiration photo or typing a conceptual text prompt—the system passes the input through the CLIP processor, translating it into the exact mathematical vector space as the database.
|
| 124 |
+
2. **Cosine Similarity:** The engine measures the geometric angle between the user's vector and the 5,000 vectors in the catalog.
|
| 125 |
3. **Top-K Retrieval:** The algorithm ranks the closest spatial matches and retrieves the top 3 results alongside their exact confidence scores.
|
| 126 |
|
| 127 |
<div align="center">
|
| 128 |
+
<img src="https://huggingface.co/spaces/matanzig/Interior-Design-GenAI/resolve/main/pic.png" alt="Recommendation Engine Results" />
|
| 129 |
<br>
|
| 130 |
+
<em>Figure 5: System successfully matching a textual/visual input to real-world interior designs.</em>
|
| 131 |
</div>
|
| 132 |
|
| 133 |
+
### 🎯 Conclusion & Technical Achievements
|
| 134 |
+
Testing the complete pipeline confirmed that the system goes far beyond matching raw colors. It successfully extracts and matches abstract architectural concepts and stylistic intent. The backend logic proved highly reliable and was successfully deployed as a fully interactive Hugging Face Space frontend application, demonstrating a complete end-to-end Machine Learning lifecycle.
|