matanzig commited on
Commit
8f86e3d
·
verified ·
1 Parent(s): 7874cf0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +41 -40
README.md CHANGED
@@ -11,70 +11,77 @@ pinned: false
11
  license: mit
12
  ---
13
 
14
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
15
 
 
 
 
 
16
 
17
- # 🏡 VISIONARY: AI-Powered Interior Design Engine
18
-
19
- > An advanced, multi-modal recommendation system leveraging Deep Learning (CLIP), Unsupervised Clustering (K-Means), and Generative AI to understand stylistic intent and retrieve context-aware interior design architecture.
20
 
21
  ---
22
 
23
- ## 📖 Executive Summary
24
- Welcome to **Visionary**. Traditional visual search engines rely on basic pixel-matching or color histograms, which often fail to grasp the structural and semantic nuances of interior design. This project solves that by utilizing the `openai/clip-vit-base-patch32` vision-language model to project raw images into a 512-dimensional semantic space. By doing so, the system understands abstract concepts—such as "minimalist," "industrial," or "warm lighting"—and retrieves highly accurate room designs based on user inspiration photos or textual descriptions.
 
 
 
 
25
 
26
  ---
27
 
28
  ## 🗄️ 1. Dataset Architecture
29
 
30
- For the foundation of this engine, we selected the `my_interior_design_dataset` from Hugging Face. This dataset is exceptionally well-suited for high-level visual recommendation tasks.
31
 
32
- * **Total Volume:** 33,453 images (~33,500 in the training split).
33
- * **Storage Size:** 535 MB.
34
- * **Standardization:** All image matrices are perfectly uniform at a `256x256` pixel resolution.
35
- * **Categorical Richness:** Every sample is enriched with two primary ground-truth labels:
36
- * **Style:** 11 distinct design classes (e.g., *Rustic, Bohemian, Transitional, Mid-Century Modern*).
37
- * **Room:** 5 functional architectural classes (e.g., *Bedroom, Kitchen, Dining Room, Living Room, Bathroom*).
 
38
 
39
  ---
40
 
41
  ## 📊 2. Exploratory Data Analysis (EDA)
42
 
43
- Before feeding data into deep neural networks, we conducted rigorous Exploratory Data Analysis to validate structural integrity and uncover hidden visual biases.
44
 
45
  ### 2.1 Initial Sanity Checks & Data Integrity
46
- * **Missing Value Verification:** We inspected the metadata arrays and confirmed **zero missing values (NaNs)**. Every visual sample is explicitly paired with valid ground-truth labels.
47
- * **Dimension Uniformity:** Batch tensor operations require exact spatial dimensions. Our checks passed flawlessly—100% of the tested samples maintain the strict 256x256 boundary, eliminating the need for dynamic resizing during training.
48
 
49
  ### 2.2 Class Distribution Analysis
50
  Understanding the distribution of our labels is critical to identifying potential model biases.
51
- * **Room Types:** We observed a moderate class imbalance. *Class 4 (Living Room)* is the dominant category with >10,000 samples, whereas *Class 3* contains ~4,000.
52
  * **Interior Styles:** The stylistic distribution is more balanced, gently sloping from ~4,000 samples down to ~2,100 in the least frequent class.
53
  * **Takeaway:** While the model has ample data to generate robust embeddings across all vectors, the recommendation engine might naturally perform with higher confidence on Living Rooms due to broader exposure during the embedding phase.
54
 
55
  <div align="center">
56
- <img src="REPLACE_THIS_WITH_IMAGE_LINK.png" alt="Class Distribution Bar Charts" width="800"/>
57
  <br>
58
  <em>Figure 1: Distribution of Room Types and Interior Styles across the dataset.</em>
59
  </div>
60
 
61
  ### 2.3 RGB Channel Distribution
62
  Unlike tabular data, images are composed of pixel intensities across Red, Green, and Blue channels.
63
- * **Insights:** The generated histogram reveals a dataset that is predominantly bright and well-lit. Notably, the **Red channel dominates** the higher intensity ranges. This perfectly aligns with real-world interior photography, which heavily features warm ambient lighting, wooden textures, and earth-toned furniture. Our CLIP model utilizes these specific warm-light biases as strong features for calculating room similarities.
64
 
65
  <div align="center">
66
- <img src="REPLACE_THIS_WITH_IMAGE_LINK.png" alt="RGB Histogram" width="600"/>
67
  <br>
68
  <em>Figure 2: RGB Pixel Intensity Distribution highlighting warm-tone dominance.</em>
69
  </div>
70
 
71
  ### 2.4 Spatial Variance & The "Mean Image"
72
- To identify structural patterns, we computed the mathematical "Mean Image" by averaging pixel arrays within specific classes (e.g., Bedrooms).
73
  * **Insights:** The resulting images were highly amorphous and blurry, lacking distinct structural boundaries like beds or counters. This proves a high degree of spatial variance—rooms are photographed from vastly different angles and depths.
74
- * **Conclusion:** This strictly validates our decision to use Deep Learning embeddings over traditional computer vision techniques; we need a model that detects *semantic objects*, regardless of where they are spatially located in the frame.
75
 
76
  <div align="center">
77
- <img src="REPLACE_THIS_WITH_IMAGE_LINK.png" alt="Mean Images by Room Type" width="800"/>
78
  <br>
79
  <em>Figure 3: Mean Image representation showing high spatial variance.</em>
80
  </div>
@@ -83,13 +90,13 @@ To identify structural patterns, we computed the mathematical "Mean Image" by av
83
 
84
  ## 🧠 3. Semantic Embeddings & Clustering Pipeline
85
 
86
- We extracted a randomized, reproducible subset of 5,000 images (`seed=42`) to optimize computational efficiency while fully capturing the dataset's architectural diversity.
87
 
88
  ### 3.1 Feature Extraction (CLIP)
89
- We deployed Hugging Face's `openai/clip-vit-base-patch32`. Leveraging PyTorch (`torch.no_grad()`) and GPU acceleration, we processed the visual data through the transformer, extracting a highly dense **512-dimensional vector** for each image. These vectors capture complex stylistic signatures.
90
 
91
  ### 3.2 Unsupervised K-Means Clustering
92
- To test the quality of these embeddings, we applied a K-Means clustering algorithm, explicitly instructing it to find 5 clusters (`n_clusters=5`) to mirror our 5 ground-truth room types.
93
 
94
  ### 3.3 Dimensionality Reduction (t-SNE) & Coherence
95
  Projecting 512 dimensions down to a 2D visual scatter plot via t-SNE yielded outstanding results. Rather than a chaotic cloud, the data formed highly cohesive "islands."
@@ -101,33 +108,27 @@ By cross-tabulating the generated clusters against actual room labels, distinct
101
  * **Clusters 2 & 3 (Soft-Furnishings):** A logical blending of Bedrooms and Living Rooms, driven by shared semantic textures (rugs, beds, upholstery).
102
 
103
  <div align="center">
104
- <img src="REPLACE_THIS_WITH_IMAGE_LINK.png" alt="t-SNE Scatter Plot" width="700"/>
105
  <br>
106
  <em>Figure 4: t-SNE 2D Projection showing distinct semantic clustering of 512D embeddings.</em>
107
  </div>
108
 
109
- <div align="center">
110
- <img src="REPLACE_THIS_WITH_IMAGE_LINK.png" alt="Cross-Tabulation Matrix" width="600"/>
111
- <br>
112
- <em>Figure 5: Cross-tabulation proving alignment between unsupervised clusters and ground-truth labels.</em>
113
- </div>
114
-
115
  ---
116
 
117
  ## 🚀 4. Recommendation Engine & Final Application
118
 
119
- The highly coherent 512D matrix was exported as an optimized `.parquet` file to serve as the brain of our production recommendation engine.
120
 
121
  ### System Architecture:
122
- 1. **Input Processing:** When a user interacts with the Gradio UI—either by uploading an inspiration photo or typing a conceptual text prompt—the system passes the input through the CLIP processor, translating it into the exact mathematical vector space as our database.
123
- 2. **Cosine Similarity:** The engine measures the geometric angle between the user's vector and the 5,000 vectors in our catalog.
124
  3. **Top-K Retrieval:** The algorithm ranks the closest spatial matches and retrieves the top 3 results alongside their exact confidence scores.
125
 
126
  <div align="center">
127
- <img src="REPLACE_THIS_WITH_IMAGE_LINK.png" alt="Recommendation Engine Results" width="900"/>
128
  <br>
129
- <em>Figure 6: System successfully matching a textual/visual input to real-world interior designs.</em>
130
  </div>
131
 
132
- ### 🎯 Final Verdict
133
- Testing the complete pipeline confirmed that the system does not simply match raw colors. It successfully extracts and matches abstract architectural concepts and stylistic intent. The backend logic proved highly reliable and was successfully deployed as a fully interactive Hugging Face Space frontend application.
 
11
  license: mit
12
  ---
13
 
14
+ <h1 align="center">**VISIONARY: AI-Powered Interior Design Engine** 🏡</h1>
15
 
16
+ <div align="center">
17
+
18
+ **Matan Zigelman**
19
+ *Reichman University*
20
 
21
+ </div>
 
 
22
 
23
  ---
24
 
25
+ ## 📖 Project Overview
26
+ Traditional visual search engines often rely on basic pixel-matching or color histograms, which struggle to grasp the structural and semantic nuances of architectural spaces. **Visionary** was developed as a comprehensive academic and technical showcase to solve this limitation.
27
+
28
+ By utilizing the `openai/clip-vit-base-patch32` vision-language model, this project projects raw images into a 512-dimensional semantic space. As a result, the system autonomously understands abstract concepts—such as "minimalist," "industrial," or "warm lighting"—and retrieves highly accurate room designs based on user inspiration photos or textual descriptions.
29
+
30
+ **Tech Stack:** `Python` | `PyTorch` | `Hugging Face (Transformers, Datasets)` | `Scikit-Learn (K-Means, t-SNE)` | `Pandas` | `Gradio`
31
 
32
  ---
33
 
34
  ## 🗄️ 1. Dataset Architecture
35
 
36
+ For the foundation of this engine, I selected the `my_interior_design_dataset` from Hugging Face. This dataset is exceptionally well-suited for high-level visual recommendation tasks.
37
 
38
+ | Feature | Details |
39
+ | :--- | :--- |
40
+ | **Total Volume** | 33,453 images (~33,500 in the training split). |
41
+ | **Storage Size** | 535 MB. |
42
+ | **Standardization**| All image matrices are perfectly uniform at a `256x256` pixel resolution. |
43
+ | **Style Labels** | 11 distinct design classes (e.g., *Rustic, Bohemian, Transitional*). |
44
+ | **Room Labels** | 5 functional architectural classes (e.g., *Bedroom, Kitchen, Living Room*). |
45
 
46
  ---
47
 
48
  ## 📊 2. Exploratory Data Analysis (EDA)
49
 
50
+ Before feeding data into deep neural networks, I conducted rigorous Exploratory Data Analysis to validate structural integrity and uncover hidden visual biases.
51
 
52
  ### 2.1 Initial Sanity Checks & Data Integrity
53
+ * **Missing Value Verification:** I inspected the metadata arrays and confirmed **zero missing values (NaNs)**. Every visual sample is explicitly paired with valid ground-truth labels.
54
+ * **Dimension Uniformity:** Batch tensor operations require exact spatial dimensions. The checks passed flawlessly—100% of the tested samples maintain the strict 256x256 boundary, eliminating the need for dynamic resizing during training.
55
 
56
  ### 2.2 Class Distribution Analysis
57
  Understanding the distribution of our labels is critical to identifying potential model biases.
58
+ * **Room Types:** There is a moderate class imbalance. *Class 4 (Living Room)* is the dominant category with >10,000 samples, whereas *Class 3* contains ~4,000.
59
  * **Interior Styles:** The stylistic distribution is more balanced, gently sloping from ~4,000 samples down to ~2,100 in the least frequent class.
60
  * **Takeaway:** While the model has ample data to generate robust embeddings across all vectors, the recommendation engine might naturally perform with higher confidence on Living Rooms due to broader exposure during the embedding phase.
61
 
62
  <div align="center">
63
+ <img src="https://huggingface.co/spaces/matanzig/Interior-Design-GenAI/resolve/main/plot.png" alt="Class Distribution Bar Charts" />
64
  <br>
65
  <em>Figure 1: Distribution of Room Types and Interior Styles across the dataset.</em>
66
  </div>
67
 
68
  ### 2.3 RGB Channel Distribution
69
  Unlike tabular data, images are composed of pixel intensities across Red, Green, and Blue channels.
70
+ * **Insights:** The generated histogram reveals a dataset that is predominantly bright and well-lit. Notably, the **Red channel dominates** the higher intensity ranges. This perfectly aligns with real-world interior photography, which heavily features warm ambient lighting, wooden textures, and earth-toned furniture. The CLIP model utilizes these specific warm-light biases as strong features for calculating room similarities.
71
 
72
  <div align="center">
73
+ <img src="https://huggingface.co/spaces/matanzig/Interior-Design-GenAI/resolve/main/rgb.png" alt="RGB Histogram" />
74
  <br>
75
  <em>Figure 2: RGB Pixel Intensity Distribution highlighting warm-tone dominance.</em>
76
  </div>
77
 
78
  ### 2.4 Spatial Variance & The "Mean Image"
79
+ To identify structural patterns, I computed the mathematical "Mean Image" by averaging pixel arrays within specific classes (e.g., Bedrooms).
80
  * **Insights:** The resulting images were highly amorphous and blurry, lacking distinct structural boundaries like beds or counters. This proves a high degree of spatial variance—rooms are photographed from vastly different angles and depths.
81
+ * **Conclusion:** This strictly validates the decision to use Deep Learning embeddings over traditional computer vision techniques; the system requires a model that detects *semantic objects*, regardless of where they are spatially located in the frame.
82
 
83
  <div align="center">
84
+ <img src="https://huggingface.co/spaces/matanzig/Interior-Design-GenAI/resolve/main/mid.png" alt="Mean Images by Room Type" />
85
  <br>
86
  <em>Figure 3: Mean Image representation showing high spatial variance.</em>
87
  </div>
 
90
 
91
  ## 🧠 3. Semantic Embeddings & Clustering Pipeline
92
 
93
+ I extracted a randomized, reproducible subset of 5,000 images (`seed=42`) to optimize computational efficiency while fully capturing the dataset's architectural diversity.
94
 
95
  ### 3.1 Feature Extraction (CLIP)
96
+ I deployed Hugging Face's `openai/clip-vit-base-patch32`. Leveraging PyTorch (`torch.no_grad()`) and GPU acceleration, the visual data was processed through the transformer, extracting a highly dense **512-dimensional vector** for each image. These vectors capture complex stylistic signatures.
97
 
98
  ### 3.2 Unsupervised K-Means Clustering
99
+ To test the quality of these embeddings, I applied a K-Means clustering algorithm, explicitly instructing it to find 5 clusters (`n_clusters=5`) to mirror the 5 ground-truth room types.
100
 
101
  ### 3.3 Dimensionality Reduction (t-SNE) & Coherence
102
  Projecting 512 dimensions down to a 2D visual scatter plot via t-SNE yielded outstanding results. Rather than a chaotic cloud, the data formed highly cohesive "islands."
 
108
  * **Clusters 2 & 3 (Soft-Furnishings):** A logical blending of Bedrooms and Living Rooms, driven by shared semantic textures (rugs, beds, upholstery).
109
 
110
  <div align="center">
111
+ <img src="https://huggingface.co/spaces/matanzig/Interior-Design-GenAI/resolve/main/tsne.png" alt="t-SNE Scatter Plot" />
112
  <br>
113
  <em>Figure 4: t-SNE 2D Projection showing distinct semantic clustering of 512D embeddings.</em>
114
  </div>
115
 
 
 
 
 
 
 
116
  ---
117
 
118
  ## 🚀 4. Recommendation Engine & Final Application
119
 
120
+ The highly coherent 512D matrix was exported as an optimized `.parquet` file to serve as the core engine of the production application.
121
 
122
  ### System Architecture:
123
+ 1. **Input Processing:** When a user interacts with the Gradio UI—either by uploading an inspiration photo or typing a conceptual text prompt—the system passes the input through the CLIP processor, translating it into the exact mathematical vector space as the database.
124
+ 2. **Cosine Similarity:** The engine measures the geometric angle between the user's vector and the 5,000 vectors in the catalog.
125
  3. **Top-K Retrieval:** The algorithm ranks the closest spatial matches and retrieves the top 3 results alongside their exact confidence scores.
126
 
127
  <div align="center">
128
+ <img src="https://huggingface.co/spaces/matanzig/Interior-Design-GenAI/resolve/main/pic.png" alt="Recommendation Engine Results" />
129
  <br>
130
+ <em>Figure 5: System successfully matching a textual/visual input to real-world interior designs.</em>
131
  </div>
132
 
133
+ ### 🎯 Conclusion & Technical Achievements
134
+ Testing the complete pipeline confirmed that the system goes far beyond matching raw colors. It successfully extracts and matches abstract architectural concepts and stylistic intent. The backend logic proved highly reliable and was successfully deployed as a fully interactive Hugging Face Space frontend application, demonstrating a complete end-to-end Machine Learning lifecycle.