README.md

---
title: Nature Recommender
emoji: 👀
colorFrom: green
colorTo: yellow
sdk: gradio
sdk_version: 6.14.0
python_version: '3.13'
app_file: app.py
pinned: false
license: mit
---


# 🌿 Nature Scene Recommender

A visual similarity search app — upload a nature image and get the 3 most similar scenes from a dataset of 15,000 nature images, using CLIP embeddings and cosine similarity.

---

## 🎬 Presentation Video
<video src="https://huggingface.co/spaces/Danielhalali/nature-recommender/resolve/main/assignment 3 video.mp4" controls="controls" style="max-width: 720px;"></video>

## 📁 Part 1: Dataset

**Dataset:** [mertcobanov/nature-dataset](https://huggingface.co/datasets/mertcobanov/nature-dataset)

**Source:** Hugging Face — a curated collection of high-quality nature and landscape images.

**Size:** 50,000 images with natural scene captions.

**Features:**
- `image` — RGB photo of a natural landscape (512x512px)
- `caption` — a short textual description of the scene (e.g., "a large mountain with a very tall peak")

**Use case:** Given an input landscape image, the app finds the Top 3 most visually similar nature scenes using image embeddings and cosine similarity.

---

## 📊 Part 2: Exploratory Data Analysis (EDA)

The dataset contains 50,000 nature images — mountains, deserts, forests, waterfalls and beaches — each with a text caption. Every image is 512x512px.

**Caption statistics:**
- Total captions: 50,000
- Average caption length: ~10 words
- Min: 4 words | Max: 16 words

**Sanity Checks:** No missing images, no missing captions, all images are RGB and the same size. The dataset is clean and ready.

### Most Common Words in Captions
The 15 most common words reveal the diversity of the dataset — mountain, desert, trees, waterfall, forest...

![Most Common Words](pic1.png)

### Sample Images from Dataset
8 sample images showing the variety of nature scenes in the dataset, each with its caption.

![Sample Images](pic2.png)

---

## 🧠 Part 3: Embeddings

**Model:** `openai/clip-vit-base-patch32` — trained on millions of image-text pairs, enabling it to understand complex visual relationships.

**Sample size:** 15,000 images out of 50,000.

**Embedding size:** 768 dimensions per image — each image is converted to a numerical vector representing its visual content.

**How it works:** Two images of mountains will get very similar vectors. A mountain image and a beach image will get very different vectors.

### 3.1 PCA Visualization

PCA reduces the 768 dimensions to 2 for visualization. The first plot shows the sample index as color, the second shows the K-Means cluster assignment.

![PCA - Sample Index](pic3.png)

### 3.2 K-Means Clustering

Applied K-Means with **8 clusters**. Each cluster represents a different type of nature scene.

![PCA - K-Means Clusters](pic4.png)

### t-SNE Visualization

t-SNE provides a clearer picture of the clusters — the 8 groups are much more distinct here. CLIP successfully grouped visually and semantically similar nature scenes together without any manual labeling.

![t-SNE Clusters](pic5.png)

**Cluster distribution:**
| Cluster | Size | Theme |
|---------|------|-------|
| Cluster 0 | 1,487 | 🏜️ Desert & People |
| Cluster 1 | 3,261 | 🏔️💧 Waterfalls & Mountains |
| Cluster 2 | 900 | 🌲 Dense Forests & Trees |
| Cluster 3 | 1,810 | 🪨 Rocky Landscapes |
| Cluster 4 | 1,016 | 🌊 Coastal & Open Scenes |
| Cluster 5 | 2,042 | ⬛ Mixed B&W & Rocky |
| Cluster 6 | 1,831 | 🌵 Desert Vegetation |
| Cluster 7 | 2,653 | ⛰️ Mountain Forests |

### 3.4 Saved Embeddings

The embeddings were saved to a `nature_embeddings.parquet` file containing the original image index, caption, and embedding vector — so the Space can use them directly without recomputing every time.

---

## ⚙️ Part 4: Inputs & Outputs

### 4.1 Prepare Embeddings
Loaded the embeddings file — extracting the embedding matrix, captions, and original image indices.
- Loaded embeddings shape: **(15,000, 768)**

### 4.2 User Input
A function that converts any uploaded image into a 768-dimensional embedding vector using CLIP — exactly like we did for the dataset images.

### 4.3 Similarity Scores
A function that calculates cosine similarity between the uploaded image's vector and all 15,000 vectors in the dataset — returning a score between 0 (completely different) and 1 (identical).

### 4.4 Top 3 Elements
The main recommendation function — converts the image to an embedding, calculates similarity against all 15,000 images, and returns the 3 most similar ones with their captions and similarity scores.

---

## 🚀 Part 5: HuggingFace Space

The app is built with **Gradio** and hosted on HuggingFace Spaces.

**How to use:**
1. Upload a nature image
2. Click **Find Similar Scenes 🔍**
3. Get the Top 3 most similar scenes with scene type, caption, and similarity score

🔗 [Open the App](https://huggingface.co/spaces/Danielhalali/nature-recommender)

---

## 🎁 Bonus

### Bonus 1: Similarity Score Visualization
Visualizing the top 10 cosine similarity scores for a query image — higher scores indicate greater visual similarity between embeddings.

![Similarity Scores](pic6.png)

### Bonus 2: Auto-Classification of Nature Scenes
Each image is automatically classified into a nature scene category based on caption keywords: Mountain, Water, Forest, Desert, Coastal.

**Distribution:**
- 🏔️ Mountain: 6,796 images (45.3%)
- 💧 Water: 3,147 images (21.0%)
- 🌲 Forest: 1,835 images (12.2%)
- 🌿 Other Nature: 1,341 images (8.9%)
- 🏜️ Desert: 1,323 images (8.8%)
- 🌊 Coastal: 558 images (3.7%)

![Scene Classification](pic7.png)

### Bonus 3: Interactive Cluster Visualization
An interactive t-SNE plot where each point represents an image embedding, colored by cluster. Hover over any point to see the image caption.

![Interactive t-SNE](pic8.png)

---

## 📝 Submission

Submitted on Moodle:
`https://huggingface.co/spaces/Danielhalali/nature-recommender`