Upload README.md

README.md

@@ -1,13 +1,106 @@
----
-title: CLIP Search Edit
-emoji: 📚
-colorFrom: gray
-colorTo: purple
-sdk: gradio
-sdk_version: 5.49.1
-app_file: app.py
-pinned: false
-license: mit
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+---
+title: CLIP Search Edit
+emoji: 📚
+colorFrom: gray
+colorTo: purple
+sdk: gradio
+sdk_version: 5.49.1
+app_file: app.py
+pinned: false
+license: mit
+---
+
+# CLIP Search & Edit Engine
+
+A multimodal application combining semantic image search with lightweight, text-guided image editing. This project utilizes OpenAI's CLIP model for retrieving images from the Flickr30k dataset and a custom FiLM-conditioned U-Net for performing style transfer based on text prompts.
+
+## 🚀 Features
+
+* **Semantic Image Search**: Search through thousands of images in the Flickr30k dataset using natural language queries (e.g., "a dog on a boat", "neon lights").
+* **Text-Guided Image Editing**: Apply artistic styles (Sketch, Van Gogh, Cyberpunk) to images using a lightweight U-Net architecture conditioned on CLIP text embeddings.
+* **Efficiency**: Includes benchmarking tools to measure model FLOPs, parameters, and inference speed.
+* **Streamed Dataset**: Uses Deep Lake to stream dataset images, eliminating the need for massive local downloads.
+
+## 🛠️ Installation
+
+### Prerequisites
+* Python 3.8+
+* CUDA-enabled GPU (recommended for faster indexing and inference)
+* **Git Xet**: Required for cloning large files from Hugging Face.
+
+### Steps
+
+1. **Setup Git Xet and Clone:**
+   To clone the repository from Hugging Face Spaces, ensure `git-xet` is installed to handle large files.
+
+<pre><code># Install git-xet (macOS example)
+brew install git-xet
+git xet install
+
+# Clone the repository
+git clone https://huggingface.co/spaces/CISC473-Group19/CLIP-Search-Edit</code></pre>
+
+   *Alternative: Clone without large files (pointers only):*
+<pre><code>GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/spaces/CISC473-Group19/CLIP-Search-Edit</code></pre>
+
+2. **Install dependencies:**
+<pre><code>cd CLIP-Search-Edit
+pip install -r requirements.txt</code></pre>
+
+
+## ⚙️ Setup & Usage
+
+### 1. Build the Search Index
+Before using the search functionality, you must index the dataset. This script downloads image features from Flickr30k and saves them locally.
+
+<pre><code>python indexer.py</code></pre>
+
+* **Note:** This will create a file named `flickr_embeddings.pt` containing the CLIP embeddings for the images.
+* **Config:** You can adjust `INDEX_LIMIT` in `indexer.py` to change the number of images indexed (default is 5,000).
+
+### 2. Download/Place Model Weights
+The image editing module requires pre-trained U-Net weights. Ensure the following files are in your root directory (or update the paths in `app.py`):
+* `unet_charcoal-sketch.pth`
+* `unet_van-gogh-painting.pth`
+* `unet_neon-cyberpunk.pth`
+
+### 3. Run the Application
+Launch the Gradio web interface:
+
+<pre><code>python app.py</code></pre>
+
+* Open the link provided in the terminal (usually `http://127.0.0.1:7860`).
+* **Tab 1 (Retrieval):** Enter a text query to find semantically related images.
+* **Tab 2 (Image Editing):** Upload an image and select a style to transform it.
+
+## 📊 Benchmarking & Evaluation
+
+* **Measure Efficiency:**
+Run the benchmark script to calculate the U-Net model's parameters, FLOPs, and FPS on CPU/GPU.
+<pre><code>python benchmark_efficiency.py</code></pre>
+
+* **Measure Search Recall:**
+Calculate retrieval metrics (R@1, R@5, R@10) for the indexed dataset.
+<pre><code>python measure_recall.py</code></pre>
+
+## 📂 Project Structure
+
+| File | Description |
+| :--- | :--- |
+| `app.py` | Main entry point. Launches the Gradio UI for Search and Editing. |
+| `indexer.py` | Generates CLIP embeddings for the dataset and saves them to `flickr_embeddings.pt`. |
+| `dataset.py` | Handles streaming images from the Deep Lake Flickr30k dataset. |
+| `cnn1.py` | Defines the lightweight `UNet` architecture with FiLM layers for text conditioning. |
+| `clip_styler.py` | Logic for applying style transfer using the U-Net and CLIP models. |
+| `style_net.py` | Alternative/Legacy ResNet-based style network architecture. |
+| `util.py` | Utility functions for image normalization, loading, and loss calculations. |
+| `requirements.txt` | List of Python dependencies. |
+
+## 🧠 Model Architecture
+
+The editing module uses a custom **U-Net** (`cnn1.py`) enhanced with **FiLM (Feature-wise Linear Modulation)** layers.
+1.  **Encoder:** Extracts image features.
+2.  **FiLM Layers:** Modulate the feature maps based on the CLIP text embedding of the target style (e.g., "charcoal sketch").
+3.  **Decoder:** Reconstructs the stylized image.
+
+This approach allows for fast, text-controllable style transfer without requiring heavy diffusion models.