Update README.md

README.md

@@ -11,4 +11,164 @@ license: mit
 short_description: Interactive tool to visualize patch-wise similarity in image
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# DINOv3 Patch Similarity Viewer [Github Repo](https://github.com/devMuniz02/DINOv3-Interactive-Patch-Cosine-Similarity) 
+
+![Gradio Test app](assets/GradioAppTest.gif)
+
+> **Note:** This README and repository are for educational purposes. The creation of this repo was inspired by the DINOv3 paper to help visualize and understand the output of the model.
+
+## Purpose
+
+This repository provides interactive tools to visualize and explore patch-wise similarity in images using the DINOv3 vision transformer model. It is designed for researchers, students, and practitioners interested in understanding how self-supervised vision transformers perceive and relate different regions of an image.
+
+## About DINOv3
+
+- **Paper:** [DINOv3: Self-supervised Vision Transformers with Enormous Teacher Models](https://arxiv.org/abs/2508.10104)
+- **Meta Research Page:** [Meta DINOv3 Publication](https://ai.meta.com/dinov3/)
+- **Official GitHub:** [facebookresearch/dinov3](https://github.com/facebookresearch/dinov3)
+
+**Note:**  
+The DINOv3 model weights require access approval.  
+You can request access via the [Meta Research page](https://ai.meta.com/resources/models-and-libraries/dinov3-downloads/) or by selecting the desired model on [Hugging Face model collection](https://huggingface.co/collections/facebook/dinov3-68924841bd6b561778e31009).
+
+## Features
+
+- **Interactive Visualization:** Click on image patches or use arrow keys to explore patch similarity heatmaps.
+- **Single or Two-Image Mode:** If one image is specified, shows self-similarity. If two images are specified, shows both self-similarity and cross-image similarity overlays interactively.
+- **Image Preprocessing:** Loads and pads images without resizing, preserving the original aspect ratio.
+- **Cosine Similarity Calculation:** Computes and visualizes cosine similarity between image patches.
+- **Robust Fallback:** If an image URL fails to load, a default image is used.
+
+## Installation
+
+Install dependencies with:
+
+```bash
+pip install -r requirements.txt
+```
+
+## Model Selection
+
+You can choose from several DINOv3 models available on Hugging Face (click to view each model card):
+
+LVD-1689M Dataset (Web data)
+- ViT
+    - [facebook/dinov3-vit7b16-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-vit7b16-pretrain-lvd1689m)
+    - [facebook/dinov3-vits16-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-vits16-pretrain-lvd1689m)
+    - [facebook/dinov3-vits16plus-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-vits16plus-pretrain-lvd1689m)
+    - [facebook/dinov3-vitb16-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-vitb16-pretrain-lvd1689m)
+    - [facebook/dinov3-vitl16-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-vitl16-pretrain-lvd1689m)
+    - [facebook/dinov3-vith16plus-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-vith16plus-pretrain-lvd1689m)
+
+- ConvNeXt
+    - [facebook/dinov3-convnext-tiny-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-convnext-tiny-pretrain-lvd1689m)
+    - [facebook/dinov3-convnext-small-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-convnext-small-pretrain-lvd1689m)
+    - [facebook/dinov3-convnext-base-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-convnext-base-pretrain-lvd1689m)
+    - [facebook/dinov3-convnext-large-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-convnext-large-pretrain-lvd1689m)
+
+SAT-493M Dataset (Satellite data)
+- ViT
+    - [facebook/dinov3-vitl16-pretrain-sat493m](https://huggingface.co/facebook/dinov3-vitl16-pretrain-sat493m)
+    - [facebook/dinov3-vit7b16-pretrain-sat493m](https://huggingface.co/facebook/dinov3-vit7b16-pretrain-sat493m)
+
+## Usage
+
+### Gradio app
+
+Run the Gradio app:
+
+```bash
+python app.py
+```
+
+After runnig the app, go to [http://localhost:7860/](http://localhost:7860/) to see the app running.
+
+Then: 
+- Choose Dataset and model name
+- For Single image similarity:
+    - Choose only one file or URL
+- For 2 image similarity:
+    - Choose images from file and/or URL
+- Click button "Initialize / Update "
+- Select the desired patch from the image
+- Watch the results
+
+**Note:** 
+*Overlay alpha* is the intensity of the overlay of patches on top of image
+
+### Python Script
+
+Run the interactive viewer with the default COCO image:
+
+```bash
+python DINOv3CosSimilarity.py
+```
+
+#### Single Image Mode
+
+Specify your own image (local path or URL):
+
+```bash
+python DINOv3CosSimilarity.py --image path/to/your/image.jpg
+python DINOv3CosSimilarity.py --image https://yourdomain.com/image.png
+```
+
+#### Two Image Mode
+
+Specify two images (local paths or URLs):
+
+```bash
+python DINOv3CosSimilarity.py --image1 path/to/image1.jpg --image2 path/to/image2.jpg
+python DINOv3CosSimilarity.py --image1 https://yourdomain.com/image1.png --image2 https://yourdomain.com/image2.png
+```
+
+#### Model Selection
+
+Specify the model with `--model` (default is vits16):
+
+```bash
+python DINOv3CosSimilarity.py --model facebook/dinov3-vitb16-pretrain-lvd1689m
+```
+
+#### Other Options
+
+- `--show_grid` : Draw patch grid
+- `--annotate_indices` : Write patch indices on cells
+- `--overlay_alpha <float>` : Set heatmap alpha (default 0.55)
+- `--patch_size <int>` : Override patch size (default: model's patch size)
+
+#### Controls
+
+- Mouse click to select a patch
+- Arrow keys to move selection
+- '1', '2', or 't' to switch active image (in two-image mode)
+- 'q' to quit
+
+## Demo Single Image
+
+![Interactive Patch Similarity Demo](assets/Test_Interactive_video.gif)
+
+## Demo 2 Images
+
+![Multiple Interactive Patch Similarity Demo](assets/Multiple_Interactive_test_video.gif)
+
+### Jupyter Notebook
+
+1. Open `PatchCosSimilarity.ipynb` in Jupyter Notebook.
+2. Run the cells to load an image and visualize patch similarities.
+3. Set `url1` for single-image mode, or both `url1` and `url2` for two-image mode.
+4. If an image fails to load, a default image will be used automatically.
+5. Set the `model_id` variable to any of the models listed above (see commented lines at the top of the notebook).
+
+**Notebook Controls:**  
+- Mouse click to select a patch  
+- Arrow keys to move selection  
+- '1', '2', or 't' to switch active image (in two-image mode)
+
+## License
+
+This project is licensed under the MIT License. See the `LICENSE` file for details.
+
+## Acknowledgments
+
+This project utilizes the DINOv3 model from Hugging Face's Transformers library, along with PyTorch, Matplotlib, and Pillow