| # SatFetch - Hackathon Submission Details |
|
|
| **SatFetch** is a state-of-the-art, multi-sensor satellite image intelligence retrieval system developed by **Team 4MISTAKES** (RGIPT). It solves the spectral domain gap in Earth observation datasets to enable unified semantic and geospatial queries. |
|
|
| --- |
|
|
| ## 🌟 Key Features |
|
|
| 1. **Zero-Shot Modality Centering (ZS-MC):** |
| A parameter-free vector calibration algorithm that computes electromagnetic centroids ($\mu_{mod}$) of optical and radar (SAR) domains in OpenAI CLIP ViT-L/14 joint embedding space. Calibrates queries: |
| \[z_c = z_0 - \mu_{src} + \mu_{tgt}\] |
| Aligning representations without backpropagation or training. |
| |
| 2. **Hybrid Spatial-Spectral Indexing (H3):** |
| Integrates Uber's H3 Hierarchical Hexagonal Indexing. Maps geographic coordinates to resolution-7 hexagons (edge length ~1.22km). Ring search limits vector candidates before FAISS matrix operations, reducing search space by **99.4%**. |
| |
| 3. **Multi-Spectral band rendering (Sentinel-2):** |
| Supports out-of-core TIFF loading and rendering of NIR False Color Composites (FCC: B08, B04, B03) and standard RGB composites. Offers interactive Sentinel-2 reflectance curve plots. |
| |
| 4. **Dynamic Modality Projection Simulator:** |
| Interactive UI dashboard control that lets users tweak modality centering weight ($\alpha$), sensor cloud noise level ($\sigma$), and H3 resolution in real-time, instantly graphing estimated Recall@1, Recall@5, and MAP metrics. |
| |
| --- |
| |
| ## 🛠️ Technology Stack |
| |
| * **Frontend:** Vanilla HTML5, CSS3 (Glassmorphism, custom animations, custom typography), JavaScript (ES6+, Leaflet.js for maps, Chart.js for data visualization). |
| * **Backend:** FastAPI (Python 3.10), Uvicorn server, Gradio (hybrid mount). |
| * **Vector Index:** FAISS (IndexFlatIP), NumPy, PyTorch. |
| * **Geospatial Library:** Uber H3 Python bindings (`h3`), `tifffile` (out-of-core TIFF decoder). |
| * **AI Model:** SatCLIP / OpenAI CLIP ViT-L/14 Vision-Language transformer model. |
| |
| --- |
| |
| ## 📁 Repository Structure |
| |
| ``` |
| ├── app.py # FastAPI App Entry point (port 7860) |
| ├── requirements.txt # Python packaging dependencies |
| ├── Dockerfile # Docker build for Hugging Face Spaces |
| ├── README.md # Setup and Deployment Guide |
| ├── PROJECT_DETAILS.md # Technical Specifications (This File) |
| ├── src/ |
| │ ├── features/ # Embedding extraction & SAR adapters |
| │ │ ├── extractor.py # SatCLIP vision & text encoder wrapper |
| │ │ ├── satclip_encoder.py # CLIP ViT-L/14 backend |
| │ │ └── sar_adapter.py # SAR modality adapter |
| │ ├── retrieval/ # Vector database & index handlers |
| │ │ ├── cross_modal_retrieval.py # Multi-index and H3 spatial search |
| │ │ └── index.py # FAISS index wrapper |
| │ ├── geo/ # Spatial indexing functions |
| │ │ └── spatial.py # H3 coordinates resolver |
| │ └── ui/ # UI Assets and Templates |
| │ └── static/ |
| │ ├── index.html # Main Landing and Dashboard Portal |
| │ ├── style.css # Custom Glassmorphic layout styles |
| │ ├── app.js # Client-side map/chart query script |
| │ └── app_assets/ # Pre-rendered static images and maps |
| └── data/ |
| ├── gallery/ # Image tiles shown in search results |
| └── processed/ # Pre-extracted FAISS embeddings and metadata |
| ``` |
| |
