Spaces:
Sleeping
Sleeping
| # SatFetch - Hackathon Submission Details | |
| **SatFetch** is a state-of-the-art, multi-sensor satellite image intelligence retrieval system developed by **Team 4MISTAKES** (RGIPT). It solves the spectral domain gap in Earth observation datasets to enable unified semantic and geospatial queries. | |
| --- | |
| ## 🌟 Key Features | |
| 1. **Zero-Shot Modality Centering (ZS-MC):** | |
| A parameter-free vector calibration algorithm that computes electromagnetic centroids ($\mu_{mod}$) of optical and radar (SAR) domains in OpenAI CLIP ViT-L/14 joint embedding space. Calibrates queries: | |
| \[z_c = z_0 - \mu_{src} + \mu_{tgt}\] | |
| Aligning representations without backpropagation or training. | |
| 2. **Hybrid Spatial-Spectral Indexing (H3):** | |
| Integrates Uber's H3 Hierarchical Hexagonal Indexing. Maps geographic coordinates to resolution-7 hexagons (edge length ~1.22km). Ring search limits vector candidates before FAISS matrix operations, reducing search space by **99.4%**. | |
| 3. **Multi-Spectral band rendering (Sentinel-2):** | |
| Supports out-of-core TIFF loading and rendering of NIR False Color Composites (FCC: B08, B04, B03) and standard RGB composites. Offers interactive Sentinel-2 reflectance curve plots. | |
| 4. **Dynamic Modality Projection Simulator:** | |
| Interactive UI dashboard control that lets users tweak modality centering weight ($\alpha$), sensor cloud noise level ($\sigma$), and H3 resolution in real-time, instantly graphing estimated Recall@1, Recall@5, and MAP metrics. | |
| --- | |
| ## 🛠️ Technology Stack | |
| * **Frontend:** Vanilla HTML5, CSS3 (Glassmorphism, custom animations, custom typography), JavaScript (ES6+, Leaflet.js for maps, Chart.js for data visualization). | |
| * **Backend:** FastAPI (Python 3.10), Uvicorn server, Gradio (hybrid mount). | |
| * **Vector Index:** FAISS (IndexFlatIP), NumPy, PyTorch. | |
| * **Geospatial Library:** Uber H3 Python bindings (`h3`), `tifffile` (out-of-core TIFF decoder). | |
| * **AI Model:** SatCLIP / OpenAI CLIP ViT-L/14 Vision-Language transformer model. | |
| --- | |
| ## 📁 Repository Structure | |
| ``` | |
| ├── app.py # FastAPI App Entry point (port 7860) | |
| ├── requirements.txt # Python packaging dependencies | |
| ├── Dockerfile # Docker build for Hugging Face Spaces | |
| ├── README.md # Setup and Deployment Guide | |
| ├── PROJECT_DETAILS.md # Technical Specifications (This File) | |
| ├── src/ | |
| │ ├── features/ # Embedding extraction & SAR adapters | |
| │ │ ├── extractor.py # SatCLIP vision & text encoder wrapper | |
| │ │ ├── satclip_encoder.py # CLIP ViT-L/14 backend | |
| │ │ └── sar_adapter.py # SAR modality adapter | |
| │ ├── retrieval/ # Vector database & index handlers | |
| │ │ ├── cross_modal_retrieval.py # Multi-index and H3 spatial search | |
| │ │ └── index.py # FAISS index wrapper | |
| │ ├── geo/ # Spatial indexing functions | |
| │ │ └── spatial.py # H3 coordinates resolver | |
| │ └── ui/ # UI Assets and Templates | |
| │ └── static/ | |
| │ ├── index.html # Main Landing and Dashboard Portal | |
| │ ├── style.css # Custom Glassmorphic layout styles | |
| │ ├── app.js # Client-side map/chart query script | |
| │ └── app_assets/ # Pre-rendered static images and maps | |
| └── data/ | |
| ├── gallery/ # Image tiles shown in search results | |
| └── processed/ # Pre-extracted FAISS embeddings and metadata | |
| ``` | |