--- title: SatFetch emoji: πŸ›°οΈ colorFrom: indigo colorTo: purple sdk: docker app_file: app.py pinned: false ---

SatFetch Banner

Python PyTorch FastAPI License

SatFetch resolves the **spectral domain gap** in Earth observation datasets, enabling unified semantic text-to-image and cross-modal image-to-image queries across **Optical**, **SAR (radar)**, and **Multispectral** satellite sensors β€” with hybrid spatial filtering via **Uber H3** hexagonal indexing. Developed by **Team 4MISTAKES** (RGIPT) for the **ISRO Bharatiya Antariksh Hackathon 2026**. --- ## Pipeline

System Pipeline

Input imagery from any sensor modality is preprocessed per-channel, encoded via pre-trained vision models (DOFA-CLIP / SARCLIP), fused with spatial encoding, and indexed into FAISS HNSW. A query image or text passes through the same encoding pipeline, then retrieves top-K ranked results by cosine similarity with optional H3 geographic filtering. --- ## Architecture

System Architecture

### Data Layer | Input | Format | Channels | |-------|--------|----------| | **Optical (RGB)** | `.png` / `.jpg` | 3 (R, G, B) | | **SAR (Radar)** | `.tif` (single-channel) | 1 (VV/VH) | | **Multispectral** | `.tif` (multi-band) | 4–13 (Sentinel-2) | | **Text** | Natural language | β€” | ### Feature Processing - **Per-modality preprocessing:** Normalize, resize 224Γ—224, channel mapping - **Pre-trained encoders:** DOFA-CLIP (optical), SARCLIP (SAR), OpenAI CLIP ViT-L/14 (text) - **Hybrid fusion:** Concatenate visual + text embeddings with H3 spatial encoding ### Retrieval Engine - **FAISS HNSW index** for approximate nearest-neighbor search - **Zero-Shot Modality Centering (ZS-MC):** Parameter-free vector calibration aligning optical and SAR embeddings in CLIP joint space - **Uber H3 spatial filter:** Maps coordinates to resolution-7 hexagons, ring-search limits candidates before FAISS operations ### Presentation - **Gradio web UI** with map visualization (Leaflet), spectral band rendering, and metrics dashboard - **REST API** (FastAPI) for programmatic access --- ## Sensor Modalities

Sensor Modalities

SatFetch supports **same-modal** (opticalβ†’optical, SARβ†’SAR) and **cross-modal** (opticalβ†’SAR, textβ†’multispectral, etc.) retrieval. Cross-modal is the harder problem β€” the system bridges the spectral domain gap using **Zero-Shot Modality Centering (ZS-MC)**, a training-free calibration technique. ### Zero-Shot Modality Centering ZS-MC computes electromagnetic centroids (ΞΌ_mod) of each sensor modality in the CLIP joint embedding space, then calibrates queries by translating the query vector: **z_c = z_q βˆ’ ΞΌ_src + ΞΌ_tgt** This aligns representations across spectral domains without backpropagation or training. --- ## Evaluation

Evaluation Metrics

### Benchmark Results | Model | Same R@1 | Same R@5 | Same R@10 | Cross R@1 | Cross R@5 | Cross R@10 | Latency | |-------|----------|----------|-----------|-----------|-----------|------------|---------| | Baseline CLIP | 0.320 | 0.450 | 0.520 | 0.080 | 0.150 | 0.220 | 28ms | | Linear CCA | 0.330 | 0.460 | 0.530 | 0.120 | 0.280 | 0.360 | 33ms | | **SatFetch ZS-MC** | **0.335** | **0.465** | **0.540** | **0.245** | **0.485** | **0.590** | **31ms** | | SatFetch ZS-MC + Spec. Cal. | **0.355** | **0.510** | **0.605** | **0.280** | **0.535** | **0.625** | **32ms** | Key improvements over baseline: - **Cross-modal R@5:** 0.150 β†’ 0.535 (3.6Γ—) - **Cross-modal R@10:** 0.220 β†’ 0.625 (2.8Γ—) - **Latency overhead:** +4ms over baseline (negligible) --- ## Quick Start ### Prerequisites - Python 3.10+ - Git LFS (for embedding weights, if applicable) ### Installation ```bash # Clone & enter git clone https://github.com/your-org/satfetch.git cd satfetch # Virtual environment python -m venv .venv # Windows: .venv\Scripts\activate # Linux/Mac: source .venv/bin/activate # Install dependencies pip install -r requirements.txt ``` ### Run ```bash python app.py ``` Open **http://localhost:7860** in your browser. ### Verify ```bash pytest tests/ -v ``` --- ## Deployment (Hugging Face Spaces) ### Option A: Git Deploy (Recommended) 1. Create a **new Space** at [huggingface.co/spaces](https://huggingface.co/spaces) β€” SDK: **Docker**, template: **Blank** 2. Clone the space repo, copy project files, commit, push: ```bash git clone https://huggingface.co/spaces/YOU/SPACE cp -r satfetch/* SPACE/ cd SPACE git add . && git commit -m "Deploy SatFetch" git push ``` 3. Hugging Face auto-detects the `Dockerfile` and builds. ### Option B: Web Upload Upload the project directory via the **Files and versions** tab in the Hugging Face Space UI. --- ## Project Structure ``` β”œβ”€β”€ app.py # FastAPI server entry point β”œβ”€β”€ Dockerfile # Hugging Face Spaces container β”œβ”€β”€ requirements.txt # Python dependencies β”œβ”€β”€ media/ # README diagrams (SVG) β”œβ”€β”€ src/ β”‚ β”œβ”€β”€ features/ # Embedding extraction & SAR adapters β”‚ β”‚ β”œβ”€β”€ extractor.py # FeatureExtractor wrapper β”‚ β”‚ β”œβ”€β”€ satclip_encoder.py β”‚ β”‚ └── sar_adapter.py β”‚ β”œβ”€β”€ retrieval/ # FAISS index & cross-modal search β”‚ β”‚ β”œβ”€β”€ cross_modal_retrieval.py β”‚ β”‚ └── index.py β”‚ β”œβ”€β”€ geo/ # H3 spatial indexing β”‚ β”‚ └── spatial.py β”‚ β”œβ”€β”€ evaluation/ # Ground truth & metrics β”‚ └── ui/ # Gradio + static web app β”‚ └── static/ # index.html, style.css, app.js β”œβ”€β”€ data/ β”‚ β”œβ”€β”€ gallery/ # Searchable image tiles β”‚ β”œβ”€β”€ processed/ # Pre-computed embeddings β”‚ └── raw/ # Source datasets β”œβ”€β”€ tests/ └── notebooks/ ``` --- ## Team 4MISTAKES Built at **Rajiv Gandhi Institute of Petroleum Technology (RGIPT)** for the **ISRO Bharatiya Antariksh Hackathon 2026**. - **Anurag** - **Ayush** - **Karan** ---

SatFetch β€” ISRO Bharatiya Antariksh Hackathon 2026