README.md

metadata

title: SatFetch
emoji: 🛰️
colorFrom: indigo
colorTo: purple
sdk: docker
app_file: app.py
pinned: false

SatFetch resolves the spectral domain gap in Earth observation datasets, enabling unified semantic text-to-image and cross-modal image-to-image queries across Optical, SAR (radar), and Multispectral satellite sensors — with hybrid spatial filtering via Uber H3 hexagonal indexing.

Developed by Team 4MISTAKES (RGIPT) for the ISRO Bharatiya Antariksh Hackathon 2026.

---

Pipeline

Input imagery from any sensor modality is preprocessed per-channel, encoded via pre-trained vision models (DOFA-CLIP / SARCLIP), fused with spatial encoding, and indexed into FAISS HNSW. A query image or text passes through the same encoding pipeline, then retrieves top-K ranked results by cosine similarity with optional H3 geographic filtering.

---

Architecture

Data Layer

Input	Format	Channels
Optical (RGB)	.png / .jpg	3 (R, G, B)
SAR (Radar)	.tif (single-channel)	1 (VV/VH)
Multispectral	.tif (multi-band)	4–13 (Sentinel-2)
Text	Natural language	—

Feature Processing

• Per-modality preprocessing: Normalize, resize 224×224, channel mapping
• Pre-trained encoders: DOFA-CLIP (optical), SARCLIP (SAR), OpenAI CLIP ViT-L/14 (text)
• Hybrid fusion: Concatenate visual + text embeddings with H3 spatial encoding

Retrieval Engine

• FAISS HNSW index for approximate nearest-neighbor search
• Zero-Shot Modality Centering (ZS-MC): Parameter-free vector calibration aligning optical and SAR embeddings in CLIP joint space
• Uber H3 spatial filter: Maps coordinates to resolution-7 hexagons, ring-search limits candidates before FAISS operations

Presentation

• Gradio web UI with map visualization (Leaflet), spectral band rendering, and metrics dashboard
• REST API (FastAPI) for programmatic access

---

Sensor Modalities

SatFetch supports same-modal (optical→optical, SAR→SAR) and cross-modal (optical→SAR, text→multispectral, etc.) retrieval. Cross-modal is the harder problem — the system bridges the spectral domain gap using Zero-Shot Modality Centering (ZS-MC), a training-free calibration technique.

Zero-Shot Modality Centering

ZS-MC computes electromagnetic centroids (μ_mod) of each sensor modality in the CLIP joint embedding space, then calibrates queries by translating the query vector:

z_c = z_q − μ_src + μ_tgt

This aligns representations across spectral domains without backpropagation or training.

---

Evaluation

Benchmark Results

Model	Same R@1	Same R@5	Same R@10	Cross R@1	Cross R@5	Cross R@10	Latency
Baseline CLIP	0.320	0.450	0.520	0.080	0.150	0.220	28ms
Linear CCA	0.330	0.460	0.530	0.120	0.280	0.360	33ms
SatFetch ZS-MC	0.335	0.465	0.540	0.245	0.485	0.590	31ms
SatFetch ZS-MC + Spec. Cal.	0.355	0.510	0.605	0.280	0.535	0.625	32ms

Key improvements over baseline:

• Cross-modal R@5: 0.150 → 0.535 (3.6×)
• Cross-modal R@10: 0.220 → 0.625 (2.8×)
• Latency overhead: +4ms over baseline (negligible)

---

Quick Start

Prerequisites

• Python 3.10+
• Git LFS (for embedding weights, if applicable)

Installation

# Clone & enter
git clone https://github.com/your-org/satfetch.git
cd satfetch

# Virtual environment
python -m venv .venv
# Windows: .venv\Scripts\activate
# Linux/Mac: source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt

Run

python app.py

Open http://localhost:7860 in your browser.

Verify

pytest tests/ -v

---

Deployment (Hugging Face Spaces)

Option A: Git Deploy (Recommended)

1. Create a new Space at huggingface.co/spaces — SDK: Docker, template: Blank
2. Clone the space repo, copy project files, commit, push:

   git clone https://huggingface.co/spaces/YOU/SPACE
   cp -r satfetch/* SPACE/
   cd SPACE
   git add . && git commit -m "Deploy SatFetch"
   git push
   

3. Hugging Face auto-detects the Dockerfile and builds.

Option B: Web Upload

Upload the project directory via the Files and versions tab in the Hugging Face Space UI.

---

Project Structure

├── app.py                  # FastAPI server entry point
├── Dockerfile              # Hugging Face Spaces container
├── requirements.txt        # Python dependencies
├── media/                  # README diagrams (SVG)
├── src/
│   ├── features/           # Embedding extraction & SAR adapters
│   │   ├── extractor.py    # FeatureExtractor wrapper
│   │   ├── satclip_encoder.py
│   │   └── sar_adapter.py
│   ├── retrieval/          # FAISS index & cross-modal search
│   │   ├── cross_modal_retrieval.py
│   │   └── index.py
│   ├── geo/                # H3 spatial indexing
│   │   └── spatial.py
│   ├── evaluation/         # Ground truth & metrics
│   └── ui/                 # Gradio + static web app
│       └── static/         # index.html, style.css, app.js
├── data/
│   ├── gallery/            # Searchable image tiles
│   ├── processed/          # Pre-computed embeddings
│   └── raw/                # Source datasets
├── tests/
└── notebooks/

---

Team 4MISTAKES

Built at Rajiv Gandhi Institute of Petroleum Technology (RGIPT) for the ISRO Bharatiya Antariksh Hackathon 2026.

• Anurag
• Ayush
• Karan

---

_{SatFetch — ISRO Bharatiya Antariksh Hackathon 2026}