README.md

---
title: SatFetch
emoji: 🛰️
colorFrom: indigo
colorTo: purple
sdk: docker
app_file: app.py
pinned: false
---

<p align="center">
  <img src="media/banner.svg" alt="SatFetch Banner" width="800">
</p>

<p align="center">
  <a href="#"><img src="https://img.shields.io/badge/python-3.10%2B-blue" alt="Python"></a>
  <a href="#"><img src="https://img.shields.io/badge/pytorch-2.0%2B-orange" alt="PyTorch"></a>
  <a href="#"><img src="https://img.shields.io/badge/fastapi-0.100%2B-green" alt="FastAPI"></a>
  <a href="#"><img src="https://img.shields.io/badge/license-MIT-yellow" alt="License"></a>
</p>

SatFetch resolves the **spectral domain gap** in Earth observation datasets, enabling unified semantic text-to-image and cross-modal image-to-image queries across **Optical**, **SAR (radar)**, and **Multispectral** satellite sensors — with hybrid spatial filtering via **Uber H3** hexagonal indexing.

Developed by **Team 4MISTAKES** (RGIPT) for the **ISRO Bharatiya Antariksh Hackathon 2026**.

---

## Pipeline

<p align="center">
  <img src="media/pipeline.svg" alt="System Pipeline" width="900">
</p>

Input imagery from any sensor modality is preprocessed per-channel, encoded via pre-trained vision models (DOFA-CLIP / SARCLIP), fused with spatial encoding, and indexed into FAISS HNSW. A query image or text passes through the same encoding pipeline, then retrieves top-K ranked results by cosine similarity with optional H3 geographic filtering.

---

## Architecture

<p align="center">
  <img src="media/architecture.svg" alt="System Architecture" width="900">
</p>

### Data Layer

| Input | Format | Channels |
|-------|--------|----------|
| **Optical (RGB)** | `.png` / `.jpg` | 3 (R, G, B) |
| **SAR (Radar)** | `.tif` (single-channel) | 1 (VV/VH) |
| **Multispectral** | `.tif` (multi-band) | 4–13 (Sentinel-2) |
| **Text** | Natural language | — |

### Feature Processing

- **Per-modality preprocessing:** Normalize, resize 224×224, channel mapping
- **Pre-trained encoders:** DOFA-CLIP (optical), SARCLIP (SAR), OpenAI CLIP ViT-L/14 (text)
- **Hybrid fusion:** Concatenate visual + text embeddings with H3 spatial encoding

### Retrieval Engine

- **FAISS HNSW index** for approximate nearest-neighbor search
- **Zero-Shot Modality Centering (ZS-MC):** Parameter-free vector calibration aligning optical and SAR embeddings in CLIP joint space
- **Uber H3 spatial filter:** Maps coordinates to resolution-7 hexagons, ring-search limits candidates before FAISS operations

### Presentation

- **Gradio web UI** with map visualization (Leaflet), spectral band rendering, and metrics dashboard
- **REST API** (FastAPI) for programmatic access

---

## Sensor Modalities

<p align="center">
  <img src="media/modalities.svg" alt="Sensor Modalities" width="900">
</p>

SatFetch supports **same-modal** (optical→optical, SAR→SAR) and **cross-modal** (optical→SAR, text→multispectral, etc.) retrieval. Cross-modal is the harder problem — the system bridges the spectral domain gap using **Zero-Shot Modality Centering (ZS-MC)**, a training-free calibration technique.

### Zero-Shot Modality Centering

ZS-MC computes electromagnetic centroids (μ_mod) of each sensor modality in the CLIP joint embedding space, then calibrates queries by translating the query vector:

**z_c = z_q − μ_src + μ_tgt**

This aligns representations across spectral domains without backpropagation or training.

---

## Evaluation

<p align="center">
  <img src="media/metrics.svg" alt="Evaluation Metrics" width="850">
</p>

### Benchmark Results

| Model | Same R@1 | Same R@5 | Same R@10 | Cross R@1 | Cross R@5 | Cross R@10 | Latency |
|-------|----------|----------|-----------|-----------|-----------|------------|---------|
| Baseline CLIP | 0.320 | 0.450 | 0.520 | 0.080 | 0.150 | 0.220 | 28ms |
| Linear CCA | 0.330 | 0.460 | 0.530 | 0.120 | 0.280 | 0.360 | 33ms |
| **SatFetch ZS-MC** | **0.335** | **0.465** | **0.540** | **0.245** | **0.485** | **0.590** | **31ms** |
| SatFetch ZS-MC + Spec. Cal. | **0.355** | **0.510** | **0.605** | **0.280** | **0.535** | **0.625** | **32ms** |

Key improvements over baseline:
- **Cross-modal R@5:** 0.150 → 0.535 (3.6×)
- **Cross-modal R@10:** 0.220 → 0.625 (2.8×)
- **Latency overhead:** +4ms over baseline (negligible)

---

## Quick Start

### Prerequisites

- Python 3.10+
- Git LFS (for embedding weights, if applicable)

### Installation

```bash
# Clone & enter
git clone https://github.com/your-org/satfetch.git
cd satfetch

# Virtual environment
python -m venv .venv
# Windows: .venv\Scripts\activate
# Linux/Mac: source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt
```

### Run

```bash
python app.py
```

Open **http://localhost:7860** in your browser.

### Verify

```bash
pytest tests/ -v
```

---

## Deployment (Hugging Face Spaces)

### Option A: Git Deploy (Recommended)

1. Create a **new Space** at [huggingface.co/spaces](https://huggingface.co/spaces) — SDK: **Docker**, template: **Blank**
2. Clone the space repo, copy project files, commit, push:
   ```bash
   git clone https://huggingface.co/spaces/YOU/SPACE
   cp -r satfetch/* SPACE/
   cd SPACE
   git add . && git commit -m "Deploy SatFetch"
   git push
   ```
3. Hugging Face auto-detects the `Dockerfile` and builds.

### Option B: Web Upload

Upload the project directory via the **Files and versions** tab in the Hugging Face Space UI.

---

## Project Structure

```
├── app.py                  # FastAPI server entry point
├── Dockerfile              # Hugging Face Spaces container
├── requirements.txt        # Python dependencies
├── media/                  # README diagrams (SVG)
├── src/
│   ├── features/           # Embedding extraction & SAR adapters
│   │   ├── extractor.py    # FeatureExtractor wrapper
│   │   ├── satclip_encoder.py
│   │   └── sar_adapter.py
│   ├── retrieval/          # FAISS index & cross-modal search
│   │   ├── cross_modal_retrieval.py
│   │   └── index.py
│   ├── geo/                # H3 spatial indexing
│   │   └── spatial.py
│   ├── evaluation/         # Ground truth & metrics
│   └── ui/                 # Gradio + static web app
│       └── static/         # index.html, style.css, app.js
├── data/
│   ├── gallery/            # Searchable image tiles
│   ├── processed/          # Pre-computed embeddings
│   └── raw/                # Source datasets
├── tests/
└── notebooks/
```

---

## Team 4MISTAKES

Built at **Rajiv Gandhi Institute of Petroleum Technology (RGIPT)** for the **ISRO Bharatiya Antariksh Hackathon 2026**.

- **Anurag**
- **Ayush**
- **Karan**

---

<p align="center">
  <sub>SatFetch — ISRO Bharatiya Antariksh Hackathon 2026</sub>
</p>