README.md

---
title: AsteroidNET
emoji: ☄️
colorFrom: blue
colorTo: blue
sdk: docker
pinned: false
license: mit
short_description: Automated NEO detection — IASC/Pan-STARRS/ZTF pipeline
---

# ☄️ AsteroidNET v0.2

**Automated Near-Earth Object Detection System**  
Dr. Matheus Machado Rech · IASC / Pan-STARRS / ZTF

[![CI](https://github.com/mmrech/asteroidnet/actions/workflows/ci.yml/badge.svg)](https://github.com/mmrech/asteroidnet/actions/workflows/ci.yml)
[![Deploy](https://github.com/mmrech/asteroidnet/actions/workflows/deploy-hf.yml/badge.svg)](https://github.com/mmrech/asteroidnet/actions/workflows/deploy-hf.yml)
[![HuggingFace](https://img.shields.io/badge/HuggingFace-Space-yellow)](https://huggingface.co/spaces/mmrech/asteroidnet)

---

## What it does

AsteroidNET is a 6-stage automated pipeline for detecting moving solar system objects in multi-epoch FITS imaging data, compatible with IASC campaign packages from the **Caça Asteroides MCTI** program.

```
FITS Frames (4×, ~30 min cadence)
  ↓  Stage 1: Ingest + validate (TAI/UTC corrected, byte-swapped)
  ↓  Stage 2: Preprocess (two-pass background, cosmic-ray rejection, alignment)
  ↓  Stage 3: Source extraction (two-pass DAOStarFinder, aperture photometry)
  ↓  Stage 4: Catalog matching (Gaia DR3 stars + SkyBoT known SSOs removed)
  ↓  Stage 5: Tracklet linking (Hough-transform + KD-tree, kinematic validation)
  ↓  Stage 6: Classification (RF → CNN two-stage) + Orbit determination
  ↓  Output: MPC 80-column astrometric records, ready for submission
```

## New in v0.2

| Feature | Details |
|---------|---------|
| **Real FITS support** | Upload IASC campaign packages directly in the UI |
| **TAI/UTC correction** | PS1 `MJD-OBS` is TAI; ZTF is UTC — 37-second offset handled correctly |
| **Byte-order fix** | FITS big-endian data converted to `float32` native before `Background2D` (silent `bottleneck` bug prevented) |
| **Two-pass background** | Source masking for unbiased sky estimation in crowded fields |
| **SkyBoT integration** | IMCCE cone-search removes all known solar system objects from candidates |
| **PS1 header fixes** | Missing `TIMESYS=TAI` and `RADESYS=FK5` added defensively |
| **ZTF data access** | IRSA IBE API for multi-epoch science images |
| **Training data builder** | Mine PS1/ZTF archives with MPC/SkyBoT labels for classifier training |
| **GitHub Actions CI/CD** | Tests run on every PR; auto-deploys to HF Spaces on push to `main` |

## Gradio UI Tabs

1. **Processar Imagens IASC** — Upload real FITS files, run full pipeline, get MPC records
2. **Pipeline Simulator** — Simulate on synthetic data with configurable parameters
3. **MPC Formatter** — Generate exact 80-column MPC astrometric records
4. **Tracklet Visualizer** — Inspect sky motion, ΔRA/ΔDec, and residuals
5. **About** — Documentation and usage guide

## Using with IASC / Caça Asteroides MCTI

1. Register at [iasc.cosmosearch.org](https://iasc.cosmosearch.org)
2. Download a campaign FITS package (4 frames, ~30 min cadence, same sky field)
3. Upload all 4 `.fits` files in the **Processar Imagens IASC** tab
4. Enter your MPC observatory code (`F51` for Pan-STARRS; use `500` if unknown)
5. Click **Run Pipeline** — candidate tracklets are detected and MPC records generated
6. Copy the MPC records and submit to IASC for verification

## Installation (local)

```bash
git clone https://github.com/mmrech/asteroidnet
cd asteroidnet
pip install -r requirements-dev.txt
pip install -e .
pytest tests/ -v
python app.py
```

## Data Sources

| Source | Type | URL |
|--------|------|-----|
| Pan-STARRS DR2 | Single-epoch warp FITS images (0.25″/px) | [ps1images.stsci.edu](https://ps1images.stsci.edu) |
| ZTF DR8 | Science + difference images (1.012″/px) | [irsa.ipac.caltech.edu](https://irsa.ipac.caltech.edu) |
| IMCCE SkyBoT | Known SSO cone-search (1889–2060) | [ssp.imcce.fr/webservices/skybot](https://ssp.imcce.fr/webservices/skybot/) |
| MPC MPCORB | Orbital elements for all known minor planets | [minorplanetcenter.net](https://www.minorplanetcenter.net/iau/MPCORB.html) |
| JPL Horizons | High-precision ephemerides via `astroquery` | [ssd.jpl.nasa.gov](https://ssd.jpl.nasa.gov/horizons/) |

## Critical Implementation Notes

### TAI vs UTC (the most important gotcha)
Pan-STARRS `MJD-OBS` is in **TAI** (International Atomic Time), which is 37 seconds ahead of UTC. ZTF uses **UTC**. A 37-second error corresponds to 0.5–2 arcseconds of apparent asteroid motion — enough to place a predicted position outside the detection aperture.

```python
# Pan-STARRS: MJD-OBS is TAI
t_ps1 = Time(header["MJD-OBS"], format="mjd", scale="tai").utc

# ZTF: OBSMJD is UTC
t_ztf = Time(header["OBSMJD"], format="mjd", scale="utc")
```

### Byte-order and bottleneck
FITS data is stored big-endian. The `bottleneck` library (used by `Background2D` for speed) silently falls back to slower numpy when given non-native-endian arrays — but with **different numerical results** due to different summation order. Fix: always call `.astype(np.float32)` after reading FITS data.

### Two-pass background subtraction
Sources bias the background estimate upward if not masked. Always:
1. Rough background → detect sources → build mask
2. Refined background with masked sources → final subtraction

## Architecture

```
asteroidnet/
├── config/           loader.py, defaults.yaml
├── data_access/      ps1_client.py, ztf_client.py, skybot_client.py
├── fits_ingestor/    ingestor.py
├── image_preprocessor/ preprocessor.py
├── source_extractor/ detector.py
├── catalog_matcher/  matcher.py
├── tracklet_linker/  linker.py
├── candidate_classifier/ classifier.py
├── orbit_determination/  gauss_method.py
├── reporting/        mpc_formatter.py
├── training/         dataset_builder.py
├── pipeline/         runner.py
└── utils/            time_utils.py, synthetic.py
```

## Performance Targets

| Metric | Target | Notes |
|--------|--------|-------|
| Recovery rate (SNR ≥ 5) | ≥ 90% | SC-001 |
| False positive rate | < 1% | SC-002 |
| Star removal completeness | > 99.5% | Gaia DR3 |
| Astrometric residual RMS | < 1 arcsec | Per tracklet |
| Processing time (4 frames) | < 5 min | On CPU |

## License

MIT — see [LICENSE](LICENSE)

## Citation

If you use AsteroidNET in your research, please cite:

```bibtex
@software{rech2026asteroidnet,
  author = {Rech, Matheus Machado},
  title  = {AsteroidNET: Automated Near-Earth Object Detection System},
  year   = {2026},
  url    = {https://github.com/mmrech/asteroidnet}
}
```