mmrech commited on
Commit
7ca3199
·
verified ·
1 Parent(s): 860eb1d

fix: valid HF Space card colorTo

Browse files
Files changed (1) hide show
  1. README.md +131 -27
README.md CHANGED
@@ -2,50 +2,154 @@
2
  title: AsteroidNET
3
  emoji: ☄️
4
  colorFrom: blue
5
- colorTo: indigo
6
  sdk: docker
7
  pinned: false
8
  license: mit
9
- short_description: Automated NEO detection pipeline interactive demo
10
  ---
11
 
12
- # ☄️ AsteroidNET
13
 
14
- **Automated Near-Earth Object Detection System** — Interactive Demo
 
15
 
16
- Built by **Dr. Matheus Machado Rech** using Spec-Driven Development (Spec-Kit) + Denario research pipeline.
 
 
17
 
18
- ## What This App Does
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
 
20
- AsteroidNET is a full 6-stage astronomical pipeline for detecting previously unknown asteroids and near-Earth objects (NEOs) from time-series FITS imaging data.
 
 
 
 
 
 
 
 
 
 
21
 
22
- ### Tabs
23
 
24
- | Tab | Description |
25
- |-----|-------------|
26
- | **Pipeline Runner** | Simulate a full pipeline run configure frame count, SNR, velocity range, detection threshold. See live stats, sky motion charts, completeness curves, and a detection table. |
27
- | **MPC Formatter** | Generate a Minor Planet Center 80-column astrometric observation record. |
28
- | **Tracklet Visualizer** | Inspect multi-frame tracklet motion: sky plane, ΔRA/ΔDec vs time, linear residuals. |
29
- | **Orbit Inspector** | Visualise Keplerian orbital elements (Gauss method) with uncertainty bars. |
30
- | **About** | Full pipeline docs, performance targets, and tech stack. |
31
 
32
- ## Pipeline Architecture
33
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
34
  ```
35
- FITS Files → [Ingestion] → [Preprocessing] → [Source Extraction]
36
- [Star Removal] → [Tracklet Linking] → [ML Classification]
37
- → [Orbit Determination] → MPC 80-column Format
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
38
  ```
39
 
40
  ## Performance Targets
41
 
42
- - Detection completeness 90% at SNR ≥ 5
43
- - False positive rate < 1%
44
- - 500+ FITS frames processed in < 4 hours
45
- - Star removal completeness > 99.5% (Gaia DR3)
46
- - MPC format output: exactly 80 columns
 
 
 
 
47
 
48
- ## Technology Stack
49
 
50
- `astropy` · `photutils` · `astroquery` · `reproject` · `astroscrappy` · `sbpy`
51
- `scikit-learn` · `PyTorch` · `SQLAlchemy` · `Celery` · `FastAPI` · `PostgreSQL+PostGIS`
 
 
 
 
 
 
 
 
 
 
 
2
  title: AsteroidNET
3
  emoji: ☄️
4
  colorFrom: blue
5
+ colorTo: blue
6
  sdk: docker
7
  pinned: false
8
  license: mit
9
+ short_description: Automated NEO detection — IASC/Pan-STARRS/ZTF pipeline
10
  ---
11
 
12
+ # ☄️ AsteroidNET v0.2
13
 
14
+ **Automated Near-Earth Object Detection System**
15
+ Dr. Matheus Machado Rech · IASC / Pan-STARRS / ZTF
16
 
17
+ [![CI](https://github.com/mmrech/asteroidnet/actions/workflows/ci.yml/badge.svg)](https://github.com/mmrech/asteroidnet/actions/workflows/ci.yml)
18
+ [![Deploy](https://github.com/mmrech/asteroidnet/actions/workflows/deploy-hf.yml/badge.svg)](https://github.com/mmrech/asteroidnet/actions/workflows/deploy-hf.yml)
19
+ [![HuggingFace](https://img.shields.io/badge/HuggingFace-Space-yellow)](https://huggingface.co/spaces/mmrech/asteroidnet)
20
 
21
+ ---
22
+
23
+ ## What it does
24
+
25
+ AsteroidNET is a 6-stage automated pipeline for detecting moving solar system objects in multi-epoch FITS imaging data, compatible with IASC campaign packages from the **Caça Asteroides MCTI** program.
26
+
27
+ ```
28
+ FITS Frames (4×, ~30 min cadence)
29
+ ↓ Stage 1: Ingest + validate (TAI/UTC corrected, byte-swapped)
30
+ ↓ Stage 2: Preprocess (two-pass background, cosmic-ray rejection, alignment)
31
+ ↓ Stage 3: Source extraction (two-pass DAOStarFinder, aperture photometry)
32
+ ↓ Stage 4: Catalog matching (Gaia DR3 stars + SkyBoT known SSOs removed)
33
+ ↓ Stage 5: Tracklet linking (Hough-transform + KD-tree, kinematic validation)
34
+ ↓ Stage 6: Classification (RF → CNN two-stage) + Orbit determination
35
+ ↓ Output: MPC 80-column astrometric records, ready for submission
36
+ ```
37
+
38
+ ## New in v0.2
39
 
40
+ | Feature | Details |
41
+ |---------|---------|
42
+ | **Real FITS support** | Upload IASC campaign packages directly in the UI |
43
+ | **TAI/UTC correction** | PS1 `MJD-OBS` is TAI; ZTF is UTC — 37-second offset handled correctly |
44
+ | **Byte-order fix** | FITS big-endian data converted to `float32` native before `Background2D` (silent `bottleneck` bug prevented) |
45
+ | **Two-pass background** | Source masking for unbiased sky estimation in crowded fields |
46
+ | **SkyBoT integration** | IMCCE cone-search removes all known solar system objects from candidates |
47
+ | **PS1 header fixes** | Missing `TIMESYS=TAI` and `RADESYS=FK5` added defensively |
48
+ | **ZTF data access** | IRSA IBE API for multi-epoch science images |
49
+ | **Training data builder** | Mine PS1/ZTF archives with MPC/SkyBoT labels for classifier training |
50
+ | **GitHub Actions CI/CD** | Tests run on every PR; auto-deploys to HF Spaces on push to `main` |
51
 
52
+ ## Gradio UI Tabs
53
 
54
+ 1. **Processar Imagens IASC** — Upload real FITS files, run full pipeline, get MPC records
55
+ 2. **Pipeline Simulator** — Simulate on synthetic data with configurable parameters
56
+ 3. **MPC Formatter** — Generate exact 80-column MPC astrometric records
57
+ 4. **Tracklet Visualizer** Inspect sky motion, ΔRA/ΔDec, and residuals
58
+ 5. **About** Documentation and usage guide
 
 
59
 
60
+ ## Using with IASC / Caça Asteroides MCTI
61
 
62
+ 1. Register at [iasc.cosmosearch.org](https://iasc.cosmosearch.org)
63
+ 2. Download a campaign FITS package (4 frames, ~30 min cadence, same sky field)
64
+ 3. Upload all 4 `.fits` files in the **Processar Imagens IASC** tab
65
+ 4. Enter your MPC observatory code (`F51` for Pan-STARRS; use `500` if unknown)
66
+ 5. Click **Run Pipeline** — candidate tracklets are detected and MPC records generated
67
+ 6. Copy the MPC records and submit to IASC for verification
68
+
69
+ ## Installation (local)
70
+
71
+ ```bash
72
+ git clone https://github.com/mmrech/asteroidnet
73
+ cd asteroidnet
74
+ pip install -r requirements-dev.txt
75
+ pip install -e .
76
+ pytest tests/ -v
77
+ python app.py
78
  ```
79
+
80
+ ## Data Sources
81
+
82
+ | Source | Type | URL |
83
+ |--------|------|-----|
84
+ | Pan-STARRS DR2 | Single-epoch warp FITS images (0.25″/px) | [ps1images.stsci.edu](https://ps1images.stsci.edu) |
85
+ | ZTF DR8 | Science + difference images (1.012″/px) | [irsa.ipac.caltech.edu](https://irsa.ipac.caltech.edu) |
86
+ | IMCCE SkyBoT | Known SSO cone-search (1889–2060) | [ssp.imcce.fr/webservices/skybot](https://ssp.imcce.fr/webservices/skybot/) |
87
+ | MPC MPCORB | Orbital elements for all known minor planets | [minorplanetcenter.net](https://www.minorplanetcenter.net/iau/MPCORB.html) |
88
+ | JPL Horizons | High-precision ephemerides via `astroquery` | [ssd.jpl.nasa.gov](https://ssd.jpl.nasa.gov/horizons/) |
89
+
90
+ ## Critical Implementation Notes
91
+
92
+ ### TAI vs UTC (the most important gotcha)
93
+ Pan-STARRS `MJD-OBS` is in **TAI** (International Atomic Time), which is 37 seconds ahead of UTC. ZTF uses **UTC**. A 37-second error corresponds to 0.5–2 arcseconds of apparent asteroid motion — enough to place a predicted position outside the detection aperture.
94
+
95
+ ```python
96
+ # Pan-STARRS: MJD-OBS is TAI
97
+ t_ps1 = Time(header["MJD-OBS"], format="mjd", scale="tai").utc
98
+
99
+ # ZTF: OBSMJD is UTC
100
+ t_ztf = Time(header["OBSMJD"], format="mjd", scale="utc")
101
+ ```
102
+
103
+ ### Byte-order and bottleneck
104
+ FITS data is stored big-endian. The `bottleneck` library (used by `Background2D` for speed) silently falls back to slower numpy when given non-native-endian arrays — but with **different numerical results** due to different summation order. Fix: always call `.astype(np.float32)` after reading FITS data.
105
+
106
+ ### Two-pass background subtraction
107
+ Sources bias the background estimate upward if not masked. Always:
108
+ 1. Rough background → detect sources → build mask
109
+ 2. Refined background with masked sources → final subtraction
110
+
111
+ ## Architecture
112
+
113
+ ```
114
+ asteroidnet/
115
+ ├── config/ loader.py, defaults.yaml
116
+ ├── data_access/ ps1_client.py, ztf_client.py, skybot_client.py
117
+ ├── fits_ingestor/ ingestor.py
118
+ ├── image_preprocessor/ preprocessor.py
119
+ ├── source_extractor/ detector.py
120
+ ├── catalog_matcher/ matcher.py
121
+ ├── tracklet_linker/ linker.py
122
+ ├── candidate_classifier/ classifier.py
123
+ ├── orbit_determination/ gauss_method.py
124
+ ├── reporting/ mpc_formatter.py
125
+ ├── training/ dataset_builder.py
126
+ ├── pipeline/ runner.py
127
+ └── utils/ time_utils.py, synthetic.py
128
  ```
129
 
130
  ## Performance Targets
131
 
132
+ | Metric | Target | Notes |
133
+ |--------|--------|-------|
134
+ | Recovery rate (SNR 5) | 90% | SC-001 |
135
+ | False positive rate | < 1% | SC-002 |
136
+ | Star removal completeness | > 99.5% | Gaia DR3 |
137
+ | Astrometric residual RMS | < 1 arcsec | Per tracklet |
138
+ | Processing time (4 frames) | < 5 min | On CPU |
139
+
140
+ ## License
141
 
142
+ MIT see [LICENSE](LICENSE)
143
 
144
+ ## Citation
145
+
146
+ If you use AsteroidNET in your research, please cite:
147
+
148
+ ```bibtex
149
+ @software{rech2026asteroidnet,
150
+ author = {Rech, Matheus Machado},
151
+ title = {AsteroidNET: Automated Near-Earth Object Detection System},
152
+ year = {2026},
153
+ url = {https://github.com/mmrech/asteroidnet}
154
+ }
155
+ ```