satnogs-signal-classifier

ResNet-18 fine-tuned to classify SatNOGS waterfall spectrograms as signal vs no-signal (narrowband FSK/GFSK cubesat telemetry). A read-only triage aid — it suggests, it does not auto-vet.

Inputs are cropped to the spectrogram region (colorbar + axes removed), which re-centers the signal.

Held-out test metrics

Held-out test set (436 observations) combining two axes held out of training: an entirely unseen satellite (FrontierSat) plus held-out ground stations whose noise/RFI fingerprint is kept out of training:

Metric	Model	Classical baseline
ROC-AUC	0.827	0.570
PR-AUC	0.829	0.557
precision@10	1.000	0.600

Cross-satellite generalization (held-out FrontierSat, 240 obs, never trained on): ROC-AUC 0.772. By mode: GFSK 0.93, FSK 0.92, FSK AX.100 Mode 5 0.79.

Labels

0 = without-signal, 1 = with-signal; predict P(with-signal) = softmax index 1. Trained on gold human waterfall_status vettings — never the decode-based observation status.

Training data

Dataset: ryroeu/satnogs-signal-waterfalls — 4 train satellites (OTP-2, CUBEBEL-2, AEPEX, CatSat), held-out satellite FrontierSat; ~1,189 gold waterfalls, cropped to the spectrogram.

Limits & caveats

• Sampling bias: gold labels skew toward clearer passes than the unvetted firehose; real-world performance on marginal/faint observations will be lower than these numbers.
• Test-set holdout is two-axis, not intersectional: the unseen-satellite slice is held out by satellite, so some of its passes come from stations that also appear in training — that slice measures cross-satellite generalization, not fully station-unseen performance.
• Narrow family: trained on narrowband FSK/GFSK telemetry; generalization beyond it is unverified.
• Read-only triage aid, not an auto-vetter. precision@10 = 1.0 means the top of the ranked queue is reliable.

Usage

from transformers import pipeline
clf = pipeline("image-classification", model="ryroeu/satnogs-signal-classifier")
clf("waterfall.png")