README.md

---
license: mit
library_name: transformers
pipeline_tag: image-classification
tags:
- satnogs
- radio
- signal-detection
- waterfall
datasets:
- ryroeu/satnogs-signal-waterfalls
---

# satnogs-signal-classifier

ResNet-18 fine-tuned to classify SatNOGS **waterfall** spectrograms as **signal vs no-signal**
(narrowband FSK/GFSK cubesat telemetry). A read-only **triage aid** — it suggests, it does not auto-vet.

Inputs are **cropped to the spectrogram region** (colorbar + axes removed), which re-centers the signal.

## Held-out test metrics

Held-out test set (436 observations) combining two axes held out of training: an **entirely unseen
satellite** (FrontierSat) plus **held-out ground stations** whose noise/RFI fingerprint is kept out
of training:

| Metric | Model | Classical baseline |
|---|---|---|
| ROC-AUC | **0.827** | 0.570 |
| PR-AUC | 0.829 | 0.557 |
| precision@10 | **1.000** | 0.600 |

Cross-satellite generalization (held-out **FrontierSat**, 240 obs, never trained on): ROC-AUC **0.772**.
By mode: GFSK 0.93, FSK 0.92, FSK AX.100 Mode 5 0.79.

## Labels

`0 = without-signal`, `1 = with-signal`; predict P(with-signal) = softmax index 1.
Trained on **gold human `waterfall_status` vettings** — never the decode-based observation `status`.

## Training data

Dataset: [ryroeu/satnogs-signal-waterfalls](https://huggingface.co/datasets/ryroeu/satnogs-signal-waterfalls)
— 4 train satellites (OTP-2, CUBEBEL-2, AEPEX, CatSat), held-out satellite FrontierSat; ~1,189 gold
waterfalls, cropped to the spectrogram.

## Limits & caveats

- **Sampling bias:** gold labels skew toward clearer passes than the unvetted firehose; real-world
  performance on marginal/faint observations will be lower than these numbers.
- **Test-set holdout is two-axis, not intersectional:** the unseen-satellite slice is held out by
  satellite, so some of its passes come from stations that also appear in training — that slice
  measures cross-satellite generalization, not fully station-unseen performance.
- **Narrow family:** trained on narrowband FSK/GFSK telemetry; generalization beyond it is unverified.
- **Read-only triage aid**, not an auto-vetter. precision@10 = 1.0 means the *top of the ranked queue* is reliable.

## Usage

```python
from transformers import pipeline
clf = pipeline("image-classification", model="ryroeu/satnogs-signal-classifier")
clf("waterfall.png")
```