ShantyCam/audiodet-ced-mini

ShantyCam AudioDet — CED-mini sound-event detector

A small, CPU-real-time sound-event detector for surveillance / edge use. It combines a frozen CED-mini audio-tagging backbone (Apache-2.0, Xiaomi CED) with a compact ShantyCam MLP classifier head that maps the backbone's 527-dim AudioSet logits to 17 alarm-relevant classes.

Runs on a Raspberry-Pi-class CPU in real time (2 s window, ~17 ms/window on one core) with no Python at inference time via ced.cpp GGUF inference.

Classes (17)

siren, horn, gunshot, glass, knock, loud_voice, dog_bark, baby_cry, chainsaw, drilling, smoke_alarm, speech, cat, explosion, vehicle_crash, scream, music

Per-class decision thresholds ship in each head's head_meta.json.

Architecture

raw waveform 16 kHz, 2 s window
      │
      ▼
CED-mini backbone  (ced-mini-q8.gguf, ced.cpp)   → 527 AudioSet logits/probs
      │
      ▼
StandardScaler + MLP  527 → 512 → 256 → 128 → 17  (ReLU, dropout, sigmoid)
      │
      ▼
per-class probabilities → thresholds → two-consecutive-window decision

• Window / hop: 2 s / 1 s sliding; clip score = max_t min(p[t], p[t+1]) (temporal hysteresis kills isolated single-window spikes).
• Head weight layout (head_weights.bin, little-endian float32): scaler.mean[527], scaler.scale[527], then per layer W[out,in] row-major
  • b[out].

Files

Path	Description
ced-mini-q8.gguf	CED-mini backbone, q8_0 (~10 MB) — production
ced-mini-f16.gguf	CED-mini backbone, f16 (~19 MB) — higher precision
heads/v4_ced/	Head trained on ced.cpp q8 embeddings (hard labels)
heads/v4_distill/	Head distilled from a PaSST teacher into the CED head (best domain F1)
onnx/head.onnx (+ .data)	ONNX head for the CED ONNX frontend (mispeech/ced-mini model.onnx)
recipe/	Full training / calibration / synthesis code
LICENSING.md	Provenance & licensing audit (read this)
NOTICE	Apache-2.0 attribution for the CED backbone

Which head?

v4_distill is the recommended head (knowledge-distilled, best held-out F1 on the ShantyCam domain set). v4_ced is the plain hard-label baseline. Both are the same architecture and are drop-in interchangeable.

Usage (GGUF via ced.cpp)

Build ced.cpp, then stream 2 s windows (16 kHz float32) through the backbone to get 527-dim logits, and apply the head:

ced-batch ced-mini-q8.gguf --threads 8   # stdin: N×[32000] f32 → stdout: N×[527] f32

Load heads/v4_distill/head_weights.bin per the layout above, apply (x - mean) / scale, the 4 linear+ReLU layers, a final sigmoid, and compare each class to its threshold in head_meta.json. See recipe/train_v4_ced.py / recipe/train_v4_distill.py for the exact reference implementation.

The GGUF backbone is also published canonically by LocalAI at mudler/ced-gguf; it is included here so the head + backbone version are pinned together for reproducibility.

Training data & the important caveat

The head was trained on frozen CED-mini embeddings from a mixture of public datasets and a private ShantyCam camera-domain audio set that is not released (GDPR — real people/premises). The released files are model weights only: a non-reversible transformation of pooled 527-dim logits from which no source audio or personal data can be recovered.

No training audio is redistributed in this repo. Public datasets are referenced, not re-hosted; non-commercial datasets (ESC-50, UrbanSound8K, TUT Rare, CryCeleb, FSD50K-NC subset) and any augmentation derived from them are excluded from release. The one clean, fully-synthetic positive set we generated ourselves is published separately as ShantyCam/audiodet-synth-smoke.

Full details: LICENSING.md.

Evaluation (domain held-out, indicative)

Small held-out set (89 positives / 166 negatives), deploy-simulated (2 s/1 s sliding + two-window rule). v4_distill (best grid point α=0.5, T=4): recall ≈ 81 %, false-positive rate ≈ 18 %, F1 ≈ 0.75. These numbers are noisy on a set this small and should be read as indicative, not benchmark-grade. See heads/v4_distill/report.json.

License

• Backbone GGUF: Apache-2.0 (derivative of mispeech/ced-mini; see NOTICE).
• ShantyCam heads, ONNX head, recipe: Apache-2.0.
• GGUF conversion / inference tooling ced.cpp: MIT (referenced).