README.md

---
title: Does It Sound Broken
emoji: 🔊
colorFrom: yellow
colorTo: red
sdk: gradio
sdk_version: "5.50.0"
python_version: "3.10"
app_file: app.py
short_description: Diagnose appliance faults from sound.
pinned: false
tags:
  - hackathon
  - build-small
  - backyard-ai
  - nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16
  - tiny-titan
  - off-brand
  - well-tuned
  - audio
  - appliance-diagnosis
  - edge-inference
---

# 🔊 Does It Sound Broken?

> Record 10 seconds of an appliance. Get back **what's wrong, how urgent, and
> what to check first** — grounded in measured acoustics, explained by a 4B model.

## Elevator pitch
Mechanical faults announce themselves acoustically long before failure, but
ordinary people can't decode the sound and a technician callout costs more than
the diagnosis is worth. This app closes that gap: deterministic DSP extracts ~14
acoustic features, a transparent rule engine ranks likely faults, and
**NVIDIA Nemotron-3-Nano-4B** explains the best-supported one in plain language.
**The model never hears raw audio** — it reasons only over confirmed measurements,
which is exactly why a 4B model is reliable here.

## ✨ Features
- 🎙️ Record or upload — diagnose 12 appliance types
- 🔬 14 deterministic acoustic features (librosa/scipy, on Modal)
- 🧭 Transparent rule-grounded fault ranking (no hallucinated faults)
- 🚦 Urgency banner (CRITICAL → LOW) + 3 concrete checks + safety note
- 🧾 "Evidence" panel: every diagnosis traces to a measured value and a rule
- 📊 Before/after compare tab — prove the fix measurably worked
- 🛡️ Hardened: silence, clipping, NaN, corrupt files, stereo, and runaway
  model output are all handled — the UI never crashes (106 passing tests)
- 🔒 No audio stored

## 🧠 How it works
```
HF Space (thin Gradio client)
   │  audio bytes + appliance
   ▼
Modal GPU container ── librosa features ─► trained anomaly detector (DCASE 2025)
                    ─► rule engine ranks candidates
                    ─► structured prompt ─► Nemotron-3-Nano-4B + LoRA ─► validated JSON
   │  result dict
   ▼
HF Space renders verdict + evidence
```
**Limited resources by design:** the Space does NO heavy compute — it only ships
audio to Modal and renders the answer. All librosa/torch/transformers work runs in
the Modal image on Modal's hardware. The rule engine is the floor: the model can
refine and explain, but cannot name a fault the measurements don't support
(`json_guard.py` snaps ungrounded output back).

## 🏗️ Architecture
| File | Role | Runs on |
|---|---|---|
| `app.py` | Thin Gradio client (only `gradio` + `modal`) | HF Space |
| `modal_backend.py` | GPU class running the whole pipeline | Modal (A100-40GB) |
| `audio_analyzer.py` | Deterministic 14-feature extraction | Modal |
| `fault_rules.py` | Per-appliance rule tables → ranked candidates | Modal |
| `feature_prompt.py` | Formats facts + candidates into the prompt | Modal |
| `json_guard.py` | Validates output, guarantees a safe result | Modal |
| `finetune/train.py` | DCASE 2025 classifier + LoRA training pipeline | Modal (CPU/GPU) |
| `mock_model.py` | Deterministic LLM stand-in for tests/dry-runs | local |

Model: `nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16` (override with `MODEL_ID`).

## 🚀 Run locally
```bash
pip install -r requirements.txt        # thin client deps (gradio, modal)
python assets/generate_samples.py      # create test WAVs

# UI only, no Modal/GPU (canned output):
SOUNDBROKEN_MOCK=1 python app.py

# Real backend: deploy Modal once, then run the client:
modal deploy modal_backend.py
python app.py
```

## ☁️ Deploy
1. **Modal backend:** `modal deploy modal_backend.py` (the image installs
   librosa/torch/transformers; weights cache on a Modal Volume).
2. **HF Space:** SDK **Gradio**, free CPU tier is enough (thin client). Add your
   Modal credentials as Space secrets: `MODAL_TOKEN_ID`, `MODAL_TOKEN_SECRET`.
3. First request cold-starts the Modal container (~20–30 s); then it's warm.

## 🧪 Sample audio
- `sample_washer_bearing.wav` — rhythmic 4 Hz impacts (bearing fault)
- `sample_fan_imbalanced.wav` — 50 Hz hum with amplitude wobble (imbalance)
- `sample_motor_squeal.wav` — 2.5 kHz modulated tone (belt/brush squeal)
- `sample_washer_good.wav` — steady hum (healthy baseline)

## 🧪 Trained ML models

### Anomaly detector (binary: normal vs anomaly)
- **Dataset:** DCASE 2025 Task 2 (`HTill/dcase2025_task2_dev`) — real industrial machine audio
- **Accuracy:** 71.2% | **ROC-AUC:** 0.725 | **5-fold CV AUC:** 0.758 ± 0.025
- **Features:** 14 deterministic librosa features (RMS, spectral centroid, onset rate, harmonic ratio, etc.)
- **Ensemble:** VotingClassifier (GradientBoosting + RandomForest, soft voting)
- **Threshold:** Youden-J optimal 0.45 (balanced) / 0.22 (safety-mode, ≥80% recall)
- **Per-machine accuracy:** bearing 66%, fan 72%, gearbox 70%, slider 66%, valve 82%

### LoRA fine-tune (Well-Tuned badge)
- **Base:** `nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16` with LoRA (r=16, alpha=32)
- **Targets:** q_proj, v_proj, k_proj, o_proj (1.59M trainable params, 0.04%)
- **Training:** 300 pairs × 2 epochs on A100-80GB, loss 1.85 → 0.72
- **Pairs:** DCASE clips → (features + rule candidates → grounded JSON response)
- **Adapter:** `mitvho09/sound-broken-nemotron-lora` on HF Hub
- **Inference:** Manual weight merging (no peft at runtime) — LoRA fused into base model weights at container start for transformers 5.x compatibility

## 🔭 Future work
- More domains: HVAC, power tools, vehicles
- On-device GGUF (Q4_K_M) build for offline field use
- Trend analysis with before/after comparison

## 🏆 Hackathon tracks
Judges' Wildcard · Backyard AI · NVIDIA Nemotron Quest · Tiny Titan · Off-Brand · Well-Tuned

## 📄 License
Apache-2.0

## 👥 Contributors
- You