README.md

metadata

title: Does It Sound Broken
emoji: 🔊
colorFrom: yellow
colorTo: red
sdk: gradio
sdk_version: 5.50.0
python_version: '3.10'
app_file: app.py
short_description: Diagnose appliance faults from sound.
pinned: false
tags:
  - hackathon
  - build-small
  - backyard-ai
  - nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16
  - tiny-titan
  - off-brand
  - well-tuned
  - audio
  - appliance-diagnosis
  - edge-inference

🔊 Does It Sound Broken?

Record 10 seconds of an appliance. Get back what's wrong, how urgent, and what to check first — grounded in measured acoustics, explained by a 4B model.

Elevator pitch

Mechanical faults announce themselves acoustically long before failure, but ordinary people can't decode the sound and a technician callout costs more than the diagnosis is worth. This app closes that gap: deterministic DSP extracts ~14 acoustic features, a transparent rule engine ranks likely faults, and NVIDIA Nemotron-3-Nano-4B explains the best-supported one in plain language. The model never hears raw audio — it reasons only over confirmed measurements, which is exactly why a 4B model is reliable here.

✨ Features

• 🎙️ Record or upload — diagnose 12 appliance types
• 🔬 14 deterministic acoustic features (librosa/scipy, on Modal)
• 🧭 Transparent rule-grounded fault ranking (no hallucinated faults)
• 🚦 Urgency banner (CRITICAL → LOW) + 3 concrete checks + safety note
• 🧾 "Evidence" panel: every diagnosis traces to a measured value and a rule
• 📊 Before/after compare tab — prove the fix measurably worked
• 🛡️ Hardened: silence, clipping, NaN, corrupt files, stereo, and runaway model output are all handled — the UI never crashes (106 passing tests)
• 🔒 No audio stored

🧠 How it works

HF Space (thin Gradio client)
   │  audio bytes + appliance
   ▼
Modal GPU container ── librosa features ─► trained anomaly detector (DCASE 2025)
                    ─► rule engine ranks candidates
                    ─► structured prompt ─► Nemotron-3-Nano-4B + LoRA ─► validated JSON
   │  result dict
   ▼
HF Space renders verdict + evidence

Limited resources by design: the Space does NO heavy compute — it only ships audio to Modal and renders the answer. All librosa/torch/transformers work runs in the Modal image on Modal's hardware. The rule engine is the floor: the model can refine and explain, but cannot name a fault the measurements don't support (json_guard.py snaps ungrounded output back).

🏗️ Architecture

File	Role	Runs on
app.py	Thin Gradio client (only gradio + modal)	HF Space
modal_backend.py	GPU class running the whole pipeline	Modal (A100-40GB)
audio_analyzer.py	Deterministic 14-feature extraction	Modal
fault_rules.py	Per-appliance rule tables → ranked candidates	Modal
feature_prompt.py	Formats facts + candidates into the prompt	Modal
json_guard.py	Validates output, guarantees a safe result	Modal
finetune/train.py	DCASE 2025 classifier + LoRA training pipeline	Modal (CPU/GPU)
mock_model.py	Deterministic LLM stand-in for tests/dry-runs	local

Model: nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 (override with MODEL_ID).

🚀 Run locally

pip install -r requirements.txt        # thin client deps (gradio, modal)
python assets/generate_samples.py      # create test WAVs

# UI only, no Modal/GPU (canned output):
SOUNDBROKEN_MOCK=1 python app.py

# Real backend: deploy Modal once, then run the client:
modal deploy modal_backend.py
python app.py

☁️ Deploy

1. Modal backend: modal deploy modal_backend.py (the image installs librosa/torch/transformers; weights cache on a Modal Volume).
2. HF Space: SDK Gradio, free CPU tier is enough (thin client). Add your Modal credentials as Space secrets: MODAL_TOKEN_ID, MODAL_TOKEN_SECRET.
3. First request cold-starts the Modal container (~20–30 s); then it's warm.

🧪 Sample audio

• sample_washer_bearing.wav — rhythmic 4 Hz impacts (bearing fault)
• sample_fan_imbalanced.wav — 50 Hz hum with amplitude wobble (imbalance)
• sample_motor_squeal.wav — 2.5 kHz modulated tone (belt/brush squeal)
• sample_washer_good.wav — steady hum (healthy baseline)

🧪 Trained ML models

Anomaly detector (binary: normal vs anomaly)

• Dataset: DCASE 2025 Task 2 (HTill/dcase2025_task2_dev) — real industrial machine audio
• Accuracy: 71.2% | ROC-AUC: 0.725 | 5-fold CV AUC: 0.758 ± 0.025
• Features: 14 deterministic librosa features (RMS, spectral centroid, onset rate, harmonic ratio, etc.)
• Ensemble: VotingClassifier (GradientBoosting + RandomForest, soft voting)
• Threshold: Youden-J optimal 0.45 (balanced) / 0.22 (safety-mode, ≥80% recall)
• Per-machine accuracy: bearing 66%, fan 72%, gearbox 70%, slider 66%, valve 82%

LoRA fine-tune (Well-Tuned badge)

• Base: nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 with LoRA (r=16, alpha=32)
• Targets: q_proj, v_proj, k_proj, o_proj (1.59M trainable params, 0.04%)
• Training: 300 pairs × 2 epochs on A100-80GB, loss 1.85 → 0.72
• Pairs: DCASE clips → (features + rule candidates → grounded JSON response)
• Adapter: mitvho09/sound-broken-nemotron-lora on HF Hub
• Inference: Manual weight merging (no peft at runtime) — LoRA fused into base model weights at container start for transformers 5.x compatibility

🔭 Future work

• More domains: HVAC, power tools, vehicles
• On-device GGUF (Q4_K_M) build for offline field use
• Trend analysis with before/after comparison

🏆 Hackathon tracks

Judges' Wildcard · Backyard AI · NVIDIA Nemotron Quest · Tiny Titan · Off-Brand · Well-Tuned

📄 License

Apache-2.0

👥 Contributors

• You