joakes90 commited on
Commit
e6ff95c
·
verified ·
1 Parent(s): 990a00c

Clean up repo: proper model card, drop Space-specific gradio_app/ contents

Browse files

Moves the checkpoint and class_names.json to repo root, replaces the stub README with a full model card, and removes gradio_app/ (app.py, requirements.txt, examples) since that belonged to a Space, not a model repo. The Gradio demo has been replaced by a static info Space at joakes90/engine-sound-classifier.

README.md CHANGED
@@ -1,3 +1,107 @@
1
  ---
2
  license: cc-by-sa-4.0
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-sa-4.0
3
+ pipeline_tag: audio-classification
4
+ tags:
5
+ - audio-classification
6
+ - audio
7
+ - pytorch
8
+ - panns
9
+ - audioset
10
+ - engine-sound
11
  ---
12
+
13
+ # Engine Sound Classifier (PANNs CNN14 fine-tune)
14
+
15
+ A fine-tuned [PANNs CNN14](https://github.com/qiuqiangkong/audioset_tagging_cnn)
16
+ (pretrained on AudioSet) that classifies short audio clips of running engines
17
+ into one of 34 configurations — cylinder count, layout, and stroke type (e.g.
18
+ `i4`, `v8_cross`, `single_two_stroke`, `2_rotor`).
19
+
20
+ **Status: early, ambitious, not yet accurate.** This is a first published
21
+ checkpoint, shared to show the approach and current state, warts included.
22
+ Recording-level balanced accuracy is **~0.33** on held-out test data (random
23
+ guessing over 34 classes is ~0.03).
24
+
25
+ An informational demo Space (sample clips, performance breakdown) is at
26
+ [joakes90/engine-sound-classifier](https://huggingface.co/spaces/joakes90/engine-sound-classifier).
27
+ The training code, notebook, and full diagnostic write-up live at
28
+ [github.com/joakes90/auto_sound_train](https://github.com/joakes90/auto_sound_train).
29
+
30
+ ## Files
31
+
32
+ - `best_model_run_d_soft_weights.pth` — fine-tuned state dict (~325 MB), Run D
33
+ checkpoint (best balanced accuracy so far).
34
+ - `class_names.json` — ordered list of the 34 class labels the output layer
35
+ corresponds to.
36
+
37
+ ## Model details
38
+
39
+ - **Architecture:** CNN14 from PANNs, pretrained on AudioSet, with the final
40
+ `fc_audioset` layer replaced by a 34-way linear head and fine-tuned.
41
+ - **Input:** mono audio, resampled to 32 kHz, in 2-second windows (matching
42
+ the training manifest's window length).
43
+ - **Output:** softmax over 34 engine configurations. For clips longer than
44
+ 2s, run overlapping windows (e.g. 1s hop) and average per-window
45
+ probabilities before taking the top class — this "recording-level" pooling
46
+ is consistently more accurate than scoring a single window.
47
+
48
+ ## Classes
49
+
50
+ `2_rotor`, `h12`, `h2`, `h4`, `h6`, `i2_180`, `i2_180_two_stroke`, `i2_270`,
51
+ `i2_360`, `i2_360_two_stroke`, `i3`, `i4`, `i4_crossplane`, `i4_diesel`,
52
+ `i5`, `i5_diesel`, `i6`, `i6_diesel`, `single_four_stroke`,
53
+ `single_two_stroke`, `v10_72`, `v12`, `v2_45`, `v2_90`, `v4`,
54
+ `v4_two_stroke`, `v6_120`, `v6_60`, `v6_90_even`, `v8_cross`, `v8_diesel`,
55
+ `v8_flat`, `v8_voodoo`, `vr6`
56
+
57
+ ## Performance
58
+
59
+ | Run | Change | Balanced acc | Micro acc |
60
+ |---|---|---|---|
61
+ | A | baseline (no regularization) | — | 0.38 (overfit) |
62
+ | B | + SpecAugment/noise/mixup | 0.297 | 0.311 (underfit) |
63
+ | C | fixed SpecAugment time-mask scale | 0.323 | 0.343 |
64
+ | D | + softened class weights (`counts^-0.5`) | **0.330** | **0.376** |
65
+
66
+ This checkpoint is Run D. Cylinder-family accuracy (does it at least get the
67
+ cylinder count right) is meaningfully higher than exact-class accuracy —
68
+ most confusion is between siblings within the same engine family, not wild
69
+ misfires. Known weak spots: engines with many cylinders (8/12) are confused
70
+ with close siblings far more than 1–2 cylinder engines, and a couple of
71
+ classes (`v12`, `2_rotor`) remain poorly calibrated. Full diagnosis in
72
+ `FINDINGS.md` in the training repo.
73
+
74
+ ## Usage
75
+
76
+ ```python
77
+ import json
78
+ import torch
79
+ from huggingface_hub import hf_hub_download
80
+ from panns_inference import AudioTagging
81
+
82
+ REPO_ID = "joakes90/engine_sound_cassifier"
83
+ CHECKPOINT = "best_model_run_d_soft_weights.pth"
84
+
85
+ class_names = json.load(
86
+ open(hf_hub_download(REPO_ID, "class_names.json"))
87
+ )["class_names"]
88
+
89
+ # Loads PANNs' pretrained AudioSet CNN14, then swaps in our fine-tuned head.
90
+ at = AudioTagging(checkpoint_path=None, device="cpu")
91
+ backbone = at.model.module if isinstance(at.model, torch.nn.DataParallel) else at.model
92
+ backbone.fc_audioset = torch.nn.Linear(backbone.fc_audioset.in_features, len(class_names))
93
+
94
+ state_dict = torch.load(hf_hub_download(REPO_ID, CHECKPOINT), map_location="cpu")
95
+ backbone.load_state_dict(state_dict)
96
+ backbone.eval()
97
+
98
+ # waveform: mono float tensor at 32kHz, shape (num_windows, window_samples)
99
+ with torch.inference_mode():
100
+ logits = backbone.fc_audioset(backbone(waveform)["embedding"])
101
+ probs = torch.softmax(logits, dim=1).mean(dim=0)
102
+ top_class = class_names[probs.argmax()]
103
+ ```
104
+
105
+ ## License
106
+
107
+ CC BY-SA 4.0.
gradio_app/best_model_run_d_soft_weights.pth → best_model_run_d_soft_weights.pth RENAMED
File without changes
gradio_app/class_names.json → class_names.json RENAMED
File without changes
gradio_app/README.md DELETED
@@ -1,53 +0,0 @@
1
- ---
2
- title: Engine Sound Classifier
3
- emoji: 🔧
4
- colorFrom: blue
5
- colorTo: gray
6
- sdk: gradio
7
- sdk_version: 6.20.0
8
- app_file: app.py
9
- pinned: false
10
- ---
11
-
12
- # Engine Sound Classifier (early checkpoint)
13
-
14
- Fine-tuned [PANNs CNN14](https://github.com/qiuqiangkong/audioset_tagging_cnn)
15
- (pretrained on AudioSet) that classifies short engine-audio clips into one of
16
- 34 configurations — cylinder count, layout, and stroke type (e.g. `i4`,
17
- `v8_cross`, `single_two_stroke`, `2_rotor`).
18
-
19
- This is a **first published checkpoint**, not a finished product. It's shared
20
- to show the approach and current state, warts included.
21
-
22
- ## Current performance
23
-
24
- - Recording-level balanced accuracy: **~0.33** on held-out test data (random
25
- guessing over 34 classes is ~0.03).
26
- - Cylinder-family accuracy (does it at least get the cylinder count right) is
27
- meaningfully higher than exact-class accuracy — most confusion is between
28
- siblings within the same engine family, not wild misfires.
29
- - Known weak spots: engines with many cylinders (8/12) are confused with
30
- close siblings far more than 1–2 cylinder engines, which have more
31
- distinctive, widely-spaced firing pulses. A couple of classes (`v12`,
32
- `2_rotor`) remain poorly calibrated even after two rounds of fixes.
33
-
34
- The full diagnostic history — what was tried, what broke, and why — lives in
35
- `FINDINGS.md` in the training repo this Space was built from.
36
-
37
- ## How predictions are made
38
-
39
- Audio is resampled to 32 kHz mono, split into overlapping 2-second windows
40
- (matching the training data's window length), and the model's per-window
41
- class probabilities are averaged before picking the top prediction. This
42
- "recording-level" pooling is consistently more accurate than scoring a single
43
- window.
44
-
45
- ## Deploying this Space
46
-
47
- The fine-tuned checkpoint (`best_model_run_d_soft_weights.pth`, ~325 MB) is
48
- not committed here. Either:
49
-
50
- 1. Add it to this Space's repo via `git lfs`, or
51
- 2. Upload it to a separate Hugging Face **model** repo and set the `HF_MODEL_REPO`
52
- (and optionally `HF_MODEL_FILENAME`) Space secret/variable — `app.py` will
53
- fetch it with `huggingface_hub.hf_hub_download` at startup.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
gradio_app/app.py DELETED
@@ -1,152 +0,0 @@
1
- """
2
- Gradio demo for the PANNs CNN14 engine-sound classifier (34 engine
3
- configurations, e.g. i4, v8_cross, single_two_stroke).
4
-
5
- This is an early checkpoint (Run D: SpecAugment-fixed + softened class
6
- weights). Recording-level balanced accuracy is ~0.33 on the held-out test
7
- set -- well above the 1/34 ~= 0.03 random baseline, but far from solved.
8
- See the README on this Space for known failure modes before trusting a
9
- prediction.
10
- """
11
-
12
- import json
13
- import os
14
- from pathlib import Path
15
-
16
- import gradio as gr
17
- import torch
18
- import torchaudio
19
- from panns_inference import AudioTagging
20
-
21
- APP_DIR = Path(__file__).parent
22
- SAMPLE_RATE = 32000
23
- WINDOW_SEC = 2.0 # matches the training manifest's window length
24
- HOP_SEC = 1.0 # 50% overlap keeps short clips from being a single window
25
-
26
- # Local file takes priority (e.g. copied alongside app.py, or git-lfs'd into
27
- # the Space). Falls back to a Hugging Face *model* repo so the Space repo
28
- # itself doesn't have to carry a 300+MB checkpoint in git.
29
- CHECKPOINT_FILENAME = "best_model_run_d_soft_weights.pth"
30
- HF_MODEL_REPO = os.environ.get("HF_MODEL_REPO") # e.g. "username/engine-cnn14"
31
- HF_MODEL_FILENAME = os.environ.get("HF_MODEL_FILENAME", CHECKPOINT_FILENAME)
32
-
33
- device = "cuda" if torch.cuda.is_available() else "cpu"
34
-
35
- with open(APP_DIR / "class_names.json") as f:
36
- CLASS_NAMES = json.load(f)["class_names"]
37
-
38
-
39
- def _resolve_checkpoint_path() -> str:
40
- local_path = APP_DIR / CHECKPOINT_FILENAME
41
- if local_path.exists():
42
- return str(local_path)
43
- if HF_MODEL_REPO:
44
- from huggingface_hub import hf_hub_download
45
-
46
- return hf_hub_download(repo_id=HF_MODEL_REPO, filename=HF_MODEL_FILENAME)
47
- raise FileNotFoundError(
48
- f"No checkpoint at {local_path} and HF_MODEL_REPO is not set. Either "
49
- f"place {CHECKPOINT_FILENAME} next to app.py, or set HF_MODEL_REPO to "
50
- "a Hugging Face model repo that holds it."
51
- )
52
-
53
-
54
- def _load_model() -> torch.nn.Module:
55
- # checkpoint_path=None lets panns_inference fetch/cache its own pretrained
56
- # AudioSet weights -- this mirrors exactly how training started, then we
57
- # overwrite the head and load our fine-tuned state on top.
58
- at = AudioTagging(checkpoint_path=None, device=device)
59
- # AudioTagging only wraps the model in DataParallel on CUDA, so on a
60
- # CPU-only Space (the common free tier) `.module` would not exist.
61
- backbone = at.model.module if isinstance(at.model, torch.nn.DataParallel) else at.model
62
- backbone.fc_audioset = torch.nn.Linear(
63
- in_features=backbone.fc_audioset.in_features, out_features=len(CLASS_NAMES)
64
- )
65
- state_dict = torch.load(_resolve_checkpoint_path(), map_location=device)
66
- backbone.load_state_dict(state_dict)
67
- backbone.to(device)
68
- backbone.eval()
69
- return backbone
70
-
71
-
72
- model = _load_model()
73
-
74
-
75
- def _load_waveform(path: str) -> torch.Tensor:
76
- waveform, sr = torchaudio.load(path)
77
- if waveform.shape[0] > 1:
78
- waveform = waveform.mean(dim=0, keepdim=True)
79
- if sr != SAMPLE_RATE:
80
- waveform = torchaudio.functional.resample(waveform, sr, SAMPLE_RATE)
81
- return waveform.squeeze(0)
82
-
83
-
84
- def _windows(waveform: torch.Tensor) -> torch.Tensor:
85
- win = int(WINDOW_SEC * SAMPLE_RATE)
86
- hop = int(HOP_SEC * SAMPLE_RATE)
87
- if waveform.shape[-1] < win:
88
- waveform = torch.nn.functional.pad(waveform, (0, win - waveform.shape[-1]))
89
- return waveform.unsqueeze(0)
90
- starts = list(range(0, waveform.shape[-1] - win + 1, hop))
91
- if starts[-1] + win < waveform.shape[-1]:
92
- starts.append(waveform.shape[-1] - win)
93
- return torch.stack([waveform[s : s + win] for s in starts])
94
-
95
-
96
- @torch.inference_mode()
97
- def classify(audio_path: str):
98
- if audio_path is None:
99
- return None, "Upload or record a clip first."
100
-
101
- waveform = _load_waveform(audio_path)
102
- duration_sec = waveform.shape[-1] / SAMPLE_RATE
103
- batch = _windows(waveform).to(device)
104
-
105
- logits = model(batch)["embedding"]
106
- logits = model.fc_audioset(logits)
107
- # mean_prob pooling: bounded per window, so one loud/confident window
108
- # can't dominate the recording-level call (see FINDINGS.md).
109
- probs = torch.softmax(logits, dim=1).mean(dim=0)
110
-
111
- top = torch.topk(probs, k=min(5, len(CLASS_NAMES)))
112
- result = {CLASS_NAMES[i]: float(p) for p, i in zip(top.values, top.indices)}
113
- details = (
114
- f"{duration_sec:.1f}s clip -> {batch.shape[0]} overlapping "
115
- f"{WINDOW_SEC:.0f}s window(s), averaged."
116
- )
117
- return result, details
118
-
119
-
120
- EXAMPLES_DIR = APP_DIR / "examples"
121
- examples = [str(p) for p in sorted(EXAMPLES_DIR.glob("*.wav"))]
122
-
123
- DESCRIPTION = """
124
- Fine-tuned PANNs CNN14 (pretrained on AudioSet), classifying short engine
125
- audio clips into one of 34 configurations (cylinder count/layout/stroke,
126
- e.g. `i4`, `v8_cross`, `single_two_stroke`, `2_rotor`).
127
-
128
- **Status: early, ambitious, not accurate.** Recording-level balanced
129
- accuracy is ~0.33 on held-out data (random guessing over 34 classes is
130
- ~0.03). Cylinder-family-level accuracy (does it at least get the cylinder
131
- count right) is meaningfully higher. Known weak spots: engines with many
132
- cylinders (8/12) are confused with siblings far more than 1-2 cylinder
133
- engines; a few classes (`v12`, `2_rotor`) are still poorly calibrated.
134
- Full write-up of the diagnosis and next steps is in the project's
135
- `FINDINGS.md`.
136
- """
137
-
138
- demo = gr.Interface(
139
- fn=classify,
140
- inputs=gr.Audio(type="filepath", label="Engine audio (upload or record)"),
141
- outputs=[
142
- gr.Label(num_top_classes=5, label="Predicted configuration"),
143
- gr.Textbox(label="Details", interactive=False),
144
- ],
145
- examples=examples if examples else None,
146
- cache_examples=False,
147
- title="Engine Sound Classifier (early checkpoint)",
148
- description=DESCRIPTION,
149
- )
150
-
151
- if __name__ == "__main__":
152
- demo.launch()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
gradio_app/examples/i2_360.wav DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:09928bcb73b6b8a35531d651e71b1a14bd0d80a4d7009fbe367075fb2f56909f
3
- size 512078
 
 
 
 
gradio_app/examples/single_two_stroke.wav DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:29ade754afe5e7358efba7deb28323018c19fd1f750a82d34fc35d7861677399
3
- size 512078
 
 
 
 
gradio_app/examples/v2_45.wav DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:909d6080bf8f94b78611e5fd46e9d9740a1c7b493ffe9e09593c068fb2af8015
3
- size 512078
 
 
 
 
gradio_app/requirements.txt DELETED
@@ -1,8 +0,0 @@
1
- gradio
2
- torch
3
- torchaudio
4
- panns_inference
5
- torchlibrosa
6
- matplotlib
7
- numpy
8
- huggingface_hub