Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,74 @@
+---
+license: bsd-3-clause
+tags:
+- executorch
+- xnnpack
+- pte
+- on-device
+- audio-classification
+base_model:
+- MIT/ast-finetuned-audioset-10-10-0.4593
+---
+# AST audio event classification — ExecuTorch
+
+Ten seconds of sound in, 527 AudioSet labels out: speech, music, a dog, a door, an engine.
+The shelf had speech recognition and speaker identity; this is the one that says what a
+sound *is*.
+
+- **Source**: MIT/ast-finetuned-audioset-10-10-0.4593 — 86M parameters, Audio Spectrogram Transformer
+- **License**: bsd-3-clause
+- **Input**: `input_values` `[1, 1024, 128]` fp32 — log-mel filterbank [1, 1024, 128] fp32 — `torchaudio.compliance.kaldi.fbank` at 16000 Hz with 128 mel bins, padded or trimmed to 1024 frames (10.24 s), then normalised with mean=-4.2677393 std=4.5689974. `ASTFeatureExtractor` does exactly this
+- **Output**: logits [1, 527] — AudioSet labels, multi-label: apply sigmoid, not softmax
+
+## Variants
+
+| build | file | size (MB) | Mac median (ms)* | top-1 vs eager | worst probability shift |
+|---|---|---|---|---|---|
+| fp32 | `audiocls_audioset_xnnpack_fp32.pte` | 346.6 | 285.0 | 8 of 8 | 0.0000 |
+| fp16 | `audiocls_audioset_xnnpack_fp16.pte` | 173.9 | 545.1 | 8 of 8 | 0.0014 |
+| int8 (dynamic) | `audiocls_audioset_xnnpack_int8.pte` | 90.9 | 267.3 | 8 of 8 | 0.0160 |
+| Core ML (fp16, iOS) | `audiocls_audioset_coreml_all.pte` | 173.7 | 74.6 | 8 of 8 | 0.0033 |
+
+\*Mac arm64, single process, median of 10, one 10.24 s clip. PyTorch eager fp32 on the same
+machine is **122.7 ms**. Core ML at 74.6 ms is 1.6x that; int8 at 267.3 ms
+is the fastest portable build and a quarter of the fp32 file. fp16 is **slower than fp32**
+here (545.1 ms) — XNNPACK emulates it — and only earns its place by halving the file.
+
+## What the classifier actually says
+
+The test clips are speech, and every build puts `Speech` in the top five on all
+8 of them. The top-1 label matches eager on 8 of 8, the top-five sets overlap
+40 of 40, and no sigmoid probability moves by more than the figure in the table.
+
+The distance the error has to cover is printed too: the gap between the winning logit and
+the runner-up is at least **3.09** on these clips, which every build's shift is far
+inside. Agreement alone would not show this — a build returning a constant vector would
+agree with a broken reference on every clip — so the label check is there as well.
+
+## The features are the caller's job, and the recipe is exact
+
+AST's front end is `torchaudio.compliance.kaldi.fbank`: a Kaldi-compatible filterbank with
+its own windowing and edge handling. Reimplementing it inside the graph would be a second
+model's worth of work for a transform `transformers` runs in two lines, so the graph starts
+at the spectrogram. The recipe is read off the model's own preprocessor rather than written
+from memory:
+
+```python
+from transformers import AutoFeatureExtractor
+extractor = AutoFeatureExtractor.from_pretrained("MIT/ast-finetuned-audioset-10-10-0.4593")
+inputs = extractor(waveform, sampling_rate=16000, return_tensors="pt")["input_values"]
+```
+
+Getting it wrong does not throw. It shifts every probability.
+
+**The output is multi-label**: apply `sigmoid`, not `softmax`. A ten-second clip can be
+speech *and* music *and* a car at once, which is the point of AudioSet.
+
+## Conversion
+
+```bash
+python convert/export_audiocls.py audioset
+python convert/check_audiocls.py audioset int8
+```
+
+(conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models))