File size: 7,719 Bytes
fcd63e1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
---
license: apache-2.0
library_name: litert
pipeline_tag: audio-classification
tags:
- mediapipe
- litert
- tflite
- audio-classification
- on-device
- android
- ios
- web
- python
- flutter
- audio
- audioset
- yamnet
- sound-event-detection
- fp32
- int8
---

# MediaPipe YAMNet (Audio Classifier)

Audio event classification model (`yamnet.tflite`, 4.1 MB, float32 PCM I/O with internal 8-bit weight quantization) trained on AudioSet to recognize **521 sound classes** (speech, musical instruments, animals, alarms, environmental sounds) in the [MediaPipe Audio Classifier (`AudioClassifier`)](https://developers.google.com/edge/mediapipe/solutions/audio/audio_classifier) task.

<div style="border: 1px solid rgba(59, 130, 246, 0.28); background-color: rgba(59, 130, 246, 0.06); border-radius: 8px; padding: 12px 16px; margin: 16px 0;">
  <p style="margin: 0 0 6px 0;"><strong>Documentation:</strong> <a href="https://developers.google.com/edge/mediapipe/solutions/audio/audio_classifier">MediaPipe Audio Classifier Docs</a></p>
  <p style="margin: 0 0 6px 0;"><strong>Samples:</strong> <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/audio_classifier/android">Android</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/audio_classifier/ios">iOS</a> · <a href="https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/audio_classifier/python/audio_classification.ipynb">Python Colab</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/audio-classifier.ts">Web</a> · <a href="https://github.com/google/flutter-mediapipe/tree/main/samples/audio/audio_classifier">Flutter</a></p>
  <p style="margin: 0;"><strong>Try live:</strong> <a href="https://google-ai-edge.github.io/mediapipe-samples-web/#/audio/audio_classifier">Web Demo</a></p>
</div>

Takes 16 kHz mono float32 waveform windows (`15600` samples = `0.975 s`, normalized in `[-1.0, 1.0]`), computes log-mel spectrograms internally, and outputs `[1, 521]` class scores aligned with [`yamnet_label_list.txt`](./yamnet_label_list.txt) (embedded inside `TFLITE_METADATA` and included as a companion file).

## Files

| File | Precision | Size | SHA-256 |
|---|---|---:|---|
| [`yamnet.tflite`](https://huggingface.co/litert-community/MediaPipe-AudioClassifier-YAMNet/blob/main/yamnet.tflite) | `float32` | 4.13 MB (4,126,810 B) | `4d8b4a53282dc83ef04e3e7dbc4fbc98082e34e44ed798e16c3a0cdd4c584faf` |

### Companion Files in Repo Root

| File | Role | Size | SHA-256 |
|---|---|---:|---|
| [`yamnet_label_list.txt`](./yamnet_label_list.txt) | Label map | 6.2 KB (6,230 B) | `8e1267a120c1932b7273c0d0e0c5529edbb9a35512b437b1c8982baa59047051` |

### Download

```bash
pip install -U huggingface_hub
hf download litert-community/MediaPipe-AudioClassifier-YAMNet yamnet.tflite --local-dir .
```

## Inputs, Outputs, and Model Specifications

### `yamnet.tflite`

- Embedded `TFLITE_METADATA`: name `yamnet/classification` · version `1` · author `MediaPipe` · packed associated files: `yamnet_label_list.txt`

| Direction | Tensor | Shape | Type | Notes |
|---|---|---|---|---|
| Input | `waveform_binary` | `[15600]` | `float32` | audio_clip — Input audio clip to be classified (16 kHz float32 waveform samples in range -1.0..1.0). |
| Output | `tower0/network/layer32/final_output` | `[1, 521]` | `float32` | scores — Scores in range 0..1.0 for each of the 521 output classes. — Labels: `yamnet_label_list.txt` |

## Use with MediaPipe Tasks

### Python Quickstart

```bash
pip install -U mediapipe huggingface_hub
```

```python
import numpy as np
import mediapipe as mp
from huggingface_hub import hf_hub_download

model_path = hf_hub_download(repo_id="litert-community/MediaPipe-AudioClassifier-YAMNet", filename="yamnet.tflite")

BaseOptions = mp.tasks.BaseOptions
AudioClassifier = mp.tasks.audio.AudioClassifier
AudioClassifierOptions = mp.tasks.audio.AudioClassifierOptions
AudioRunningMode = mp.tasks.audio.RunningMode
AudioData = mp.tasks.components.containers.AudioData

options = AudioClassifierOptions(
    base_options=BaseOptions(model_asset_path=model_path),
    running_mode=AudioRunningMode.AUDIO_CLIPS,
    max_results=5,
)
clip_samples = np.zeros(16000, dtype=np.float32)  # replace with 16 kHz mono PCM in [-1.0, 1.0]
audio_data = AudioData.create_from_array(clip_samples, sample_rate=16000)
with AudioClassifier.create_from_options(options) as classifier:
    for window in classifier.classify(audio_data):
        print([(c.category_name, round(c.score, 3)) for c in window.classifications[0].categories])
```

### Cross-Platform Guides & Samples

- **Task overview & options:** [Audio Classifier documentation](https://developers.google.com/edge/mediapipe/solutions/audio/audio_classifier)
- **Platform setup guides:** [Python](https://developers.google.com/edge/mediapipe/solutions/setup_python) · [Web / JavaScript](https://developers.google.com/edge/mediapipe/solutions/setup_web) · [Android (Kotlin / Java)](https://developers.google.com/edge/mediapipe/solutions/setup_android) · [iOS (Swift)](https://developers.google.com/edge/mediapipe/solutions/setup_ios) · [Flutter (Dart)](https://developers.google.com/edge/mediapipe/solutions/setup_flutter)
- **Runnable sample apps:** [Web Demo](https://google-ai-edge.github.io/mediapipe-samples-web/#/audio/audio_classifier) · [Web Source](https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/audio-classifier.ts) · [Android](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/audio_classifier/android) · [iOS](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/audio_classifier/ios) · [Python Notebook](https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/audio_classifier/python/audio_classification.ipynb) · [Flutter](https://github.com/google/flutter-mediapipe/tree/main/samples/audio/audio_classifier)

## Latency Benchmarks

End-to-end MediaPipe Task pipeline average latency reported by the [Audio Classifier documentation](https://developers.google.com/edge/mediapipe/solutions/audio/audio_classifier#benchmarks):

| Model Variant | Device | CPU Latency | GPU Latency |
|---|---|---:|---:|
| YamNet | Pixel 6 | 12.29 ms | — |

## Intended Use, Limitations, and Fairness

- **Intended applications:** Real-time audio event tagging and acoustic scene recognition across the 521 AudioSet categories on mobile, web, desktop, and embedded devices.
- **Limitations & considerations:** Multi-label sound event classifier (not a speech-to-text ASR model or biometric speaker recognizer); expects 16 kHz mono audio normalized to `[-1.0, 1.0]`. No standalone Google fairness PDF is published on the MediaPipe documentation page.

## Provenance

Every model binary in this repository is byte-identical (unmodified FlatBuffer weights and embedded `TFLITE_METADATA` bytes) to the official artifact served by Google from `storage.googleapis.com/mediapipe-models`.

| Repo File | Upstream URL | GCS Generation | Upstream `Last-Modified` | Pinned Bucket Version |
|---|---|---|---|---|
| `yamnet.tflite` | [`yamnet.tflite`](https://storage.googleapis.com/mediapipe-models/audio_classifier/yamnet/float32/latest/yamnet.tflite) | `1682545011390892` | Wed, 26 Apr 2023 21:36:51 GMT | `/1/` |

## License and Citation

- License: `apache-2.0` — Stated as Apache License, Version 2.0 (`http://www.apache.org/licenses/LICENSE-2.0`) in the embedded `TFLITE_METADATA` block of the `.tflite` model file(s).

### Citation

Trained on [AudioSet](https://research.google.com/audioset/) (`TFLITE_METADATA` author: MediaPipe, Apache License 2.0).