Audio Classification
LiteRT
LiteRT
mediapipe
on-device
android
ios
web
python
flutter
audio
audioset
yamnet
sound-event-detection
fp32
int8
Instructions to use litert-community/MediaPipe-AudioClassifier-YAMNet with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use litert-community/MediaPipe-AudioClassifier-YAMNet with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
File size: 7,719 Bytes
fcd63e1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 | ---
license: apache-2.0
library_name: litert
pipeline_tag: audio-classification
tags:
- mediapipe
- litert
- tflite
- audio-classification
- on-device
- android
- ios
- web
- python
- flutter
- audio
- audioset
- yamnet
- sound-event-detection
- fp32
- int8
---
# MediaPipe YAMNet (Audio Classifier)
Audio event classification model (`yamnet.tflite`, 4.1 MB, float32 PCM I/O with internal 8-bit weight quantization) trained on AudioSet to recognize **521 sound classes** (speech, musical instruments, animals, alarms, environmental sounds) in the [MediaPipe Audio Classifier (`AudioClassifier`)](https://developers.google.com/edge/mediapipe/solutions/audio/audio_classifier) task.
<div style="border: 1px solid rgba(59, 130, 246, 0.28); background-color: rgba(59, 130, 246, 0.06); border-radius: 8px; padding: 12px 16px; margin: 16px 0;">
<p style="margin: 0 0 6px 0;"><strong>Documentation:</strong> <a href="https://developers.google.com/edge/mediapipe/solutions/audio/audio_classifier">MediaPipe Audio Classifier Docs</a></p>
<p style="margin: 0 0 6px 0;"><strong>Samples:</strong> <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/audio_classifier/android">Android</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/audio_classifier/ios">iOS</a> · <a href="https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/audio_classifier/python/audio_classification.ipynb">Python Colab</a> · <a href="https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/audio-classifier.ts">Web</a> · <a href="https://github.com/google/flutter-mediapipe/tree/main/samples/audio/audio_classifier">Flutter</a></p>
<p style="margin: 0;"><strong>Try live:</strong> <a href="https://google-ai-edge.github.io/mediapipe-samples-web/#/audio/audio_classifier">Web Demo</a></p>
</div>
Takes 16 kHz mono float32 waveform windows (`15600` samples = `0.975 s`, normalized in `[-1.0, 1.0]`), computes log-mel spectrograms internally, and outputs `[1, 521]` class scores aligned with [`yamnet_label_list.txt`](./yamnet_label_list.txt) (embedded inside `TFLITE_METADATA` and included as a companion file).
## Files
| File | Precision | Size | SHA-256 |
|---|---|---:|---|
| [`yamnet.tflite`](https://huggingface.co/litert-community/MediaPipe-AudioClassifier-YAMNet/blob/main/yamnet.tflite) | `float32` | 4.13 MB (4,126,810 B) | `4d8b4a53282dc83ef04e3e7dbc4fbc98082e34e44ed798e16c3a0cdd4c584faf` |
### Companion Files in Repo Root
| File | Role | Size | SHA-256 |
|---|---|---:|---|
| [`yamnet_label_list.txt`](./yamnet_label_list.txt) | Label map | 6.2 KB (6,230 B) | `8e1267a120c1932b7273c0d0e0c5529edbb9a35512b437b1c8982baa59047051` |
### Download
```bash
pip install -U huggingface_hub
hf download litert-community/MediaPipe-AudioClassifier-YAMNet yamnet.tflite --local-dir .
```
## Inputs, Outputs, and Model Specifications
### `yamnet.tflite`
- Embedded `TFLITE_METADATA`: name `yamnet/classification` · version `1` · author `MediaPipe` · packed associated files: `yamnet_label_list.txt`
| Direction | Tensor | Shape | Type | Notes |
|---|---|---|---|---|
| Input | `waveform_binary` | `[15600]` | `float32` | audio_clip — Input audio clip to be classified (16 kHz float32 waveform samples in range -1.0..1.0). |
| Output | `tower0/network/layer32/final_output` | `[1, 521]` | `float32` | scores — Scores in range 0..1.0 for each of the 521 output classes. — Labels: `yamnet_label_list.txt` |
## Use with MediaPipe Tasks
### Python Quickstart
```bash
pip install -U mediapipe huggingface_hub
```
```python
import numpy as np
import mediapipe as mp
from huggingface_hub import hf_hub_download
model_path = hf_hub_download(repo_id="litert-community/MediaPipe-AudioClassifier-YAMNet", filename="yamnet.tflite")
BaseOptions = mp.tasks.BaseOptions
AudioClassifier = mp.tasks.audio.AudioClassifier
AudioClassifierOptions = mp.tasks.audio.AudioClassifierOptions
AudioRunningMode = mp.tasks.audio.RunningMode
AudioData = mp.tasks.components.containers.AudioData
options = AudioClassifierOptions(
base_options=BaseOptions(model_asset_path=model_path),
running_mode=AudioRunningMode.AUDIO_CLIPS,
max_results=5,
)
clip_samples = np.zeros(16000, dtype=np.float32) # replace with 16 kHz mono PCM in [-1.0, 1.0]
audio_data = AudioData.create_from_array(clip_samples, sample_rate=16000)
with AudioClassifier.create_from_options(options) as classifier:
for window in classifier.classify(audio_data):
print([(c.category_name, round(c.score, 3)) for c in window.classifications[0].categories])
```
### Cross-Platform Guides & Samples
- **Task overview & options:** [Audio Classifier documentation](https://developers.google.com/edge/mediapipe/solutions/audio/audio_classifier)
- **Platform setup guides:** [Python](https://developers.google.com/edge/mediapipe/solutions/setup_python) · [Web / JavaScript](https://developers.google.com/edge/mediapipe/solutions/setup_web) · [Android (Kotlin / Java)](https://developers.google.com/edge/mediapipe/solutions/setup_android) · [iOS (Swift)](https://developers.google.com/edge/mediapipe/solutions/setup_ios) · [Flutter (Dart)](https://developers.google.com/edge/mediapipe/solutions/setup_flutter)
- **Runnable sample apps:** [Web Demo](https://google-ai-edge.github.io/mediapipe-samples-web/#/audio/audio_classifier) · [Web Source](https://github.com/google-ai-edge/mediapipe-samples-web/blob/main/src/tasks/audio-classifier.ts) · [Android](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/audio_classifier/android) · [iOS](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/audio_classifier/ios) · [Python Notebook](https://colab.research.google.com/github/google-ai-edge/mediapipe-samples/blob/main/examples/audio_classifier/python/audio_classification.ipynb) · [Flutter](https://github.com/google/flutter-mediapipe/tree/main/samples/audio/audio_classifier)
## Latency Benchmarks
End-to-end MediaPipe Task pipeline average latency reported by the [Audio Classifier documentation](https://developers.google.com/edge/mediapipe/solutions/audio/audio_classifier#benchmarks):
| Model Variant | Device | CPU Latency | GPU Latency |
|---|---|---:|---:|
| YamNet | Pixel 6 | 12.29 ms | — |
## Intended Use, Limitations, and Fairness
- **Intended applications:** Real-time audio event tagging and acoustic scene recognition across the 521 AudioSet categories on mobile, web, desktop, and embedded devices.
- **Limitations & considerations:** Multi-label sound event classifier (not a speech-to-text ASR model or biometric speaker recognizer); expects 16 kHz mono audio normalized to `[-1.0, 1.0]`. No standalone Google fairness PDF is published on the MediaPipe documentation page.
## Provenance
Every model binary in this repository is byte-identical (unmodified FlatBuffer weights and embedded `TFLITE_METADATA` bytes) to the official artifact served by Google from `storage.googleapis.com/mediapipe-models`.
| Repo File | Upstream URL | GCS Generation | Upstream `Last-Modified` | Pinned Bucket Version |
|---|---|---|---|---|
| `yamnet.tflite` | [`yamnet.tflite`](https://storage.googleapis.com/mediapipe-models/audio_classifier/yamnet/float32/latest/yamnet.tflite) | `1682545011390892` | Wed, 26 Apr 2023 21:36:51 GMT | `/1/` |
## License and Citation
- License: `apache-2.0` — Stated as Apache License, Version 2.0 (`http://www.apache.org/licenses/LICENSE-2.0`) in the embedded `TFLITE_METADATA` block of the `.tflite` model file(s).
### Citation
Trained on [AudioSet](https://research.google.com/audioset/) (`TFLITE_METADATA` author: MediaPipe, Apache License 2.0).
|