Instructions to use litert-community/MediaPipe-AudioClassifier-YAMNet with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use litert-community/MediaPipe-AudioClassifier-YAMNet with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
MediaPipe YAMNet (Audio Classifier)
Audio event classification model (yamnet.tflite, 4.1 MB, float32 PCM I/O with internal 8-bit weight quantization) trained on AudioSet to recognize 521 sound classes (speech, musical instruments, animals, alarms, environmental sounds) in the MediaPipe Audio Classifier (AudioClassifier) task.
Documentation: MediaPipe Audio Classifier Docs
Samples: Android · iOS · Python Colab · Web · Flutter
Try live: Web Demo
Takes 16 kHz mono float32 waveform windows (15600 samples = 0.975 s, normalized in [-1.0, 1.0]), computes log-mel spectrograms internally, and outputs [1, 521] class scores aligned with yamnet_label_list.txt (embedded inside TFLITE_METADATA and included as a companion file).
Files
| File | Precision | Size | SHA-256 |
|---|---|---|---|
yamnet.tflite |
float32 |
4.13 MB (4,126,810 B) | 4d8b4a53282dc83ef04e3e7dbc4fbc98082e34e44ed798e16c3a0cdd4c584faf |
Companion Files in Repo Root
| File | Role | Size | SHA-256 |
|---|---|---|---|
yamnet_label_list.txt |
Label map | 6.2 KB (6,230 B) | 8e1267a120c1932b7273c0d0e0c5529edbb9a35512b437b1c8982baa59047051 |
Download
pip install -U huggingface_hub
hf download litert-community/MediaPipe-AudioClassifier-YAMNet yamnet.tflite --local-dir .
Inputs, Outputs, and Model Specifications
yamnet.tflite
- Embedded
TFLITE_METADATA: nameyamnet/classification· version1· authorMediaPipe· packed associated files:yamnet_label_list.txt
| Direction | Tensor | Shape | Type | Notes |
|---|---|---|---|---|
| Input | waveform_binary |
[15600] |
float32 |
audio_clip — Input audio clip to be classified (16 kHz float32 waveform samples in range -1.0..1.0). |
| Output | tower0/network/layer32/final_output |
[1, 521] |
float32 |
scores — Scores in range 0..1.0 for each of the 521 output classes. — Labels: yamnet_label_list.txt |
Use with MediaPipe Tasks
Python Quickstart
pip install -U mediapipe huggingface_hub
import numpy as np
import mediapipe as mp
from huggingface_hub import hf_hub_download
model_path = hf_hub_download(repo_id="litert-community/MediaPipe-AudioClassifier-YAMNet", filename="yamnet.tflite")
BaseOptions = mp.tasks.BaseOptions
AudioClassifier = mp.tasks.audio.AudioClassifier
AudioClassifierOptions = mp.tasks.audio.AudioClassifierOptions
AudioRunningMode = mp.tasks.audio.RunningMode
AudioData = mp.tasks.components.containers.AudioData
options = AudioClassifierOptions(
base_options=BaseOptions(model_asset_path=model_path),
running_mode=AudioRunningMode.AUDIO_CLIPS,
max_results=5,
)
clip_samples = np.zeros(16000, dtype=np.float32) # replace with 16 kHz mono PCM in [-1.0, 1.0]
audio_data = AudioData.create_from_array(clip_samples, sample_rate=16000)
with AudioClassifier.create_from_options(options) as classifier:
for window in classifier.classify(audio_data):
print([(c.category_name, round(c.score, 3)) for c in window.classifications[0].categories])
Cross-Platform Guides & Samples
- Task overview & options: Audio Classifier documentation
- Platform setup guides: Python · Web / JavaScript · Android (Kotlin / Java) · iOS (Swift) · Flutter (Dart)
- Runnable sample apps: Web Demo · Web Source · Android · iOS · Python Notebook · Flutter
Latency Benchmarks
End-to-end MediaPipe Task pipeline average latency reported by the Audio Classifier documentation:
| Model Variant | Device | CPU Latency | GPU Latency |
|---|---|---|---|
| YamNet | Pixel 6 | 12.29 ms | — |
Intended Use, Limitations, and Fairness
- Intended applications: Real-time audio event tagging and acoustic scene recognition across the 521 AudioSet categories on mobile, web, desktop, and embedded devices.
- Limitations & considerations: Multi-label sound event classifier (not a speech-to-text ASR model or biometric speaker recognizer); expects 16 kHz mono audio normalized to
[-1.0, 1.0]. No standalone Google fairness PDF is published on the MediaPipe documentation page.
Provenance
Every model binary in this repository is byte-identical (unmodified FlatBuffer weights and embedded TFLITE_METADATA bytes) to the official artifact served by Google from storage.googleapis.com/mediapipe-models.
| Repo File | Upstream URL | GCS Generation | Upstream Last-Modified |
Pinned Bucket Version |
|---|---|---|---|---|
yamnet.tflite |
yamnet.tflite |
1682545011390892 |
Wed, 26 Apr 2023 21:36:51 GMT | /1/ |
License and Citation
- License:
apache-2.0— Stated as Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0) in the embeddedTFLITE_METADATAblock of the.tflitemodel file(s).
Citation
Trained on AudioSet (TFLITE_METADATA author: MediaPipe, Apache License 2.0).
- Downloads last month
- -