How to use from the
Use from the
LiteRT library
# No code snippets available yet for this library.

# To use this model, check the repository files and the library's documentation.

# Want to help? PRs adding snippets are welcome at:
# https://github.com/huggingface/huggingface.js

MediaPipe YAMNet (Audio Classifier)

Audio event classification model (yamnet.tflite, 4.1 MB, float32 PCM I/O with internal 8-bit weight quantization) trained on AudioSet to recognize 521 sound classes (speech, musical instruments, animals, alarms, environmental sounds) in the MediaPipe Audio Classifier (AudioClassifier) task.

Documentation: MediaPipe Audio Classifier Docs

Samples: Android · iOS · Python Colab · Web · Flutter

Try live: Web Demo

Takes 16 kHz mono float32 waveform windows (15600 samples = 0.975 s, normalized in [-1.0, 1.0]), computes log-mel spectrograms internally, and outputs [1, 521] class scores aligned with yamnet_label_list.txt (embedded inside TFLITE_METADATA and included as a companion file).

Files

File Precision Size SHA-256
yamnet.tflite float32 4.13 MB (4,126,810 B) 4d8b4a53282dc83ef04e3e7dbc4fbc98082e34e44ed798e16c3a0cdd4c584faf

Companion Files in Repo Root

File Role Size SHA-256
yamnet_label_list.txt Label map 6.2 KB (6,230 B) 8e1267a120c1932b7273c0d0e0c5529edbb9a35512b437b1c8982baa59047051

Download

pip install -U huggingface_hub
hf download litert-community/MediaPipe-AudioClassifier-YAMNet yamnet.tflite --local-dir .

Inputs, Outputs, and Model Specifications

yamnet.tflite

  • Embedded TFLITE_METADATA: name yamnet/classification · version 1 · author MediaPipe · packed associated files: yamnet_label_list.txt
Direction Tensor Shape Type Notes
Input waveform_binary [15600] float32 audio_clip — Input audio clip to be classified (16 kHz float32 waveform samples in range -1.0..1.0).
Output tower0/network/layer32/final_output [1, 521] float32 scores — Scores in range 0..1.0 for each of the 521 output classes. — Labels: yamnet_label_list.txt

Use with MediaPipe Tasks

Python Quickstart

pip install -U mediapipe huggingface_hub
import numpy as np
import mediapipe as mp
from huggingface_hub import hf_hub_download

model_path = hf_hub_download(repo_id="litert-community/MediaPipe-AudioClassifier-YAMNet", filename="yamnet.tflite")

BaseOptions = mp.tasks.BaseOptions
AudioClassifier = mp.tasks.audio.AudioClassifier
AudioClassifierOptions = mp.tasks.audio.AudioClassifierOptions
AudioRunningMode = mp.tasks.audio.RunningMode
AudioData = mp.tasks.components.containers.AudioData

options = AudioClassifierOptions(
    base_options=BaseOptions(model_asset_path=model_path),
    running_mode=AudioRunningMode.AUDIO_CLIPS,
    max_results=5,
)
clip_samples = np.zeros(16000, dtype=np.float32)  # replace with 16 kHz mono PCM in [-1.0, 1.0]
audio_data = AudioData.create_from_array(clip_samples, sample_rate=16000)
with AudioClassifier.create_from_options(options) as classifier:
    for window in classifier.classify(audio_data):
        print([(c.category_name, round(c.score, 3)) for c in window.classifications[0].categories])

Cross-Platform Guides & Samples

Latency Benchmarks

End-to-end MediaPipe Task pipeline average latency reported by the Audio Classifier documentation:

Model Variant Device CPU Latency GPU Latency
YamNet Pixel 6 12.29 ms —

Intended Use, Limitations, and Fairness

  • Intended applications: Real-time audio event tagging and acoustic scene recognition across the 521 AudioSet categories on mobile, web, desktop, and embedded devices.
  • Limitations & considerations: Multi-label sound event classifier (not a speech-to-text ASR model or biometric speaker recognizer); expects 16 kHz mono audio normalized to [-1.0, 1.0]. No standalone Google fairness PDF is published on the MediaPipe documentation page.

Provenance

Every model binary in this repository is byte-identical (unmodified FlatBuffer weights and embedded TFLITE_METADATA bytes) to the official artifact served by Google from storage.googleapis.com/mediapipe-models.

Repo File Upstream URL GCS Generation Upstream Last-Modified Pinned Bucket Version
yamnet.tflite yamnet.tflite 1682545011390892 Wed, 26 Apr 2023 21:36:51 GMT /1/

License and Citation

  • License: apache-2.0 — Stated as Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0) in the embedded TFLITE_METADATA block of the .tflite model file(s).

Citation

Trained on AudioSet (TFLITE_METADATA author: MediaPipe, Apache License 2.0).

Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including litert-community/MediaPipe-AudioClassifier-YAMNet