File size: 1,796 Bytes
9c591bb b427c65 d5d3bb6 b427c65 9c591bb b427c65 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | ---
license: apache-2.0
library_name: pytorch
pipeline_tag: audio-classification
base_model:
- justinchuby/Perch-onnx
- wrice/perch-v2-efficientnet-b3
tags:
- audio
- bioacoustics
- bird-classification
- das
- embeddings
---
# PERCH 2 PyTorch
PyTorch implementation of the complete PERCH 2 waveform model for
bioacoustic classification and 1536-dimensional audio embeddings.
## Load From Hugging Face
```python
import torch
model = torch.hub.load("janclemenslab/perch2_torch", "perch_v2").eval()
waveform = torch.zeros(5 * 32_000) # Mono 32 kHz audio.
with torch.no_grad():
outputs = model(waveform)
index = outputs["label"][0].argmax()
print(model.labels[index])
```
This call downloads `perch_v2_torch.pt` from this Hugging Face repository and
caches it locally.
Inputs are mono, 32 kHz waveforms with shape `(time,)` or `(batch, time)`.
Audio longer than five seconds is processed in overlapping windows and pooled.
## Outputs
- `embedding`: `(batch, 1536)` global embedding
- `spatial_embedding`: `(batch, time, frequency, 1536)` unpooled embedding
- `spectrogram`: log-mel spectrogram
- `label`: `(batch, 14795)` uncalibrated class logits
`model.labels[index]` maps a logit index to its embedded class name. Thresholds
should be calibrated for the target data.
## Reproducibility
The conversion code, demo notebook, and verification command are available in
[janclemenslab/perch2_torch](https://github.com/janclemenslab/perch2_torch).
## Attribution And License
Derived from the Apache-2.0 [PERCH ONNX model](https://huggingface.co/justinchuby/Perch-onnx)
and Apache-2.0 [wrice EfficientNet-B3 checkpoint](https://huggingface.co/wrice/perch-v2-efficientnet-b3).
The class taxonomy comes from the Apache-2.0 [PERCH model release](https://huggingface.co/cgeorgiaw/Perch).
|