postpop commited on
Commit
b427c65
·
verified ·
1 Parent(s): 9c591bb

Add model card

Browse files
Files changed (1) hide show
  1. README.md +57 -0
README.md CHANGED
@@ -1,3 +1,60 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ library_name: pytorch
4
+ pipeline_tag: audio-classification
5
+ base_model:
6
+ - justinchuby/Perch-onnx
7
+ - wrice/perch-v2-efficientnet-b3
8
+ tags:
9
+ - audio
10
+ - bioacoustics
11
+ - bird-classification
12
+ - embeddings
13
  ---
14
+
15
+ # PERCH 2 PyTorch
16
+
17
+ PyTorch implementation of the complete PERCH 2 waveform model for
18
+ bioacoustic classification and 1536-dimensional audio embeddings.
19
+
20
+ ## Load From Hugging Face
21
+
22
+ ```python
23
+ import torch
24
+
25
+ model = torch.hub.load("janclemenslab/perch2_torch", "perch_v2").eval()
26
+ waveform = torch.zeros(5 * 32_000) # Mono 32 kHz audio.
27
+
28
+ with torch.no_grad():
29
+ outputs = model(waveform)
30
+
31
+ index = outputs["label"][0].argmax()
32
+ print(model.labels[index])
33
+ ```
34
+
35
+ This call downloads `perch_v2_torch.pt` from this Hugging Face repository and
36
+ caches it locally.
37
+
38
+ Inputs are mono, 32 kHz waveforms with shape `(time,)` or `(batch, time)`.
39
+ Audio longer than five seconds is processed in overlapping windows and pooled.
40
+
41
+ ## Outputs
42
+
43
+ - `embedding`: `(batch, 1536)` global embedding
44
+ - `spatial_embedding`: `(batch, time, frequency, 1536)` unpooled embedding
45
+ - `spectrogram`: log-mel spectrogram
46
+ - `label`: `(batch, 14795)` uncalibrated class logits
47
+
48
+ `model.labels[index]` maps a logit index to its embedded class name. Thresholds
49
+ should be calibrated for the target data.
50
+
51
+ ## Reproducibility
52
+
53
+ The conversion code, demo notebook, and verification command are available in
54
+ [janclemenslab/perch2_torch](https://github.com/janclemenslab/perch2_torch).
55
+
56
+ ## Attribution And License
57
+
58
+ Derived from the Apache-2.0 [PERCH ONNX model](https://huggingface.co/justinchuby/Perch-onnx)
59
+ and Apache-2.0 [wrice EfficientNet-B3 checkpoint](https://huggingface.co/wrice/perch-v2-efficientnet-b3).
60
+ The class taxonomy comes from the Apache-2.0 [PERCH model release](https://huggingface.co/cgeorgiaw/Perch).