BirdNET-Plus-V3.0 / README.md
tphakala's picture
Derive backend claims from measured runs; state fp32 only
0ad5785 verified
|
Raw History Blame Contribute Delete
9.45 kB
---
pretty_name: BirdNET+ V3.0 (GPU-native full + regional catalog)
license: apache-2.0
tags:
- bioacoustics
- birdnet
- bird
- vocalization
- audio-classification
- onnx
library_name: onnx
---
# BirdNET+ V3.0 (GPU-native full + regional catalog)
BirdNET+ V3.0 acoustic classifier, re-exported to a
**GPU-native ONNX** graph, plus a catalog of **region-specific slices** that are
much smaller while staying numerically identical to the full model on the
species they keep.
## Licence and terms
Each release came with its own upstream terms, listed here with its licence. The
repo-level licence tag is the current release's.
- **BirdNET+ V3.0** (`birdnet-plus-v3.0`), Apache-2.0: the Apache-2.0 model licence and acceptable-use guidance of the BirdNET+ V3.0 release (Zenodo, https://doi.org/10.5281/zenodo.24162518), `LICENSE-v3.0.md` and `ACCEPTABLE_USE-v3.0.md` at the repo root, unchanged from upstream.
## Origin and attribution
**Powered by BirdNET** (https://birdnet.cornell.edu/). BirdNET+ V3.0 is developed
by the **K. Lisa Yang Center for Conservation Bioacoustics (Cornell University)**,
**Chemnitz University of Technology**, and the **Museum fuer Naturkunde Berlin**.
It builds on Kahl et al. (2021) and Lasseck (2018). Architecture of this release:
EfficientNetV2-S backbone, three-head classifier, 11,725 classes.
The GPU-native conversion and the regional slicing in this repo are derived work
under Apache-2.0.
## Releases in this repo
The current release is **BirdNET+ V3.0** (`birdnet-plus-v3.0`).
## What "GPU-native" means
The BirdNET team's ONNX export computes its log-mel front-end with an in-graph
ONNX `STFT` op. ONNX Runtime has no CUDA/TensorRT kernel for `STFT`, and OpenVINO
cannot translate it at all, so the stock export cannot run on GPU or OpenVINO. In
these models the `STFT` is replaced with a mathematically equivalent `Conv1d`
(framing + windowed DFT), so the graph holds no op those runtimes lack.
Every file was run on **ONNX Runtime CPU** (x86-64 and a Raspberry Pi 5) and on **OpenVINO** at f32 (CPU on both, and an Intel integrated GPU), and its scores were compared with the x86-64 ONNX Runtime scores of the same file; the manifest's `supported` and `verified_on` follow those runs. OpenVINO must run at f32: its default f16 hint gives wrong scores on the Cortex-A76 and on Intel GPUs. It is not recommended: its margin over ONNX Runtime CPU was 7% at most where it was measured. The CUDA and TensorRT entries were not run for this release. One model serves CPU and GPU; there is no separate CPU/GPU
build.
Input: raw audio `[batch, 160000]` (**5 s @ 32 kHz**, matching Perch v2 so a host
can feed both classifiers the same buffer). Outputs: `predictions [batch, 11,725]`
(a static dimension, sigmoid already applied) and `embeddings [batch, 1280]`.
## Variants and hardware
| precision | best for |
|---|---|
| `-fp32-` | **run this**: ONNX Runtime and OpenVINO (BirdNET-Go runs v3.0 at f32 on every OpenVINO device) |
This release ships FP32 only. For low-RAM CPU devices the right choice is a
**regional FP32** model.
int8-arm is not shipped for v3.0: the backbone is Conv-dominated and int8-arm only
quantizes MatMul/Gemm weights, so it barely helps here.
## Full model
| file | classes | MB |
|---|---:|---:|
| `full/birdnet-plus-v3.0-fp32-b1.onnx` | 11,725 | 361 |
| `full/birdnet-plus-v3.0-labels-b1.txt` | 11,725 | - |
## Why regional models
The three-head classifier holds a large share of the model's parameters, so
restricting it to a region's species set cuts memory and download size. Each tile
is **numerically identical** to the full model on the species it keeps (max abs
difference vs the full model < 1e-6); the only change is that out-of-region
species are not emitted, which at a fixed monitoring location is exactly what you
want. Region species sets come from the **BirdNET Geomodel v3.0.2** range filter
(species by occurrence probability, up to a per-tile cap, plus a cosmopolitan core),
using the same tiles as the Perch v2 regional catalog. The cap is 800 for most
temperate tiles and higher for species-rich ones. Every tile also contains Homo sapiens and the release's non-animal sound classes, whatever its range filter says.
A species the geomodel names differently from the classifier (a moved genus, a
changed spelling) is matched through a reviewed rename table, so the tiles keep
species an exact name match would have dropped.
## Regional catalog
Each tile ships an fp32 model + labels + indices + `coverage.png` +
metadata. `regional/groups.json` lists the continent grouping.
### Europe
| region | coverage | classes | fp32 MB |
|---|---|---:|---:|
| `azores` | <img src="regional/azores/coverage.png" width="210"> | 242 | 113.7 |
| `baltics` | <img src="regional/baltics/coverage.png" width="210"> | 460 | 118.4 |
| `british-isles` | <img src="regional/british-isles/coverage.png" width="210"> | 579 | 121.0 |
| `canary-islands` | <img src="regional/canary-islands/coverage.png" width="210"> | 412 | 117.4 |
| `central-europe` | <img src="regional/central-europe/coverage.png" width="210"> | 667 | 122.9 |
| `eastern-europe` | <img src="regional/eastern-europe/coverage.png" width="210"> | 540 | 120.1 |
| `iberia` | <img src="regional/iberia/coverage.png" width="210"> | 649 | 122.5 |
| `iceland` | <img src="regional/iceland/coverage.png" width="210"> | 409 | 117.3 |
| `madeira` | <img src="regional/madeira/coverage.png" width="210"> | 274 | 114.4 |
| `nordic` | <img src="regional/nordic/coverage.png" width="210"> | 443 | 118.0 |
| `southern-europe` | <img src="regional/southern-europe/coverage.png" width="210"> | 638 | 122.2 |
| `svalbard` | <img src="regional/svalbard/coverage.png" width="210"> | 298 | 114.9 |
| `western-palearctic` | <img src="regional/western-palearctic/coverage.png" width="210"> | 1418 | 139.0 |
### Asia
| region | coverage | classes | fp32 MB |
|---|---|---:|---:|
| `china-north-central` | <img src="regional/china-north-central/coverage.png" width="210"> | 791 | 125.5 |
| `china-northeast` | <img src="regional/china-northeast/coverage.png" width="210"> | 688 | 123.3 |
| `china-southeast` | <img src="regional/china-southeast/coverage.png" width="210"> | 1178 | 133.9 |
| `china-southwest` | <img src="regional/china-southwest/coverage.png" width="210"> | 1228 | 134.9 |
| `himalaya` | <img src="regional/himalaya/coverage.png" width="210"> | 1226 | 134.9 |
| `indo-gangetic` | <img src="regional/indo-gangetic/coverage.png" width="210"> | 1199 | 134.3 |
| `japan` | <img src="regional/japan/coverage.png" width="210"> | 609 | 121.6 |
| `south-asia-peninsular` | <img src="regional/south-asia-peninsular/coverage.png" width="210"> | 671 | 122.9 |
| `tibet` | <img src="regional/tibet/coverage.png" width="210"> | 1226 | 134.9 |
### North America
| region | coverage | classes | fp32 MB |
|---|---|---:|---:|
| `canada-alaska` | <img src="regional/canada-alaska/coverage.png" width="210"> | 777 | 125.2 |
| `north-america-east` | <img src="regional/north-america-east/coverage.png" width="210"> | 818 | 126.1 |
| `north-america-west` | <img src="regional/north-america-west/coverage.png" width="210"> | 821 | 126.2 |
### South America
| region | coverage | classes | fp32 MB |
|---|---|---:|---:|
| `amazonia` | <img src="regional/amazonia/coverage.png" width="210"> | 3182 | 177.0 |
| `andes` | <img src="regional/andes/coverage.png" width="210"> | 3314 | 179.8 |
| `eastern-brazil` | <img src="regional/eastern-brazil/coverage.png" width="210"> | 1972 | 150.9 |
| `galapagos` | <img src="regional/galapagos/coverage.png" width="210"> | 156 | 111.9 |
| `southern-cone` | <img src="regional/southern-cone/coverage.png" width="210"> | 1639 | 143.8 |
### Africa
| region | coverage | classes | fp32 MB |
|---|---|---:|---:|
| `cape-verde` | <img src="regional/cape-verde/coverage.png" width="210"> | 162 | 112.0 |
| `central-africa` | <img src="regional/central-africa/coverage.png" width="210"> | 1537 | 141.6 |
| `east-africa` | <img src="regional/east-africa/coverage.png" width="210"> | 1824 | 147.8 |
| `mauritius` | <img src="regional/mauritius/coverage.png" width="210"> | 90 | 110.4 |
| `reunion` | <img src="regional/reunion/coverage.png" width="210"> | 92 | 110.5 |
| `sao-tome-principe` | <img src="regional/sao-tome-principe/coverage.png" width="210"> | 133 | 111.4 |
| `seychelles` | <img src="regional/seychelles/coverage.png" width="210"> | 137 | 111.4 |
| `southern-africa` | <img src="regional/southern-africa/coverage.png" width="210"> | 820 | 126.1 |
| `west-africa` | <img src="regional/west-africa/coverage.png" width="210"> | 1027 | 130.6 |
### Oceania
| region | coverage | classes | fp32 MB |
|---|---|---:|---:|
| `australia-east` | <img src="regional/australia-east/coverage.png" width="210"> | 757 | 124.8 |
| `hawaii` | <img src="regional/hawaii/coverage.png" width="210"> | 294 | 114.8 |
| `new-caledonia` | <img src="regional/new-caledonia/coverage.png" width="210"> | 190 | 112.6 |
| `new-zealand` | <img src="regional/new-zealand/coverage.png" width="210"> | 289 | 114.7 |
## Deployment
- Point your host at a tile's model + its labels file (matching counts) and
restart. Labels are `Scientific name_Common name`.
- Confidence is a per-class sigmoid; slicing does not change a kept class's score
(unlike a softmax model), so thresholds transfer directly from the full model.
- Binaries are hosted here on HuggingFace; the derivation tooling lives separately.