| --- |
| language: |
| - en |
| license: mit |
| tags: |
| - bioacoustics |
| - audio-classification |
| - bird-sound-identification |
| - pytorch |
| - vision-transformers |
| - protoclr |
| - umap |
| - hdbscan |
| pipeline_tag: audio-classification |
| library_name: pytorch |
| metrics: |
| - accuracy |
| --- |
| |
| # 🦅 Edge-Optimized Bioacoustic Atlas & Real-Time Avian Classifier |
|
|
| [](https://colab.research.google.com/drive/1EL5VS_vAKvojPf5UPuQVFbK5gkgP51hB?usp=sharing) |
| [](https://huggingface.co/sukriramli/tiny-bird-diffusion) |
| [](https://pytorch.org/) |
| []() |
| [](https://opensource.org/licenses/MIT) |
|
|
| A decoupled, ultra-lightweight machine learning pipeline designed for low-latency edge deployment, automated avian species tracking, and interactive audio streaming. |
|
|
| By leveraging **Prototypical Contrastive Learning (ProtoCLR)** backbones combined with advanced topological manifold compression (**UMAP & HDBSCAN**), this system projects complex audio waveforms onto a dense, 2D geometric map. The resulting production architecture handles **168 unique biological species** across **149 autonomous eco-acoustic clusters** natively in a client browser window with sub-second latency—completely bypassing the need for compute-heavy cloud inferencing heads. |
|
|
| --- |
|
|
| ## 🛠️ The System Architecture Problem & Our Solution |
|
|
| ### Our Decoupled Geometric Solution |
| This repository implements a **decoupled mathematical pattern**. Heavy feature extraction is processed upfront. The complex high-dimensional latent space is then permanently compressed into a frozen geometric lookup coordinate plane. The client device only runs low-compute spatial distance algorithms, achieving zero-lag edge inference. |
|
|
| --- |
|
|
| ## 🔬 Core Engineering Pillars |
|
|
| ### 1. High-Ratio Manifold Compression |
| Instead of forcing edge hardware to hold dense classification layer weights, we isolate the 512-dimensional floating-point latent vectors generated by the transformer. We utilize **UMAP (Uniform Manifold Approximation and Projection)** to topology-map this high-dimensional array down to a highly constrained 2D coordinate vector (X, Y). This slashes the database RAM footprint by over **99%** while preserving semantic biological boundaries. |
|
|
| ### 2. Acoustic Domain Shift Mitigation (augment.py) |
| Pre-trained foundation models are typically trained on pristine, studio-grade wildlife audio recordings, causing them to fail frequently in noisy consumer spaces. To bridge this gap, our data preparation pipeline routes clean data shards through a custom acoustic corruption environment mimicking real-world conditions. |
|
|
| --- |
|
|
| ## 📁 Modular Codebase Layout |
|
|
| * `augment.py`: Digital Signal Processing (DSP) environment warping functions (Noise, Echo, Muffling filters). |
| * `pipeline.py`: Low-latency engineering pipeline managing model configuration and UMAP coordinate projection. |
| * `api.py`: Clean, production-ready prediction endpoint designed for real-time app integration. |
|
|
