| --- |
| license: mit |
| datasets: |
| - imageomics/BaboonLand |
| language: |
| - en |
| tags: |
| - biology |
| - CV |
| - images |
| - animals |
| - zebra |
| - giraffe |
| - behavior |
| - behavior recognition |
| - annotation |
| - UAV |
| - drone |
| - video |
| model_description: "Behavior recognition model for in situ drone videos of baboons, built using an X3D model. It was trained on the BaboonLand mini-scene dataset, which is comprised of 20 hours of aerial video footage of baboons captured using a DJI Mavic 2S drone." |
| --- |
| |
| # Model Card for x3d-BaboonLand |
|
|
| x3d-BaboonLand is a behavior recognition model for in situ drone videos of baboons, built using the X3D architecture. It was trained on the [BaboonLand](https://huggingface.co/datasets/imageomics/BaboonLand) dataset, which includes both spatiotemporal clips (mini-scenes) and behavior annotations provided by an expert behavioral ecologist. |
|
|
| ## Model Details |
|
|
| ### Model Description |
|
|
| - **Developed by:** Isla Duporge, Maksim Kholiavchenko, Roi Harel, Scott Wolf, Daniel Rubenstein, Meg Crofoot, Tanya Berger-Wolf, Stephen Lee, Julie Barreau, Jenna Kline, Michelle Ramirez, Charles Stewart |
| - **Model type:** X3D-L |
| - **License:** MIT |
| - **Fine-tuned from model:** [X3D-L](https://github.com/facebookresearch/SlowFast/blob/main/configs/Kinetics/X3D_L.yaml) |
|
|
| This model was developed for the benefit of the community as an open-source product; we request that derivative products also remain open-source. |
|
|
| ### Model Sources |
|
|
| - **Repository:** [kabr-tools](https://github.com/Imageomics/kabr-tools) |
| - **BaboonLand scripts:** [BaboonLand/scripts](https://huggingface.co/datasets/imageomics/BaboonLand/tree/main/BaboonLand/scripts) |
| - **Paper:** [BaboonLand Dataset: Tracking Primates in the Wild and Automating Behaviour Recognition from Drone Videos](https://link.springer.com/article/10.1007/s11263-025-02493-5) |
| - **Project Page:** [BaboonLand Project Page](https://baboonland.xyz) |
|
|
| ### Data Processing Software |
|
|
| The [kabr-tools](https://github.com/Imageomics/kabr-tools) repository is the primary open-source package used as the basis for processing and formatting data for behavior-recognition workflows. For BaboonLand, we did **not** duplicate the full codebase into this model repository. Instead, we used the `kabr-tools` workflow with BaboonLand-specific inputs and lightweight script adaptations. |
|
|
| In particular, several scripts used for BaboonLand were derived from `kabr-tools` utilities, but were adapted for this dataset and renamed for clarity. The resulting BaboonLand-specific scripts are provided here: |
|
|
| [BaboonLand/scripts](https://huggingface.co/datasets/imageomics/BaboonLand/tree/main/BaboonLand/scripts) |
|
|
| These scripts document the dataset-specific preprocessing used for BaboonLand, while `kabr-tools` remains the main reference implementation for the broader workflow. |
|
|
| ## Uses |
|
|
| This model is intended for baboon behavior recognition from in situ drone videos. |
|
|
| ### Out-of-Scope Use |
|
|
| This model was trained to classify behavior from drone videos of baboons in Kenya. It may not perform well for other species, environments, camera viewpoints, annotation schemes, or behavior taxonomies. |
|
|
| ## How to Get Started with the Model |
|
|
| Please see the illustrative examples in the [kabr-tools](https://imageomics.github.io/kabr-tools) for the general workflow. |
|
|
| ## Training Details |
|
|
| We include the configuration file ([config.yaml](https://huggingface.co/imageomics/x3d-BaboonLand/blob/main/config.yaml)) used for X3D training in SlowFast. |
|
|
| ### Training Data |
|
|
| This model was trained on the [BaboonLand](https://huggingface.co/datasets/imageomics/BaboonLand) dataset. |
|
|
| #### Training Hyperparameters |
|
|
| The model was trained for 120 epochs using a batch size of 5. |
| We used the EQL loss function to address the long-tailed class distribution and SGD optimization with a learning rate of `1e-5`. |
| We used a sample rate of `16x5` and random weight initialization. |
|
|
| ## Evaluation |
|
|
| The model was evaluated using the [SlowFast](https://github.com/facebookresearch/SlowFast) framework, specifically the [test_net.py](https://github.com/facebookresearch/SlowFast/blob/main/tools/test_net.py) evaluation script. |
|
|
| ### Testing Data |
|
|
| We provide a train-test split of the mini-scenes from the [BaboonLand](https://huggingface.co/datasets/imageomics/BaboonLand) dataset for evaluation, with 75% used for training and 25% for testing. No mini-scene was split across train and test partitions. |
|
|
| #### Metrics |
|
|
| We report Top-1, Top-3, and Top-5 macro-scores. For full details, please refer to the [paper](https://link.springer.com/article/10.1007/s11263-025-02493-5). |
|
|
| **Micro-Average (Per Instance) Scores** |
|
|
| | WI | BS | Top-1 | Top-3 | Top-5 | |
| |---------|----|------:|------:|------:| |
| | Random | 5 | 64.89 | 92.54 | 96.66 | |
|
|
| ### Model Architecture and Objective |
|
|
| Please see the [base model description](https://arxiv.org/pdf/2004.04730). |
|
|
| #### Hardware |
|
|
| Running the X3D-L model requires a modern NVIDIA GPU with CUDA support. X3D-L is designed to be computationally efficient and typically requires 10–16 GB of GPU memory during training. |
|
|
| ## Citation |
|
|
| **BibTeX:** |
|
|
| If you use our model in your work, please cite our paper. |
|
|
| **Paper** |
| ``` |
| @article{duporge2025baboonland, |
| title={BaboonLand Dataset: Tracking Primates in the Wild and Automating Behaviour Recognition from Drone Videos}, |
| author={Duporge, Isla and Kholiavchenko, Maksim and Harel, Roi and Wolf, Scott and Rubenstein, Daniel I and Crofoot, Margaret C and Berger-Wolf, Tanya and Lee, Stephen J and Barreau, Julie and Kline, Jenna and Ramirez, Michelle and Stewart, Charles}, |
| journal={International Journal of Computer Vision}, |
| pages={1--12}, |
| year={2025}, |
| publisher={Springer} |
| } |
| ``` |
|
|
|
|
| ## Acknowledgements |
|
|
| This work was supported by the [Imageomics Institute](https://imageomics.org), which is funded by the US National Science Foundation's Harnessing the Data Revolution (HDR) program under [Award #2118240](https://www.nsf.gov/awardsearch/showAward?AWD_ID=2118240) (Imageomics: A New Frontier of Biological Information Powered by Knowledge-Guided Machine Learning). Additional support was also provided by the [AI Institute for Intelligent Cyberinfrastructure with Computational Learning in the Environment (ICICLE)](https://icicle.osu.edu/), which is funded by the US National Science Foundation under [Award #2112606](https://www.nsf.gov/awardsearch/showAward?AWD_ID=2112606). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. |
|
|
| The data was gathered at the [Mpala Research Centre](https://mpala.org/) in Kenya, in accordance with Research License No. NACOSTI/P/22/18214. The data collection protocol adhered strictly to the guidelines set forth by the Institutional Animal Care and Use Committee under permission No. IACUC 1835F. |
|
|
|
|
| ## Model Card Authors |
|
|
| Maksim Kholiavchenko |
|
|
| ## Model Card Contact |
|
|
| For questions on this model, please open a [discussion](https://huggingface.co/imageomics/x3d-BaboonLand/discussions) on this repo. |
