Spaces:
Runtime error
Runtime error
| title: Prometheus Prototype | |
| emoji: 🔥 | |
| colorFrom: yellow | |
| colorTo: gray | |
| sdk: docker | |
| app_port: 8080 | |
| pinned: false | |
| short_description: launching my prometheus, aerial wildlife image detection pro | |
| # Prometheus — Aerial Wildlife Intelligence | |
| Automated wildlife detection and population counting from aerial drone footage, built for the **Malilangwe Wildlife Trust** — a conservation organisation managing one of Zimbabwe's most biodiverse reserves (~98,000 acres, southeastern Zimbabwe). | |
| **Prometheus** processes drone and helicopter survey footage using YOLOv11 object detection and BoT-SORT multi-object tracking, surfacing species counts, herd locations, and anomaly flags through an interactive web dashboard. | |
| --- | |
| ## The Problem | |
| Wildlife population monitoring across large terrain is costly, slow, and error-prone when done manually. Rangers walk transects or scan footage by hand — a process that takes days and introduces counting errors. Malilangwe monitors 19 priority species across terrain that is difficult to survey on foot. | |
| ## The Solution | |
| An end-to-end ML pipeline that processes aerial imagery to **detect**, **classify**, and **count** wildlife in real time. | |
| | | Pretrained (COCO) | Phase A (WAID) | Phase A+ (Merged) | | |
| |---|---|---|---| | |
| | Aerial elephants | "sheep", "bird" | not in schema | ✓ elephant | | |
| | Aerial buffalo | "horse", "cow" | not in schema | ✓ buffalo | | |
| | Aerial zebra | "horse" | ✓ zebra | ✓ zebra | | |
| | mAP50 | ~0.05 | **0.956** (yolo11s — up from 0.918 with yolo11n; see comparison below) | 0.317 and climbing | | |
| --- | |
| ## Project Structure | |
| ``` | |
| wildlife-detector-malilangwe/ | |
| ├── app.py # Prometheus Streamlit dashboard (working app — run this) | |
| ├── dashboard_mockup.html # Static UI design prototype (Leaflet.js, not connected to model) | |
| ├── api/ # FastAPI service backing the React dashboard (reuses src/detection) | |
| │ ├── main.py # App, CORS, /api/health | |
| │ ├── routes/ # detection + catalog (models/classes/capabilities/metrics) | |
| │ ├── services/ # detection_service — thin layer over Detector | |
| │ └── data/ # capabilities + training-run history (JSON) | |
| ├── dashboard/ # React + TS + Tailwind console (Vite) — see dashboard/README.md | |
| ├── config/ | |
| │ ├── default.yaml # Master config (paths, model, training, tracking) | |
| │ └── merged_classes.yaml # Unified 7-class schema + per-dataset remappings | |
| ├── data/ | |
| │ ├── waid.yaml # Ultralytics dataset YAML (WAID only) | |
| │ └── merged.yaml # Ultralytics dataset YAML (multi-dataset, generated) | |
| ├── src/ | |
| │ ├── config.py # YAML config loader with dot-access & deep merge | |
| │ ├── detection/ | |
| │ │ └── detector.py # YOLOv11 wrapper with config-driven inference | |
| │ ├── tracking/ | |
| │ │ └── tracker.py # BoT-SORT multi-object tracking interface | |
| │ ├── data/ | |
| │ │ ├── dataset.py # WAID validation & YAML generation | |
| │ │ ├── merge.py # Class remapping & multi-dataset merge utilities | |
| │ │ └── convert_aed.py # AED point-annotation CSV → YOLO format converter | |
| │ └── utils/ | |
| │ ├── logging_setup.py # Structured logging (file + stdout) | |
| │ └── visualization.py # Bounding box & summary overlay drawing | |
| ├── scripts/ | |
| │ ├── detect.py # CLI detection runner | |
| │ ├── train.py # Fine-tuning script (WAID or merged dataset) | |
| │ ├── evaluate.py # Evaluation — mAP, precision, recall | |
| │ ├── merge_datasets.py # Merge multiple datasets into unified format | |
| │ └── prepare_datasets.py # Download + structure guide for all datasets | |
| ├── weights/ # Trained model weights (not tracked — too large) | |
| ├── WAID/ # WAID labels (images not tracked) | |
| ├── requirements.txt | |
| ├── pyproject.toml | |
| ├── STATUS.md # Current training state and roadmap | |
| └── BACKLOG.md # Future scope outside current MVP | |
| ``` | |
| --- | |
| ## Quick Start | |
| ### Prerequisites | |
| - Python 3.10+ | |
| - GPU recommended for training (Google Colab T4 works well) | |
| ### Installation | |
| ```bash | |
| git clone https://github.com/Tadiwa-M/wildlife-detector-malilangwe.git | |
| cd wildlife-detector-malilangwe | |
| pip install -r requirements.txt | |
| ``` | |
| ### Launch Dashboard | |
| ```bash | |
| streamlit run app.py | |
| ``` | |
| Opens at `http://localhost:8501`. Upload an aerial image or video clip, select weights, and run detection. Supports drag-and-drop, side-by-side original vs annotated view, density heatmap, species count cards, and download of annotated results. | |
| ### Run Detection (CLI) | |
| ```bash | |
| # Single image | |
| python scripts/detect.py --source path/to/image.jpg --show | |
| # Video file | |
| yolo predict model=weights/best.pt source=path/to/clip.mp4 conf=0.20 save=True | |
| # Save annotated output | |
| python scripts/detect.py --source path/to/image.jpg --save --conf 0.20 | |
| ``` | |
| ### Multi-Dataset Training (Phase A+) | |
| ```bash | |
| # 1. See download instructions for each dataset | |
| python scripts/prepare_datasets.py | |
| # 2. Merge datasets into unified format | |
| python scripts/merge_datasets.py \ | |
| --waid WAID/WAID \ | |
| --aed path/to/AED \ | |
| --liege path/to/liege_yolo | |
| # 3. Train on merged dataset | |
| python scripts/train.py \ | |
| --dataset data/merged.yaml \ | |
| --base-weights weights/best.pt | |
| ``` | |
| ### Train on WAID Only (Phase A) | |
| ```bash | |
| git clone https://github.com/xiaohuicui/WAID.git | |
| python scripts/train.py # fresh run | |
| python scripts/train.py --resume runs/train/weights/last.pt # resume | |
| python scripts/train.py --validate-only # check dataset | |
| ``` | |
| --- | |
| ## Datasets | |
| ### Unified Class Schema (7 classes) | |
| | ID | Class | Sources | | |
| |----|-------|---------| | |
| | 0 | elephant | AED, Liege | | |
| | 1 | zebra | WAID, Liege | | |
| | 2 | buffalo | Liege | | |
| | 3 | antelope | Liege (kob, topi, waterbuck, alcelaphinae) | | |
| | 4 | cattle | WAID | | |
| | 5 | giraffe | Liege | | |
| | 6 | other | Liege (warthog) | | |
| ### Dataset Sources | |
| | Dataset | Species | Images | Format | Status | | |
| |---------|---------|--------|--------|--------| | |
| | [WAID](https://github.com/xiaohuicui/WAID) | sheep, cattle, seal, camelus, kiang, zebra | ~14,000 | YOLO | ✓ merged | | |
| | [AED](https://zenodo.org/record/3234780) | elephant | ~2,100 | CSV point annotations → auto-converted | ✓ merged | | |
| | [Liege African Mammals](https://dataverse.uliege.be) | elephant, zebra, buffalo, kob, topi, warthog, waterbuck | ~1,300 | COCO JSON → auto-converted | ✓ merged | | |
| | [WildlifeMapper](https://data.4tu.nl/articles/dataset/12713903/1) | 20 species incl. lion, giraffe | TBC | YOLO | optional | | |
| | [MMLA](https://arxiv.org/pdf/2504.07744) | 6 species, 37 aerial videos | 811K annotations | YOLO | optional | | |
| > The merge pipeline handles all format conversions automatically — no manual pre-processing needed. | |
| --- | |
| ## Architecture | |
| - **Detection:** YOLOv11n (2.6M params, 6.4 GFLOPs) — real-time, CPU-deployable | |
| - **Tracking:** BoT-SORT — multi-object tracking for counting individuals across frames | |
| - **Merge pipeline:** Unified class remapping across 5 datasets with different annotation formats | |
| - **Dashboard:** Streamlit — dark earth-tone UI, drag-and-drop upload, density heatmap, species cards | |
| - **Config:** YAML-driven — no hardcoded paths or hyperparameters | |
| --- | |
| ## Training | |
| Training runs on Google Colab T4 GPU. The one-shot Colab cell handles the full pipeline: extract → convert → merge → train → save to Drive. | |
| ### Phase A: yolo11n → yolo11s | |
| Re-running Phase A on a larger model with multi-GPU DDP produced a clear jump in both detection quality and training speed: | |
| | | yolo11n (Colab, 1x T4) | yolo11s (Kaggle, 2x T4 DDP) | | |
| |---|---|---| | |
| | **mAP50** | 0.918 | **0.956** | | |
| | **mAP50-95** | 0.552 | **0.578** | | |
| | Precision / Recall | — | 0.946 / 0.912 | | |
| | Image size | — | 1024 | | |
| | Epochs | 120 | 120 (~45s/epoch — full run in ~90 min) | | |
| > Two changes compounded here: moving to the larger **yolo11s** backbone, and training on **2x T4 with `device=[0,1]`** (Ultralytics defaults to a single GPU otherwise — DDP splits the batch automatically). Together they delivered both a stronger model *and* a dramatically shorter wall-clock time versus the single-GPU Colab runs — worth digging into further for the discussion section of a potential write-up (effect of multi-GPU DDP, and Kaggle vs. Colab as a training platform). | |
| > | |
| > Note: the yolo11s run evaluated on a different validation split (a self-carved 15% holdout from a broader 10-class dataset, 292 images) than the yolo11n run (the canonical WAID val set, 2,873 images) — see the per-run breakdowns and reproducibility note below for the full picture. | |
| **Phase A (WAID baseline):** 120 epochs, yolo11n. Evaluated on WAID validation set (2,873 images, 46,703 instances): **mAP50 = 0.918, mAP50-95 = 0.552**. Per-class mAP50: seal 0.983, cattle 0.964, sheep 0.972, zebra 0.878, kiang 0.881, camelus 0.832. | |
| **Phase A revised (yolo11s, Kaggle T4 x2):** Re-ran the WAID fine-tune on a larger, 10-class aerial-wildlife variant (sheep, cattle, seal, camelus, kiang, zebra, crocodile, elephant, deer, horse), 120 epochs at imgsz=1024, with a self-carved 15% validation holdout (292 images, 3,035 instances — not the original WAID val split, so not a direct apples-to-apples comparison with the yolo11n run above, but a stronger result on its own terms). Final: **mAP50 = 0.956, mAP50-95 = 0.578** (P = 0.946, R = 0.912). Per-class mAP50: sheep 0.978, cattle 0.968, seal 0.935, camelus 0.972, kiang 0.947, zebra 0.956, crocodile 0.928, elephant 0.952, deer 0.971, horse 0.955. This checkpoint (`waid_yolo11s_best.pt`) becomes the starting point for the merged Phase A+ run. | |
| > **Note for reproducibility:** the "10-class aerial-wildlife variant" used for the revised run is a *different* Kaggle dataset from the canonical 6-class WAID (sheep, cattle, seal, camelus, kiang, zebra) referenced in `config/merged_classes.yaml` and used for the original yolo11n baseline above — it happens to share the WAID name and most of its classes, plus four extra ones (crocodile, elephant, deer, horse). Of the canonical WAID's 6 classes, only `cattle` and `zebra` survive into the unified Prometheus schema (the other four — sheep, seal, camelus, kiang — are dropped as not African/not Malilangwe-relevant; see `dataset_mappings.waid` in `config/merged_classes.yaml`). The revised run's checkpoint is used purely as a *warm-start* for Phase A+, not as a merge input, so this naming overlap doesn't affect the merge pipeline — but it's worth knowing which "WAID" produced which numbers when comparing results. | |
| **Phase A+ (multi-dataset):** Transfer learning from Phase A weights on merged 3,300-image dataset (WAID + AED elephant + Liege African mammals). 120 epochs, patience=20, full network fine-tuning (freeze=0). Currently training — best mAP50 reached **0.180** at epoch 22, with losses still declining. | |
| --- | |
| ## Roadmap | |
| **Phase A — WAID Baseline** | |
| - [x] Project structure, config system, detection module | |
| - [x] CLI scripts: detect, train, evaluate | |
| - [x] Colab training notebook with Drive backup | |
| - [x] 120-epoch WAID training complete | |
| **Phase A+ — Multi-Dataset (current)** | |
| - [x] Unified 7-class schema | |
| - [x] AED point-annotation CSV → YOLO converter | |
| - [x] Multi-dataset merge pipeline (WAID + AED + Liege + WildlifeMapper + MMLA) | |
| - [x] Prometheus Streamlit dashboard with video support | |
| - [x] Interactive map mockup (Leaflet.js, real Malilangwe coordinates) | |
| - [ ] Complete multi-dataset training run | |
| - [ ] SAHI tiled inference for dense herds at altitude | |
| **Phase 1 — Geolocation & Population Estimation** | |
| - [ ] Geolocation layer: project pixel detections to ground coordinates using drone GPS, altitude, and gimbal/attitude data (camera calibration + terrain awareness) | |
| - [ ] Distance-sampling estimation: per-detection perpendicular distance → detection-function fit (half-normal / hazard-rate) → population density estimates with confidence intervals (faithful implementation of Buckland et al.) | |
| - [ ] Population-density mapping across a park over time (built on the geolocation layer) | |
| **Phase 2 — FastAPI Backend** | |
| - [ ] Async video processing (Celery + Redis) | |
| - [ ] Postgres detection result storage | |
| - [ ] REST API: upload, poll, retrieve | |
| **Phase 3 — React Dashboard** | |
| - [ ] React + Tailwind + shadcn/ui | |
| - [ ] Real-time processing status | |
| - [ ] Annotated video playback, population charts, map view | |
| **Phase B — Malilangwe Fine-Tune** | |
| - [ ] Blocked on labelled imagery from Malilangwe Trust (contact made) | |
| - [ ] 19 priority species: elephant, black/white rhino, lion, leopard, buffalo, giraffe, and more | |
| --- | |
| ## About Malilangwe | |
| The [Malilangwe Trust](https://www.malilangwe.org/) manages the Malilangwe Wildlife Reserve in southeastern Zimbabwe (20°58′–21°15′S, 31°47′–32°01′E) — roughly 98,000 acres (≈396 km²) of protected savanna, woodland, and wetland. The reserve hosts one of Africa's most intact wildlife communities, including black and white rhino, lion, leopard, and large elephant and buffalo herds. | |
| --- | |
| ## References | |
| - [WAID: A Large-Scale Dataset for Wildlife Detection with Drones](https://www.mdpi.com/2076-3417/13/18/10397) — Cui et al., 2023 | |
| - [Aerial Elephant Dataset](https://zenodo.org/record/3234780) — Zenodo, 2019 | |
| - [WildlifeMapper: Aerial Image Analysis for Multi-Species Detection](https://arxiv.org/abs/2311.00880) — CVPR 2024 | |
| - [Ultralytics YOLOv11](https://docs.ultralytics.com/) | |
| - [BoT-SORT: Robust Associations Multi-Pedestrian Tracking](https://arxiv.org/abs/2206.14651) | |
| ## License | |
| MIT | |