ColaRob/wildlife-vision-coreml

Wildlife Vision (Core ML)

Three models, converted to Apple Core ML format, that find animals in photos and video frames and identify their species. They run on a Mac (GPU and Neural Engine) with no network connection. They were converted for the Reef ID app.

What they do

For each photo or video frame:

1. Scene check. Decides whether the picture is underwater or on land, and whether it contains wildlife at all.
2. Detection. Finds the animals and gives each a box.
3. Identification. Names the species in each box, from a species list the app supplies.

Job	Model	Files
Identification	BioCLIP 2	bioclip_image_b1_fp32, bioclip_image_b8_fp32, bioclip_text_b64_fp32
Detection	OWLv2 (base, patch 16, ensemble)	owlv2_vision_fp32, owlv2_text_fp32
Scene check	CLIP ViT-B/32 (LAION-2B)	scene_image_b1_fp32, scene_image_b8_fp32

The files are seven .mlpackage folders, 4.25 GB in total, at full (32-bit) precision so results match the original models closely. b1 and b8 are batch sizes of 1 and 8 images. model-manifest.json lists every file with its size and SHA-256 checksum.

Using them

• Reef ID: Analysis, then Models for this app, then Download. Address: huggingface.co/ColaRob/wildlife-vision-coreml. Downloads can be cancelled and resumed, and each file is checked against the manifest.
• Other apps: download the repository, verify the files against model-manifest.json, and load the packages with Core ML.

Nothing is sent anywhere when the models run.

Limits

• Results are predictions with a confidence score, and are often wrong for blurry, distant, partly hidden or unusual animals. Check anything that matters.
• The models choose only from the species list they are given. An animal not on the list gets the closest match or a low score.
• Accuracy is better for species that are widely photographed.
• Not suitable for safety decisions or for enforcement.
• English species names only.
• The scene model's authors describe it as a research model and say deployments should be tested first. Here it is used only to separate underwater from land and to skip photos with no wildlife, and it was tested on a small set of the maintainer's own footage.

Sources and licences

These are conversions of existing models. No further training was done. Each remains under its original licence. Please cite the original work if you use these in research.

Model	Source	Licence
BioCLIP 2	https://huggingface.co/imageomics/bioclip-2	MIT
OWLv2 base patch16 ensemble (Minderer, Gritsenko, Houlsby, Scaling Open-Vocabulary Object Detection, 2023, arXiv:2306.09683)	https://huggingface.co/google/owlv2-base-patch16-ensemble	Apache 2.0
CLIP ViT-B/32 laion2B-s34B-b79K (OpenCLIP; Radford et al., 2021; Ilharco et al., 2021)	https://huggingface.co/laion/CLIP-ViT-B-32-laion2B-s34B-b79K	MIT

MIT and Apache 2.0 permit use, sharing and modification, including in apps, provided the copyright and licence notices are kept. These files are modified (converted) copies. Licence texts: https://opensource.org/license/mit and https://www.apache.org/licenses/LICENSE-2.0. The training data for BioCLIP 2 and the CLIP model has its own terms, described on the original model pages and not changed by this repository. This is a summary, not legal advice.

Problems

If a file fails its checksum, download it again. If it keeps failing, open a discussion on this page.