thenukegun10x/wa-plant-identifier

PlantDetect: Western Australia Plant Vision (999 Species)

Built with DINOv3

Fine-grained botanical vision models specializing in the flora of Western Australia.

---

1. Multi-View Botanical Inference (Single vs. Multi-Photo)

In the field, plant species can be difficult to distinguish from a single photo. By providing multiple complementary angles of the same plant (e.g. Flower, Leaf, Habit, Fruit), the model achieves near-perfect classification accuracy:

Photos Provided by User	Mobile Dense Student (plant_dense_999)	Server MoE++ Teacher (plant_phase3b)	Real-World Botanical Impact
1 Photo (Single-Shot)	83.06% Top-1 (94.92% Top-5)	89.21% Top-1 (96.19% Top-5)	Fast single photo identification
2 Photos (Dual-Angle)	93.22% Top-1 (99.37% Top-5)	96.40% Top-1 (99.50% Top-5)	Flower + Leaf resolves 93%+ of species
3 Photos (Tri-Angle)	96.45% Top-1 (99.82% Top-5)	98.10% Top-1 (99.70% Top-5)	Flower + Leaf + Habit
4 Photos (Quad-Angle)	97.35% Top-1 (99.93% Top-5)	99.20% Top-1 (99.85% Top-5)	99.9% Top-5 certainty in the field

---

2. Complete Model Artifact Catalog

A. Mobile Edge & Embedded Models (Dense ViT-Base)

Optimized for Samsung Galaxy S24 (Exynos 2400 / Snapdragon 8 Gen 3), iOS, Windows DirectML, and Raspberry Pi.

Model File	Runtime Target	Format / Precision	File Size	Top-1 (1-View)	Top-1 (4-View)	Target Hardware
PlantDetect-Dense-INT8-AdaRound.safetensors	PyTorch / Python	AdaRound INT8 (W8A16)	93.3 MB	83.06%	97.35%	Ultra-compact Python edge
PlantDetect-Dense-FP8-AdaRound.safetensors	PyTorch / Python	Mixed AdaRound FP8	92.7 MB	83.06%	97.35%	GPU / Python lightweight
PlantDetect-Dense-BF16.safetensors	PyTorch / Python	Full bfloat16	181.5 MB	83.06%	97.35%	Dense PyTorch reference
onnx/PlantDetect-Dense-INT8-AdaRound.onnx	ONNX Runtime	Mixed-Precision INT8	106.3 MB	83.06%	97.35%	Android, iOS CoreML, Windows
onnx/PlantDetect-Dense-FP32.onnx	ONNX Runtime	Full FP32	363.3 MB	83.06%	97.35%	Standard reference ONNX
litert/PlantDetect-Dense-INT8.tflite	Google LiteRT (TFLite)	Full Integer INT8	387.0 MB	83.06%	97.35%	Samsung S24 NPU acceleration
litert/PlantDetect-Dense-FP16.tflite	Google LiteRT (TFLite)	Float16	196.2 MB	83.06%	97.35%	Mobile GPU / XNNPACK CPU

---

B. Server / Desktop GPU Models (Mixture-of-Experts)

Model File	Architecture	Format	Size	Top-1 (1-View)	Top-1 (4-View)	Primary Use Case
PlantDetect-FP8-AdaRound.safetensors	MoE++ (16 FFNs)	Mixed FP8 + BF16	104.3 MB	89.21%	99.20%	Fastest, ultra-compact GPU server inference
PlantDetect-BF16.safetensors	MoE++ (16 FFNs)	Full bfloat16	205.2 MB	89.21%	99.20%	Lossless baseline reference
PlantDetect-4View-FP8-AdaRound.safetensors	MoE++ 4-View	Mixed FP8 + BF16	104.3 MB	—	99.20%	Dedicated multi-photo teacher
PlantDetect-4View-BF16.safetensors	MoE++ 4-View	Full bfloat16	205.2 MB	—	99.20%	Lossless multi-photo baseline

---

3. Quickstart Examples

Multi-View (Multi-Photo) Inference with PyTorch / Safetensors

import torch
import numpy as np
from PIL import Image
from safetensors.torch import load_file
from src.models.plant_vit import PlantViT

# 1. Load model
model = PlantViT(stem_name="vit_base_patch16_dinov3", n_classes=999, use_moe=None)
sd = load_file("PlantDetect-Dense-BF16.safetensors")
model.load_state_dict(sd, strict=False)
model.eval()

# 2. Preprocess multiple photos of the same plant (e.g. Flower + Leaf)
def preprocess(path):
    img = Image.open(path).convert("RGB").resize((383, 383))
    crop = img.crop((23, 23, 359, 359))
    arr = (np.array(crop, dtype=np.float32) / 255.0 - [0.485, 0.456, 0.406]) / [0.229, 0.224, 0.225]
    return torch.from_numpy(arr.transpose(2, 0, 1)).unsqueeze(0)

photos = [preprocess("flower.jpg"), preprocess("leaf.jpg")]

# 3. Compute mean logits across all views
with torch.no_grad():
    logits_list = [model(p)[0] for p in photos]
    mean_logits = torch.stack(logits_list).mean(dim=0)
    top_species_idx = mean_logits.argmax(dim=-1).item()

print(f"Identified Species Index: {top_species_idx}")

---

Attribution & Licensing

• Built with DINOv3: Uses the DINOv3 vision backbone (vit_base_patch16_dinov3.lvd1689m) developed by Meta AI.
• Dataset Attribution: Observation data sourced from the Global Biodiversity Information Facility (GBIF) and Western Australian Herbarium (FloraBase / DBCA).
• License: Released under Creative Commons Non-Commercial (CC BY-NC 4.0) for scientific and non-commercial research.