AbhiCommits/cropguard-models

CropGuard - Crop Disease Classifier (38 classes)

ResNet50 fine-tuned on PlantVillage, exported to ONNX and dynamically quantised to INT8 for CPU-only serving. Part of CropGuard, an end-to-end MLOps pipeline.

Results (held-out test set, n=8,125)

Model	Accuracy	Macro-F1	Note
fp32	0.9911	0.9865	reference
INT8 (dynamic)	not evaluated	not evaluated	not served - see below

Macro-F1 is the metric to read here, not accuracy: the dataset is imbalanced ~36x, so accuracy is dominated by the largest classes.

cropguard.onnx (fp32) is the model that serves traffic. cropguard.static-int8.onnx is published alongside it for comparison; the dynamic INT8 build is deliberately not published.

Dynamic quantization was the wrong tool here. quantize_dynamic rewrites every Conv into ConvInteger, which ONNX Runtime's CPU backend has no optimized kernel for: measured on an Intel Alder Lake CPU it ran at 1567 ms/image against 19 ms for fp32, a 75x regression in exchange for 4x less disk. It suits MatMul-dominated models (Transformers, RNNs), not CNNs.

Static quantization with a calibration set emits the optimized QLinearConv instead, and is 78 ms/image - 20x faster than dynamic and accuracy-neutral (0.9897 vs 0.9893 on a 3,000 image subset, 9 disagreements). It is still 3.2x slower than fp32 on the machine it was measured on, and the reason is hardware rather than the graph: that CPU has no VNNI instructions, without which ONNX Runtime emulates each INT8 multiply-accumulate in several AVX2 instructions. Server CPUs usually do have VNNI, where the ranking may reverse - which is exactly why both files are here to be benchmarked on whatever hardware you are running.

The split is grouped by leaf, and that matters

PlantVillage contains 54,305 images of only ~7,600 distinct physical leaves - roughly 7 photographs of each. A standard per-image stratified split scatters those near-duplicates across train and test, so a model can score well by memorising leaf identity rather than learning disease morphology. Measured on this dataset, a naive split leaves 74.2% of test images sharing a leaf with training.

This model was trained on a leaf-grouped split instead:

	Naive stratified	Grouped (used here)
test images sharing a leaf with train	74.2%	0.0%
train / val / test	-	38,008 / 8,172 / 8,125

So the accuracy above is measured on a holdout with no leaf overlap. Note that it is not much lower than typically published PlantVillage figures - the honest reading is that this dataset is genuinely easy, not that leakage was inflating everything.

Intended use

Identifying disease on single leaves photographed against a plain background, matching the PlantVillage capture protocol.

Limitations - read before deploying this

• Lab images, not field images. Every training image is a detached leaf on a uniform background under controlled lighting. Real photographs from a farm - variable lighting, occlusion, multiple leaves, soil backgrounds - are a different distribution, and published work on this dataset reports large drops there. This model has not been evaluated on field photographs.
• Per-class metrics for rare classes are noisy. Eight classes have fewer than 100 test images; the smallest (Potato___healthy) has 24. A recall of 0.833 there is 4 mistakes, and its confidence interval spans roughly +/-15 points. Do not read those per-class numbers as precise.
• The raw softmax is not calibrated. Training used label smoothing (0.1), which caps achievable confidence at (1-eps) + eps/K = 0.9026 and leaves the model systematically under-confident. Temperature scaling fitted on the validation split (T = 0.591) cuts expected calibration error from 0.0895 to 0.0036 and is applied in the serving path, but the ONNX graph published here emits logits - apply the temperature yourself, or the probabilities you compute from it will understate.
• uncertainty is predictive entropy, not epistemic uncertainty. It cannot distinguish an ambiguous input from one far outside the training distribution - an out-of-distribution image can produce confidently wrong output with low entropy.
• 38 classes across 14 crops only. Anything outside that set is silently forced into one of them.

Training

ResNet50 (timm, ImageNet-pretrained), 224x224, batch 64, AdamW (lr 3e-4, weight decay 1e-4), cosine schedule, label smoothing 0.1, medium augmentation, 12 epochs, mixed precision. Checkpoint selected on val_f1_macro.

Usage

import numpy as np, onnxruntime as ort
from huggingface_hub import hf_hub_download

path = hf_hub_download("XiElonMAsk/cropguard-models", "cropguard.onnx")
session = ort.InferenceSession(path, providers=["CPUExecutionProvider"])

# Preprocessing must match training: resize short side to 256, centre-crop 224,
# scale to [0,1], normalise with ImageNet mean/std, NCHW.
logits = session.run(["logits"], {"input": batch})[0]

cropguard.serving.model_loader in the repo implements exactly that preprocessing.

Citation

Dataset: Mohanty, Hughes & Salathe (2016), Using deep learning for image-based plant disease detection, Frontiers in Plant Science.