afeefaazam/tomato-ripeness-detector

Silal Tomato Ripeness Detector + Yield Estimator

YOLO11-based object detector that finds individual tomatoes in a photo or video and grades each one into a ripeness class, plus a yield-estimation layer on top (fruit-count → weight, with overlap-aware de-duplication for video). Trained on a combined drone + handheld-mobile tomato dataset, structured after the Blueberry-Ripeness-Dataset project.

Ripeness classes

Per the Silal "Quality Standard Guide For Tomato":

#	Class	Definition
1	Green	100% green
2	Breakers	less than 10% colour
3	Turning	10–40% colour
4	Pink	40–80% colour
5	Light Red	80–90% red
6	Red	more than 90% red

Training data

• Drone (SET_2): 142 frames, 1280×720, DJI aerial capture.
• Mobile (Set1_535): 535 photos, 7680×4320, handheld capture.
• Combined into one YOLO-format dataset (479 train / 100 val / 98 test images, ~14,300 instances) and trained as a single detector so the model generalizes across both aerial and close-up capture conditions rather than needing two separate deployments.

Model

• Architecture: YOLO11s (Ultralytics), 1024×1024 input, 9.4M params.

• Trained 136 epochs (early-stopped, patience 30; best weights from epoch 106) on the combined drone+mobile set: 479 train / 100 val / 98 test images.

• Held-out test-set accuracy:

  Subset	Images	Precision	Recall	mAP50	mAP50-95
  Combined (drone+mobile)	98	0.581	0.498	0.507	0.388
  Drone-only	22	0.421	0.691	0.535	0.317
  Mobile-only	76	0.509	0.503	0.482	0.386

  Per-class (combined test set): Green mAP50 0.750, Breakers 0.281, Turning 0.535, Pink 0.320, Light Red 0.732, Red 0.425 — minority classes (Breakers, Red, Pink) are weaker due to fewer training instances; see the full technical report for details.

• Video overlap test: on a reconstructed 60-frame flythrough clip, naive per-frame detection counting overcounted fruit by 2.67x (mobile, dense frame overlap) / 1.47x (drone, sparser overlap) vs. the track-ID-deduplicated count this model's video pipeline actually reports.

Yield estimation

yield_estimation.py converts detected/tracked fruit counts into an estimated kg figure using a configurable average-fruit-weight table (DEFAULT_AVG_FRUIT_WEIGHT_G, default 120 g/fruit for all classes — tomatoes reach most of their final size by the Breakers stage, so ripeness itself changes weight only modestly). This default is not calibrated to any specific farm/cultivar — before trusting an absolute kg number in production, weigh a sample of counted fruit and adjust the table accordingly.

For video, per-frame detection counts alone overcount yield whenever consecutive frames overlap in field of view (true here — the mobile capture is a fixed 10-frame stride from a source video, i.e. consecutive dataset frames are ~0.3s apart with a slowly panning camera). The video pipeline uses Ultralytics' ByteTrack integration (model.track(..., persist=True)) to assign a persistent track ID to each physical fruit, then counts unique track IDs, not raw per-frame detections. The demo app's Video tab reports both numbers side by side so you can see the overcount ratio directly.

Usage

from ultralytics import YOLO
model = YOLO("model.pt")
result = model.predict("photo.jpg", conf=0.25, imgsz=1024)[0]
result.show()  # or result.plot() for a numpy array

from yield_estimation import counts_from_detections, estimate_yield_from_counts
class_names = [result.names[int(c)] for c in result.boxes.cls.tolist()]
counts = counts_from_detections(class_names)
print(estimate_yield_from_counts(counts).as_dict())

For video with overlap-aware counting, see app.py's run_video function (uses model.track(..., persist=True, tracker="bytetrack.yaml") + summarize_tracked_video).