| | ---
|
| | license: gpl-3.0
|
| | tags:
|
| | - ultralytics
|
| | - yolo
|
| | - object-detection
|
| | - ui-detection
|
| | - computer-vision
|
| | - agent
|
| | ---
|
| |
|
| | # deki-yolo: Mobile UI Element Detection Model
|
| |
|
| | This is a YOLO model trained to identify common UI elements in mobile
|
| | screenshots. It is the core detection model for the [deki huggingface space](https://huggingface.co/spaces/orasul/deki)
|
| | or [deki github](https://github.com/RasulOs/deki)
|
| |
|
| | ## Model Description
|
| |
|
| | The model is trained to detect the following four classes of UI elements:
|
| | * `View`: General-purpose containers.
|
| | * `ImageView`: Icons and images.
|
| | * `Text`: Text elements.
|
| | * `Line`: Separators and lines.
|
| |
|
| | This model can be used as a foundational component for applications that need
|
| | to understand screen layouts, such as AI agents for mobile automation,
|
| | accessibility tools, and UI code generation.
|
| |
|
| | ---
|
| |
|
| | ## YOLO examples
|
| |
|
| | Bounding boxes with classes for bb_1:
|
| |
|
| | <img src="res/bb_1_yolo.jpeg" alt="example1" width="60%">
|
| |
|
| | Bounding boxes without classes but with IDs after NMS for bb_1:
|
| |
|
| | <img src="res/bb_1_yolo_updated.jpeg" alt="example2" width="60%">
|
| |
|
| | Bounding boxes with classes for bb_2:
|
| |
|
| | <img src="res/bb_2_yolo.jpeg" alt="example3" width="60%">
|
| |
|
| | Bounding boxes without classes but with IDs after NMS for bb_2:
|
| |
|
| | <img src="res/bb_2_yolo_updated.jpeg" alt="example4" width="60%">
|
| |
|
| | ---
|
| |
|
| | ## YOLO model accuracy
|
| |
|
| | The model was trained on 486 images and was tested on 60 images.
|
| |
|
| | Current YOLO model accuracy:
|
| | 
|
| |
|
| |
|
