README.md

metadata

title: Oculus UI Element Detector
emoji: 🔍
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 4.21.0
app_file: app.py
pinned: false
license: mit

Oculus UI Element Detector

This application detects UI elements in screenshots and annotates them with bounding boxes and text labels.

Features

• Text detection using OCR
• UI element detection using YOLOv8
• Element code assignment
• Image annotation with bounding boxes
• Arrow annotations for clarity

Usage

1. Upload Screenshot

   • Upload a screenshot to detect UI elements
   • The app will process the image and return an annotated version with detected elements

2. Use Data URI

   • Alternatively, paste an image data URI
   • Useful for integration with other applications

API Access

This app also exposes an API for programmatic access:

import gradio as gr

# Connect to the Space
client = gr.Client("https://shanurrahman-oculus-ui-detector.hf.space")

# Process an image
result = client.predict(
    "data:image/png;base64,...",  # your image data URI
    api_name="/process_image_data_uri"
)

Output Format

The app returns:

1. An annotated image with bounding boxes
2. A text representation of detected elements with their positions and content

Technologies Used

• EasyOCR for text detection
• YOLOv8 for object detection
• Gradio for the user interface
• FastAPI for the backend API