README.md

metadata

title: SightLine
emoji: 👁️
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 5.50.0
app_file: app.py
pinned: false

SightLine: See the World Through AI

SightLine is a voice-first multimodal AI assistant designed primarily for blind and low-vision users.

Features

• Real-Time Scene Understanding: Continuously analyze the camera feed to detect people, vehicles, obstacles, etc.
• Voice-First Interaction: Hands-free interaction via voice commands (e.g. "What do you see?", "Read the sign").
• Smart Scene Change Detection: Triggers descriptions only when meaningful changes occur in the scene.
• Conversational Context: Remembers recent objects and OCR text to provide natural context-aware responses.
• Accessibility First: High contrast, ARIA labels, screen reader support, large controls.

Project Structure

sightline/
├── app/
│   ├── main.py
│   ├── config.py
│   ├── vision/
│   ├── speech/
│   ├── conversation/
│   ├── camera/
│   ├── accessibility/
│   └── ui/
├── Dockerfile
├── requirements.txt
└── README.md

Running Locally

1. Create a virtual environment and install dependencies:

   python -m venv venv
   source venv/bin/activate
   pip install -r requirements.txt
   pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 --upgrade --force-reinstall
   

2. Start the application:

   python -m app.main
   

3. Open the UI at http://localhost:7860.

Docker Deployment

docker build -t sightline .
docker run -p 7860:7860 sightline

Privacy & Safety

SightLine does not permanently store camera frames or microphone recordings. It processes imagery purely to provide instant auditory feedback to the user. Note: SightLine is not a replacement for a cane, guide dog, caregiver, or professional mobility aid.