README.md

---
title: Sign Language Detector Pro
emoji: 🚀
colorFrom: red
colorTo: yellow
sdk: streamlit
app_file: src/streamlit_app.py
app_port: 8501
tags:
- streamlit
pinned: false
short_description: Streamlit template space
license: mit
---

# Sign Language Detector Pro

An advanced Python application for detecting and interpreting sign language gestures from images and videos. Features cutting-edge computer vision using MediaPipe for hand landmark detection, AI-powered gesture classification via OpenAI API, and a modern web interface for professional analysis and reporting.

## ✨ Enhanced Features

### 🎯 Core Functionality
- **Advanced Hand Detection**: MediaPipe-powered 21-point hand landmark detection
- **AI Gesture Classification**: OpenAI API integration for accurate sign language interpretation
- **Batch File Processing**: Support for multiple images and videos simultaneously
- **Professional Analytics**: Interactive charts, confidence metrics, and detailed analysis

### 🎨 Modern Web Interface
- **Professional Design**: Modern, responsive UI with gradient themes and animations
- **Interactive Visualizations**: 3D hand landmark plots, confidence charts, and timeline analysis
- **Multiple Export Formats**: JSON, CSV, and PDF report generation
- **Real-time Progress Tracking**: Enhanced progress indicators and status updates

### 📊 Advanced Analytics
- **Confidence Scoring**: Detailed confidence metrics for all detections
- **3D Visualization**: Interactive 3D plots of hand landmarks
- **Timeline Analysis**: Frame-by-frame video processing with visual timelines
- **Comparison Views**: Side-by-side before/after image comparisons

## Setup

1. Clone the repository
2. Create a virtual environment:
   ```bash
   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate
   ```

3. Install dependencies:
   ```bash
   pip install -r requirements.txt
   ```

4. Set up environment variables:
   ```bash
   cp .env.example .env
   # Edit .env and add your OpenAI API key
   ```

## Usage

### Enhanced Command Line Interface
```bash
# File processing mode (camera functionality removed)
python3 main.py --input path/to/video.mp4

# Batch processing with output directory
python3 main.py --input path/to/directory --output results/

# Disable speech output
python3 main.py --input path/to/image.jpg --no-speech
```

### Professional Web Interface
```bash
streamlit run app.py
```
**Features:**
- Drag-and-drop file upload
- Batch processing with progress tracking
- Interactive 3D visualizations
- Multiple export formats (JSON, CSV, PDF)
- Real-time analytics dashboard

### Demo Mode (No API Key Required)
```bash
python3 demo.py
```

## Project Structure

- `main.py` - Main application entry point
- `app.py` - Streamlit GUI application
- `src/` - Source code modules
  - `hand_detector.py` - Hand landmark detection
  - `gesture_extractor.py` - Gesture feature extraction
  - `openai_classifier.py` - OpenAI API integration
  - `camera_handler.py` - Real-time camera processing
  - `file_handler.py` - File input processing
  - `output_handler.py` - Text and speech output
- `tests/` - Unit tests
- `examples/` - Example videos and images

## Requirements

- Python 3.8+
- OpenAI API key (for gesture classification)
- Webcam (for real-time mode)

## Quick Start

1. **Test without API key (Demo mode):**
   ```bash
   python3 demo.py
   ```
   This will show hand detection and gesture analysis without requiring an OpenAI API key.

2. **Set up OpenAI API key:**
   ```bash
   cp .env.example .env
   # Edit .env and add: OPENAI_API_KEY=your_key_here
   ```

3. **Run real-time detection:**
   ```bash
   python3 main.py --mode realtime
   ```

4. **Process a video file:**
   ```bash
   python3 main.py --mode file --input examples/sample_video.mp4
   ```

5. **Launch web interface:**
   ```bash
   streamlit run app.py
   ```

## 🚀 Enhanced Features Delivered

### ✅ Core Processing
- **Advanced Hand Detection** - MediaPipe 21-point landmark detection with enhanced visualization
- **AI-Powered Classification** - OpenAI API integration with confidence scoring
- **Batch File Processing** - Simultaneous processing of multiple images and videos
- **Professional Analytics** - Comprehensive metrics and statistical analysis

### ✅ Modern Web Interface
- **Responsive Design** - Professional UI with gradient themes and animations
- **Interactive Visualizations** - 3D hand plots, confidence charts, timeline analysis
- **Multiple Export Formats** - JSON, CSV, and PDF report generation
- **Real-time Progress** - Enhanced progress tracking with detailed status updates

### ✅ Advanced Analytics
- **3D Visualization** - Interactive 3D hand landmark plots
- **Timeline Analysis** - Frame-by-frame video processing visualization
- **Confidence Metrics** - Detailed confidence scoring and analysis
- **Comparison Views** - Side-by-side before/after image comparisons
- **Summary Reports** - Comprehensive processing statistics and insights

### ✅ User Experience
- **Drag-and-Drop Upload** - Intuitive file upload with visual feedback
- **Settings Panel** - Configurable detection parameters
- **Error Handling** - User-friendly error messages and recovery
- **Export Functionality** - Multiple format options for results