behavior_backend/README.md

EmotiVid API

A modular FastAPI backend for video behavior and emotion analysis.

Overview

EmotiVid API is a powerful backend service that analyzes videos to detect emotions, facial expressions, body language, eye contact, and transcribe speech. It provides comprehensive analysis of the content using computer vision, machine learning, and natural language processing techniques.

Features

• Video Management: Upload, list, and retrieve video metadata
• Emotion Analysis: Detect emotions in video frames using deep learning models
• Eye Contact Analysis: Measure eye contact consistency and engagement
• Body Language Analysis: Analyze posture, gestures, and non-verbal cues
• Speech-to-Text: Transcribe speech in videos using Whisper
• AI Analysis: Process results using OpenAI and Groq for deeper insights
• Background Processing: Process videos asynchronously with status updates
• Annotated Video Generation: Generate videos with behavior annotations
• User Authentication: Secure API with JWT authentication
• API Key Authentication: Alternative authentication method for direct integrations

Project Structure

behavior_backend/
├── app/                      # Application code
│   ├── api/                  # API endpoints
│   │   ├── routes/           # Route definitions
│   ├── core/                 # Core application code
│   │   ├── config.py         # Configuration management
│   │   └── exceptions.py     # Custom exceptions
│   ├── db/                   # Database related code
│   │   ├── base.py           # Database setup
│   │   ├── models.py         # SQLAlchemy models
│   │   └── repositories/     # Database access layer
│   ├── models/               # Pydantic models for API
│   ├── services/             # Business logic
│   │   ├── video_service.py  # Video management service
│   │   └── processing/       # Processing services
│   │       ├── video_processor.py      # Main video processing pipeline
│   │       ├── emotion_analyzer.py     # Facial emotion analysis
│   │       ├── eye_contact_analyzer.py # Eye contact detection
│   │       ├── body_language_analyzer.py # Body language analysis
│   │       ├── speech_service.py       # Speech transcription
│   │       └── ai_analysis.py          # AI-powered insights
│   └── utils/                # Utility functions
├── static/                   # Static files
│   ├── uploads/              # Upload directory
│   └── results/              # Results directory
├── annotated_videos/         # Processed videos with annotations
├── temp_face_frames/         # Temporary storage for processed frames
├── logs/                     # Application logs
├── tests/                    # Test directory
├── .env                      # Environment variables
├── main.py                   # Application entry point
├── requirements.txt          # Production dependencies
├── requirements-dev.txt      # Development dependencies
├── run.sh                    # Production server script
└── start_server.sh           # Development server script

Prerequisites

• Python 3.9+
• FFmpeg (for video processing)
• GPU support (optional, for faster processing)

Installation

Option 1: Using Python Virtual Environment

1. Create and activate a virtual environment:

   # Windows
   python -m venv venv
   venv\Scripts\activate
   
   # Linux/Mac
   python -m venv venv
   source venv/bin/activate
   

2. Install dependencies:

   # For production
   pip install -r requirements.txt
   
   # For development
   pip install -r requirements-dev.txt
   

3. Create a .env file with the following variables:

   DATABASE_URL=sqlite:///app.db
   OPENAI_API_KEY=your_openai_api_key
   GROQ_API_KEY=your_groq_api_key
   SECRET_KEY=your_secret_key
   

Running the Application

Development Server

For development with hot-reload and debug features:

# Windows
venv\Scripts\activate
uvicorn main:app --reload

# Linux/Mac
source venv/bin/activate
./start_server.sh

This will start the development server with hot-reload enabled and make the API available at http://localhost:8000.

Production Server

For production deployment:

# Windows
venv\Scripts\activate
uvicorn main:app --workers 4 --host 0.0.0.0 --port 8000

# Linux/Mac
source venv/bin/activate
./run.sh

API Documentation

API documentation is available at:

• Swagger UI: http://localhost:8000/docs
• ReDoc: http://localhost:8000/redoc

Key Endpoints

• POST /api/v1/videos/upload: Upload a video
• GET /api/v1/videos: List all videos
• GET /api/v1/videos/{video_id}: Get video metadata
• POST /api/v1/processing/analyze/{video_id}: Process a video
• GET /api/v1/processing/status/{video_id}: Get processing status
• GET /api/v1/processing/results/{video_id}: Get processing results
• POST /api/v1/auth/login: User login
• POST /api/v1/auth/register: User registration
• GET /api/v1/users/me: Get current user
• POST /api/v1/videos/upload-and-process-direct: Upload and process a video with API key authentication

API Key Authentication

Some endpoints support API key authentication for direct integration with other systems. To use these endpoints:

1. Set the API_KEY environment variable or let it auto-generate
2. Include the API key in the X-API-Key header with your requests
3. Endpoints that support API key authentication are documented in the API docs

Example:

curl -X POST "http://localhost:8000/api/v1/videos/upload-and-process-direct" \
  -H "X-API-Key: your-api-key" \
  -H "accept: application/json" \
  -H "Content-Type: multipart/form-data" \
  -F "file=@video.mp4" \
  -F "frame_rate=35" \
  -F "backend=mediapipe"

Testing

Run tests with pytest:

pytest

Development Tools

The project includes several development tools:

• Black: Code formatting
• isort: Import sorting
• flake8: Code linting
• mypy: Type checking
• pytest: Testing framework

Environment Variables

Variable	Description	Default
DATABASE_URL	SQLite or PostgreSQL connection string	sqlite:///app.db
SECRET_KEY	JWT secret key	None
API_KEY	API key for direct endpoints	Auto-generated
OPENAI_API_KEY	OpenAI API key for analysis	None
GROQ_API_KEY	Groq API key for analysis	None
CORS_ORIGINS	Allowed CORS origins	["http://localhost:3000"]

License

This project is licensed under the MIT License.