| --- |
| title: VQA |
| emoji: 🚀 |
| colorFrom: gray |
| colorTo: yellow |
| sdk: docker |
| pinned: false |
| license: mit |
| short_description: VQA API Endpoint |
| --- |
| |
| Check out the configuration reference at |
| https://huggingface.co/docs/hub/spaces-config-reference |
|
|
| # VizWiz Visual Question Answering API |
|
|
| This repository contains a FastAPI backend for a Visual Question Answering (VQA) |
| system trained on the VizWiz dataset. |
|
|
| ## Features |
|
|
| - Upload images and ask questions about them |
| - Get answers with confidence scores |
| - Session management for asking multiple questions about the same image |
| - Health check endpoint for monitoring |
| - API documentation with Swagger UI |
|
|
| ## Project Structure |
|
|
| ``` |
| project_root/ |
| ├── app/ |
| │ ├── main.py # Main FastAPI application |
| │ ├── models/ # Model definitions |
| │ │ ├── __init__.py |
| │ │ └── vqa_model.py # VQA model implementation |
| │ ├── routers/ # API route definitions |
| │ │ ├── __init__.py |
| │ │ └── vqa.py # VQA-related endpoints |
| │ ├── services/ # Business logic |
| │ │ ├── __init__.py |
| │ │ ├── model_service.py # Model loading and inference |
| │ │ └── session_service.py # Session management |
| │ ├── utils/ # Utility functions |
| │ │ ├── __init__.py |
| │ │ └── image_utils.py # Image processing utilities |
| │ └── config.py # Application configuration |
| ├── models/ # Directory for model files |
| ├── uploads/ # Directory for uploaded images |
| ├── .env # Environment variables |
| └── requirements.txt # Project dependencies |
| ``` |
|
|
| ## Installation |
|
|
| 1. Clone the repository: |
|
|
| ```bash |
| git clone https://github.com/dixisouls/vizwiz-vqa-api.git |
| cd vizwiz-vqa-api |
| ``` |
|
|
| 2. Create a virtual environment: |
|
|
| ```bash |
| python -m venv venv |
| source venv/bin/activate # On Windows: venv\Scripts\activate |
| ``` |
|
|
| 3. Install dependencies: |
|
|
| ```bash |
| pip install -r requirements.txt |
| ``` |
|
|
| 4. Create necessary directories: |
|
|
| ```bash |
| mkdir -p models uploads |
| ``` |
|
|
| 5. Place your trained model in the `models` directory. |
|
|
| 6. Update the `.env` file with your configuration. |
|
|
| ## Running the Application |
|
|
| ```bash |
| uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload |
| ``` |
|
|
| The API will be available at http://localhost:8000. |
|
|
| API documentation is available at: |
|
|
| - Swagger UI: http://localhost:8000/docs |
| - ReDoc: http://localhost:8000/redoc |
|
|
| ## API Endpoints |
|
|
| ### Health Check |
|
|
| ``` |
| GET /health |
| ``` |
|
|
| Returns the health status of the API. |
|
|
| ### Upload Image |
|
|
| ``` |
| POST /api/vqa/upload |
| ``` |
|
|
| Upload an image and create a new session. |
|
|
| ### Ask Question |
|
|
| ``` |
| POST /api/vqa/ask |
| ``` |
|
|
| Ask a question about an uploaded image. |
|
|
| ### Get Session |
|
|
| ``` |
| GET /api/vqa/session/{session_id} |
| ``` |
|
|
| Get session information including question history. |
|
|
| ### Reset Session |
|
|
| ``` |
| DELETE /api/vqa/session/{session_id} |
| ``` |
|
|
| Reset a session to start fresh. |
|
|
| ## Environment Variables |
|
|
| - `DEBUG`: Enable debug mode (default: False) |
| - `MODEL_PATH`: Path to the trained model (default: ./models/vqa_model_best.pt) |
| - `TEXT_MODEL`: Name of the text model (default: bert-base-uncased) |
| - `VISION_MODEL`: Name of the vision model (default: |
| google/vit-base-patch16-384) |
| - `HUGGINGFACE_TOKEN`: Hugging Face API token |
| - `UPLOAD_DIR`: Directory for uploaded images (default: ./uploads) |
|
|
| ## License |
|
|
| [MIT License](LICENSE) |
|
|
