Set up Redis

API_DOCUMENTATION.md

@@ -0,0 +1,936 @@
+# Enflow API Documentation
+
+This document provides details about all the endpoints available in the Enflow API.
+
+## Base URL
+
+For local development: `http://localhost:5000`
+For production: Your HuggingFace space URL
+
+## Authentication
+
+Most endpoints require authentication via JWT token. To authenticate:
+
+1. Get a token by calling the login endpoint
+2. Include the token in your requests in the `Authorization` header:
+   ```
+   Authorization: Bearer <your_jwt_token>
+   ```
+
+### Authentication Endpoints
+
+#### Login
+
+```
+POST /api/auth/login
+```
+
+**Request Body:**
+```json
+{
+  "email": "user@example.com",
+  "password": "your_password"
+}
+```
+
+**Response:**
+```json
+{
+  "message": "Login successful",
+  "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
+  "user": {
+    "_id": "60d21b4967d0d8992e610c85",
+    "email": "user@example.com",
+    "name": "John Doe",
+    "permissions": "Admin",
+    "position": "Officer",
+    "department_id": "60d21b4967d0d8992e610c85",
+    "logs": [],
+    "incidents": []
+  }
+}
+```
+
+#### Get Current User
+
+```
+GET /api/auth/me
+```
+
+**Response:**
+```json
+{
+  "user": {
+    "_id": "60d21b4967d0d8992e610c85",
+    "email": "user@example.com",
+    "name": "John Doe",
+    "permissions": "Admin",
+    "position": "Officer",
+    "department_id": "60d21b4967d0d8992e610c85",
+    "logs": [],
+    "incidents": []
+  }
+}
+```
+
+#### Update Password
+
+```
+PUT /api/auth/password
+```
+
+**Request Body:**
+```json
+{
+  "current_password": "your_current_password",
+  "new_password": "your_new_password"
+}
+```
+
+**Response:**
+```json
+{
+  "message": "Password updated successfully"
+}
+```
+
+#### Update Profile
+
+```
+PUT /api/auth/profile
+```
+
+**Request Body:**
+```json
+{
+  "name": "New Name",
+  "position": "New Position"
+}
+```
+
+**Response:**
+```json
+{
+  "message": "Profile updated successfully",
+  "user": {
+    // Updated user object
+  }
+}
+```
+
+#### Reset Password (Admin Only)
+
+```
+POST /api/auth/reset-password
+```
+
+**Request Body:**
+```json
+{
+  "user_id": "60d21b4967d0d8992e610c85",
+  "new_password": "new_password" // Optional, a random password will be generated if not provided
+}
+```
+
+**Response:**
+```json
+{
+  "message": "Password reset successfully",
+  "user": {
+    // User object
+  },
+  "new_password": "random_password" // Included only if a random password was generated
+}
+```
+
+## Department Endpoints
+
+### Create Department
+
+```
+POST /api/departments
+```
+
+**Request Body:**
+```json
+{
+  "name": "Police Department",
+  "address": "123 Main St, City, State, ZIP",
+  "website": "https://example.com",
+  "admin_email": "admin@example.com",
+  "admin_name": "Admin User",
+  "admin_password": "admin_password",
+  "admin_position": "Administrator"
+}
+```
+
+**Response:**
+```json
+{
+  "message": "Department and admin user created successfully",
+  "department": {
+    "_id": "60d21b4967d0d8992e610c85",
+    "name": "Police Department",
+    "address": "123 Main St, City, State, ZIP",
+    "website": "https://example.com",
+    "members": ["60d21b4967d0d8992e610c86"],
+    "workflows": []
+  },
+  "admin_user": {
+    "_id": "60d21b4967d0d8992e610c86",
+    "email": "admin@example.com",
+    "name": "Admin User",
+    "permissions": "Admin",
+    "position": "Administrator",
+    "department_id": "60d21b4967d0d8992e610c85"
+  }
+}
+```
+
+### Get All Departments
+
+```
+GET /api/departments
+```
+
+**Response:**
+```json
+{
+  "departments": [
+    {
+      "_id": "60d21b4967d0d8992e610c85",
+      "name": "Police Department",
+      "address": "123 Main St, City, State, ZIP",
+      "website": "https://example.com",
+      "members": ["60d21b4967d0d8992e610c86"],
+      "workflows": []
+    }
+  ]
+}
+```
+
+### Get Department by ID
+
+```
+GET /api/departments/:department_id
+```
+
+**Response:**
+```json
+{
+  "department": {
+    "_id": "60d21b4967d0d8992e610c85",
+    "name": "Police Department",
+    "address": "123 Main St, City, State, ZIP",
+    "website": "https://example.com",
+    "members": ["60d21b4967d0d8992e610c86"],
+    "workflows": []
+  }
+}
+```
+
+### Update Department (Admin Only)
+
+```
+PUT /api/departments/:department_id
+```
+
+**Request Body:**
+```json
+{
+  "name": "Updated Department Name",
+  "address": "New Address",
+  "website": "https://new-website.com"
+}
+```
+
+**Response:**
+```json
+{
+  "message": "Department updated successfully",
+  "department": {
+    // Updated department object
+  }
+}
+```
+
+### Delete Department (Admin Only)
+
+```
+DELETE /api/departments/:department_id
+```
+
+**Response:**
+```json
+{
+  "message": "Department and all its users deleted successfully"
+}
+```
+
+### Get Department Members
+
+```
+GET /api/departments/:department_id/members
+```
+
+**Response:**
+```json
+{
+  "members": [
+    {
+      "_id": "60d21b4967d0d8992e610c86",
+      "email": "admin@example.com",
+      "name": "Admin User",
+      "permissions": "Admin",
+      "position": "Administrator"
+    }
+  ]
+}
+```
+
+### Add Member (Admin Only)
+
+```
+POST /api/departments/:department_id/members
+```
+
+**Request Body:**
+```json
+{
+  "email": "new_member@example.com",
+  "name": "New Member",
+  "position": "Officer",
+  "permissions": "User"
+}
+```
+
+**Response:**
+```json
+{
+  "message": "Member added successfully",
+  "user": {
+    // New user object
+  },
+  "raw_password": "random_password" // Password for the new user
+}
+```
+
+### Add Members via CSV (Admin Only)
+
+```
+POST /api/departments/:department_id/members/csv
+```
+
+**Request:**
+Multipart form data with a CSV file containing member information.
+
+**Response:**
+```json
+{
+  "message": "Processed 5 new users with 0 errors",
+  "new_users": [
+    // Array of new user objects, each containing a raw_password
+  ],
+  "errors": [
+    // Array of error messages, if any
+  ]
+}
+```
+
+### Remove Member (Admin Only)
+
+```
+DELETE /api/departments/:department_id/members/:user_id
+```
+
+**Response:**
+```json
+{
+  "message": "Member removed successfully"
+}
+```
+
+### Update Member Permissions (Admin Only)
+
+```
+PUT /api/departments/:department_id/members/:user_id/permissions
+```
+
+**Request Body:**
+```json
+{
+  "permissions": "Admin" // Or "User"
+}
+```
+
+**Response:**
+```json
+{
+  "message": "Member permissions updated successfully",
+  "user": {
+    // Updated user object
+  }
+}
+```
+
+## Workflow Endpoints
+
+### Get All Department Workflows
+
+```
+GET /api/workflows
+```
+
+**Response:**
+```json
+{
+  "workflows": [
+    {
+      "_id": "60d21b4967d0d8992e610c87",
+      "title": "Traffic Violation Workflow",
+      "description": "Process for handling traffic violations",
+      "department_id": "60d21b4967d0d8992e610c85",
+      "data_requirements": [
+        {
+          "field": "license_plate",
+          "description": "Vehicle license plate number"
+        }
+      ],
+      "form_fields": [],
+      "raw_forms": []
+    }
+  ]
+}
+```
+
+### Create Workflow (Admin Only)
+
+```
+POST /api/workflows
+```
+
+**Request Body:**
+```json
+{
+  "title": "Traffic Violation Workflow",
+  "description": "Process for handling traffic violations",
+  "data_requirements": [
+    {
+      "field": "license_plate",
+      "description": "Vehicle license plate number"
+    }
+  ],
+  "form_fields": []
+}
+```
+
+**Response:**
+```json
+{
+  "message": "Workflow created successfully",
+  "workflow": {
+    // New workflow object
+  }
+}
+```
+
+### Get Workflow by ID
+
+```
+GET /api/workflows/:workflow_id
+```
+
+**Response:**
+```json
+{
+  "workflow": {
+    "_id": "60d21b4967d0d8992e610c87",
+    "title": "Traffic Violation Workflow",
+    "description": "Process for handling traffic violations",
+    "department_id": "60d21b4967d0d8992e610c85",
+    "data_requirements": [
+      {
+        "field": "license_plate",
+        "description": "Vehicle license plate number"
+      }
+    ],
+    "form_fields": [],
+    "raw_forms": []
+  }
+}
+```
+
+### Update Workflow (Admin Only)
+
+```
+PUT /api/workflows/:workflow_id
+```
+
+**Request Body:**
+```json
+{
+  "title": "Updated Workflow Title",
+  "description": "Updated description",
+  "data_requirements": [
+    // Updated data requirements
+  ],
+  "form_fields": [
+    // Updated form fields
+  ]
+}
+```
+
+**Response:**
+```json
+{
+  "message": "Workflow updated successfully",
+  "workflow": {
+    // Updated workflow object
+  }
+}
+```
+
+### Delete Workflow (Admin Only)
+
+```
+DELETE /api/workflows/:workflow_id
+```
+
+**Response:**
+```json
+{
+  "message": "Workflow deleted successfully"
+}
+```
+
+### Add Data Requirement (Admin Only)
+
+```
+POST /api/workflows/:workflow_id/data-requirements
+```
+
+**Request Body:**
+```json
+{
+  "field": "driver_license",
+  "description": "Driver's license number"
+}
+```
+
+**Response:**
+```json
+{
+  "message": "Data requirement added successfully",
+  "workflow": {
+    // Updated workflow object
+  }
+}
+```
+
+### Remove Data Requirement (Admin Only)
+
+```
+DELETE /api/workflows/:workflow_id/data-requirements
+```
+
+**Request Body:**
+```json
+{
+  "index": 0
+}
+```
+
+**Response:**
+```json
+{
+  "message": "Data requirement removed successfully",
+  "workflow": {
+    // Updated workflow object
+  }
+}
+```
+
+### Upload Form (Admin Only)
+
+```
+POST /api/workflows/:workflow_id/forms
+```
+
+**Request:**
+Multipart form data with a PDF file.
+
+**Response:**
+```json
+{
+  "message": "Form uploaded successfully",
+  "form_url": "https://res.cloudinary.com/...",
+  "workflow": {
+    // Updated workflow object
+  }
+}
+```
+
+### Remove Form (Admin Only)
+
+```
+DELETE /api/workflows/:workflow_id/forms
+```
+
+**Request Body:**
+```json
+{
+  "form_url": "https://res.cloudinary.com/..."
+}
+```
+
+**Response:**
+```json
+{
+  "message": "Form removed successfully",
+  "workflow": {
+    // Updated workflow object
+  }
+}
+```
+
+### Add Form Field (Admin Only)
+
+```
+POST /api/workflows/:workflow_id/form-fields
+```
+
+**Request Body:**
+```json
+{
+  "position": {"x": 100, "y": 100},
+  "field_name": "license_plate"
+}
+```
+
+**Response:**
+```json
+{
+  "message": "Form field added successfully",
+  "workflow": {
+    // Updated workflow object
+  }
+}
+```
+
+### Remove Form Field (Admin Only)
+
+```
+DELETE /api/workflows/:workflow_id/form-fields
+```
+
+**Request Body:**
+```json
+{
+  "index": 0
+}
+```
+
+**Response:**
+```json
+{
+  "message": "Form field removed successfully",
+  "workflow": {
+    // Updated workflow object
+  }
+}
+```
+
+## Log Endpoints
+
+### Upload Log
+
+```
+POST /api/logs
+```
+
+**Request:**
+Multipart form data with:
+- `file`: PDF file of the log
+- `log_date`: Date in YYYY-MM-DD format
+
+**Response:**
+```json
+{
+  "message": "Log uploaded successfully and processing started",
+  "log": {
+    "_id": "60d21b4967d0d8992e610c88",
+    "user_id": "60d21b4967d0d8992e610c86",
+    "department_id": "60d21b4967d0d8992e610c85",
+    "log_date": "2023-05-15",
+    "log_file": "https://res.cloudinary.com/...",
+    "incidents": []
+  }
+}
+```
+
+### Get User Logs
+
+```
+GET /api/logs/user
+```
+
+**Response:**
+```json
+{
+  "logs": [
+    {
+      "_id": "60d21b4967d0d8992e610c88",
+      "user_id": "60d21b4967d0d8992e610c86",
+      "department_id": "60d21b4967d0d8992e610c85",
+      "log_date": "2023-05-15",
+      "log_file": "https://res.cloudinary.com/...",
+      "incidents": []
+    }
+  ]
+}
+```
+
+### Get Department Logs (Admin Only)
+
+```
+GET /api/logs/department
+```
+
+**Response:**
+```json
+{
+  "logs": [
+    // Array of all logs in the department
+  ]
+}
+```
+
+### Get Logs by Date Range
+
+```
+POST /api/logs/date-range
+```
+
+**Request Body:**
+```json
+{
+  "start_date": "2023-05-01",
+  "end_date": "2023-05-31"
+}
+```
+
+**Response:**
+```json
+{
+  "logs": [
+    // Array of logs within the date range
+  ]
+}
+```
+
+### Get Log by ID
+
+```
+GET /api/logs/:log_id
+```
+
+**Response:**
+```json
+{
+  "log": {
+    "_id": "60d21b4967d0d8992e610c88",
+    "user_id": "60d21b4967d0d8992e610c86",
+    "department_id": "60d21b4967d0d8992e610c85",
+    "log_date": "2023-05-15",
+    "log_file": "https://res.cloudinary.com/...",
+    "incidents": []
+  }
+}
+```
+
+### Delete Log
+
+```
+DELETE /api/logs/:log_id
+```
+
+**Response:**
+```json
+{
+  "message": "Log and associated incidents deleted successfully"
+}
+```
+
+## Incident Endpoints
+
+### Get User Incidents
+
+```
+GET /api/incidents/user
+```
+
+**Response:**
+```json
+{
+  "incidents": [
+    {
+      "_id": "60d21b4967d0d8992e610c89",
+      "department_id": "60d21b4967d0d8992e610c85",
+      "user_id": "60d21b4967d0d8992e610c86",
+      "workflow_id": "60d21b4967d0d8992e610c87",
+      "description": "Traffic violation at Main St",
+      "date": "2023-05-15",
+      "log_id": "60d21b4967d0d8992e610c88",
+      "activity_text": "Observed vehicle speeding...",
+      "status": "completed",
+      "filled_forms": [],
+      "extracted_data": {
+        "license_plate": "ABC123"
+      }
+    }
+  ]
+}
+```
+
+### Get Department Incidents (Admin Only)
+
+```
+GET /api/incidents/department
+```
+
+**Response:**
+```json
+{
+  "incidents": [
+    // Array of all incidents in the department
+  ]
+}
+```
+
+### Get Incidents by Date Range
+
+```
+POST /api/incidents/date-range
+```
+
+**Request Body:**
+```json
+{
+  "start_date": "2023-05-01",
+  "end_date": "2023-05-31"
+}
+```
+
+**Response:**
+```json
+{
+  "incidents": [
+    // Array of incidents within the date range
+  ]
+}
+```
+
+### Get Workflow Incidents
+
+```
+GET /api/incidents/workflow/:workflow_id
+```
+
+**Response:**
+```json
+{
+  "incidents": [
+    // Array of incidents for the specified workflow
+  ]
+}
+```
+
+### Get Incident by ID
+
+```
+GET /api/incidents/:incident_id
+```
+
+**Response:**
+```json
+{
+  "incident": {
+    "_id": "60d21b4967d0d8992e610c89",
+    "department_id": "60d21b4967d0d8992e610c85",
+    "user_id": "60d21b4967d0d8992e610c86",
+    "workflow_id": "60d21b4967d0d8992e610c87",
+    "description": "Traffic violation at Main St",
+    "date": "2023-05-15",
+    "log_id": "60d21b4967d0d8992e610c88",
+    "activity_text": "Observed vehicle speeding...",
+    "status": "completed",
+    "filled_forms": [],
+    "extracted_data": {
+      "license_plate": "ABC123"
+    }
+  }
+}
+```
+
+### Delete Incident
+
+```
+DELETE /api/incidents/:incident_id
+```
+
+**Response:**
+```json
+{
+  "message": "Incident deleted successfully"
+}
+```
+
+### Reprocess Incident (Admin Only)
+
+```
+POST /api/incidents/:incident_id/reprocess
+```
+
+**Response:**
+```json
+{
+  "message": "Incident reprocessing started",
+  "incident": {
+    // Updated incident object
+  }
+}
+```
+
+## Utility Endpoints
+
+### Health Check
+
+```
+GET /health
+```
+
+**Response:**
+```json
+{
+  "status": "healthy",
+  "mongo": "connected",
+  "env_vars": "ok",
+  "upload_dir": "/tmp/uploads",
+  "log_dir": "/tmp/logs"
+}
+```
+
+### API Test
+
+```
+GET /api/test
+```
+
+**Response:**
+```json
+{
+  "message": "API test successful",
+  "timestamp": "2023-05-15T12:00:00.000Z",
+  "environment": "development"
+}
+``` 

celery_config.py

@@ -0,0 +1,49 @@
+import os
+from dotenv import load_dotenv
+
+# Load environment variables
+load_dotenv()
+
+# Celery configuration with hosted Redis server
+redis_password = os.environ.get('REDIS_PASSWORD')
+# Make sure to set these environment variables with the correct values
+redis_host = os.environ.get('REDIS_HOST')
+redis_port = os.environ.get('REDIS_PORT', '6379')
+
+# Validate that we have the host information
+if not redis_host:
+    raise ValueError("REDIS_HOST environment variable must be set")
+
+redis_url = f"redis://:{redis_password}@{redis_host}:{redis_port}/0"
+
+broker_url = os.environ.get('REDIS_URL', redis_url)
+result_backend = os.environ.get('REDIS_URL', redis_url)
+
+# Task serialization format
+task_serializer = 'json'
+accept_content = ['json']
+result_serializer = 'json'
+
+# Enable UTC timezone
+enable_utc = True
+
+# Task execution settings
+task_acks_late = True
+task_reject_on_worker_lost = True
+worker_max_tasks_per_child = 1000
+
+# Task timeouts
+task_time_limit = 600  # 10 minutes
+task_soft_time_limit = 300  # 5 minutes
+
+# Task routes
+task_routes = {
+    'utils.celery_tasks.process_log_document': {'queue': 'logs'},
+    'utils.celery_tasks.process_incident_forms': {'queue': 'incidents'},
+}
+
+# Rate limits
+task_annotations = {
+    'utils.celery_tasks.process_log_document': {'rate_limit': '10/h'},
+    'utils.celery_tasks.process_incident_forms': {'rate_limit': '20/h'},
+}