feat(staff): implement staff list endpoint with projection support

refactor(response): standardize error response format
test: add tests for error handling and staff list endpoint
docs: remove outdated documentation files
feat(middleware): add request logging middleware

POS_COLLECTIONS.md

@@ -1,100 +0,0 @@
-# POS MongoDB Collections
-
-## Collection Naming Convention
-
-All POS MongoDB collections use the `pos_*` prefix for better organization and to avoid conflicts with other microservices.
-
-## Collection List
-
-| Collection Name | Purpose | Key Fields |
-|----------------|---------|------------|
-| `pos_customers` | Customer data | customer_id, merchant_id, name, phone, email |
-| `pos_staff` | Staff/employee data | staff_id, merchant_id, name, role, specializations |
-| `pos_catalogue_services` | Service catalog | service_id, merchant_id, name, code, pricing |
-| `pos_appointments` | Appointment bookings | appointment_id, merchant_id, customer_id, services |
-| `pos_sales` | Sales transactions | sale_id, merchant_id, customer_id, items, payments |
-| `pos_sales_items` | Individual sale items | sale_item_id, sale_id, service_id, staff_id |
-| `pos_payments` | Payment records | payment_id, sale_id, amount, payment_mode |
-| `pos_refunds` | Refund records | refund_id, sale_id, amount, reason |
-
-## Index Strategy
-
-Each collection includes standard indexes:
-- `merchant_id` - For tenant isolation
-- `created_at` - For time-based queries
-- Entity-specific indexes (e.g., `customer_id`, `staff_id`)
-
-## Usage in Code
-
-### Service Layer
-```python
-# Catalogue services
-CATALOGUE_SERVICES_COLLECTION = "pos_catalogue_services"
-db = get_database()
-services = await db[CATALOGUE_SERVICES_COLLECTION].find(query).to_list(length=limit)
-
-# Customers
-customers = await db.pos_customers.find({"merchant_id": merchant_id}).to_list(length=None)
-
-# Staff
-staff = await db.pos_staff.find({"merchant_id": merchant_id}).to_list(length=None)
-```
-
-### Sync Operations
-```python
-# MongoDB to PostgreSQL sync
-mongo_customers = await mongo_db.pos_customers.find({"merchant_id": merchant_id}).to_list(length=None)
-mongo_staff = await mongo_db.pos_staff.find({"merchant_id": merchant_id}).to_list(length=None)
-mongo_services = await mongo_db.pos_catalogue_services.find({"merchant_id": merchant_id}).to_list(length=None)
-```
-
-## Migration Notes
-
-If you have existing data in the old collection names, you'll need to migrate:
-
-```javascript
-// MongoDB migration script
-use cuatrolabs;
-
-// Migrate customers
-db.customers.find({}).forEach(function(doc) {
-    db.pos_customers.insert(doc);
-});
-
-// Migrate staff
-db.staff.find({}).forEach(function(doc) {
-    db.pos_staff.insert(doc);
-});
-
-// Migrate catalogue services
-db.catalogue_services.find({}).forEach(function(doc) {
-    db.pos_catalogue_services.insert(doc);
-});
-
-// Verify counts match
-print("Original customers:", db.customers.count());
-print("New pos_customers:", db.pos_customers.count());
-
-print("Original staff:", db.staff.count());
-print("New pos_staff:", db.pos_staff.count());
-
-print("Original catalogue_services:", db.catalogue_services.count());
-print("New pos_catalogue_services:", db.pos_catalogue_services.count());
-```
-
-## Benefits
-
-1. **Clear Separation**: Easy to identify POS-specific collections
-2. **No Conflicts**: Avoids naming conflicts with SCM or other microservices
-3. **Better Organization**: Logical grouping of related collections
-4. **Easier Maintenance**: Clear ownership and responsibility
-5. **Scalability**: Easy to add new POS collections following the same pattern
-
-## Future Collections
-
-When adding new POS collections, follow the naming convention:
-- `pos_inventory` - Inventory tracking
-- `pos_promotions` - Promotional campaigns
-- `pos_loyalty` - Loyalty program data
-- `pos_analytics` - Analytics and reporting data
-- `pos_settings` - POS-specific configuration

POS_ENDPOINT_PROTECTION_SUMMARY.md

@@ -1,212 +0,0 @@
-# POS Microservice Endpoint Protection Implementation
-
-## Overview
-All POS microservice endpoints have been protected with role-based permissions similar to the SCM microservice pattern. The implementation uses JWT token validation and permission checking against the `scm_access_roles` collection.
-
-## Protection System Components
-
-### 1. POS Permissions Module
-**File:** `app/dependencies/pos_permissions.py`
-
-- `POSPermissionChecker`: Core class for permission validation
-- `require_pos_permission(module, action)`: Dependency factory for single permission
-- `require_pos_permissions(permissions_list)`: Dependency factory for multiple permissions
-- `get_user_permissions()`: Utility to get user's full permission set
-
-### 2. Permission Modules
-The following POS modules are protected:
-
-- **sales**: Sales orders, retail sales, invoicing
-- **customers**: Customer management
-- **staff**: Staff management
-- **appointments**: Appointment booking and management
-- **catalogue**: Service catalogue management
-
-### 3. Permission Actions
-Standard CRUD actions are supported:
-
-- **view**: Read access to resources
-- **create**: Create new resources
-- **update**: Modify existing resources
-- **delete**: Remove resources
-
-## Protected Endpoints
-
-### Sales Orders (`/api/v1/sales/`)
-- `POST /order/list` - requires `sales.view`
-- `GET /order/{id}` - requires `sales.view`
-- `POST /order` - requires `sales.create`
-- `PUT /order/{id}` - requires `sales.update`
-- `POST /order/{id}/invoice` - requires `sales.create`
-- `GET /info/widgets` - requires `sales.view`
-
-### Customers (`/api/v1/customers/`)
-- `POST /` - requires `customers.create`
-- `GET /{id}` - requires `customers.view`
-- `POST /list` - requires `customers.view` (converted from GET to POST)
-- `PUT /{id}` - requires `customers.update`
-- `DELETE /{id}` - requires `customers.delete`
-
-### Staff (`/api/v1/staff/`)
-- `POST /` - requires `staff.create`
-- `POST /list` - requires `staff.view` (converted from GET to POST)
-- `GET /code/{code}` - requires `staff.view`
-- `GET /{id}` - requires `staff.view`
-- `PUT /{id}` - requires `staff.update`
-- `PATCH /{id}/status` - requires `staff.update`
-- `DELETE /{id}` - requires `staff.delete`
-
-### Appointments (`/api/v1/pos/appointments/`)
-- `POST /` - requires `appointments.create`
-- `POST /list` - requires `appointments.view` (converted from GET to POST)
-- `GET /{id}` - requires `appointments.view`
-- `PUT /{id}` - requires `appointments.update`
-- `PATCH /{id}/status` - requires `appointments.update`
-- `POST /{id}/cancel` - requires `appointments.update`
-- `POST /{id}/checkout` - requires `appointments.update`
-
-### Catalogue Services (`/api/v1/pos/catalogue/services/`)
-- `POST /` - requires `catalogue.create`
-- `POST /list` - requires `catalogue.view` (converted from GET to POST)
-- `GET /{id}` - requires `catalogue.view`
-- `PUT /{id}` - requires `catalogue.update`
-- `PATCH /{id}/status` - requires `catalogue.update`
-- `DELETE /{id}` - requires `catalogue.delete`
-
-### Retail Sales (`/api/v1/pos/sales/`)
-- `POST /` - requires `sales.create`
-- `GET /{id}` - requires `sales.view`
-- `PUT /{id}/items` - requires `sales.update`
-- `POST /{id}/payments` - requires `sales.update`
-- `POST /{id}/cancel` - requires `sales.update`
-- `POST /{id}/refund` - requires `sales.update`
-
-## API List Endpoint Standard Compliance
-
-### Changes Made
-1. **GET to POST Conversion**: All list endpoints converted from GET to POST method
-2. **Consistent Naming**: All list endpoints use `/list` path
-3. **Ready for Projection**: Endpoints prepared for projection_list parameter implementation
-
-### Endpoints Converted
-- `GET /customers/` → `POST /customers/list`
-- `GET /staff/` → `POST /staff/list`
-- `GET /pos/appointments/` → `POST /pos/appointments/list`
-- `GET /pos/catalogue/services/` → `POST /pos/catalogue/services/list`
-
-### Next Steps for Full Compliance
-To complete the API list endpoint standard implementation:
-
-1. **Add projection_list to schemas**: Update request schemas to include `projection_list: Optional[List[str]]`
-2. **Update service layers**: Implement MongoDB projection logic in service methods
-3. **Return type handling**: Return raw dict when projection used, Pydantic model otherwise
-
-## Usage Examples
-
-### Basic Permission Check
-```python
-@router.post("/orders")
-async def create_order(
-    payload: OrderCreate,
-    current_user: TokenUser = Depends(require_pos_permission("sales", "create"))
-):
-    # Only users with sales.create permission can access
-    return await OrderService.create_order(payload)
-```
-
-### Multiple Permission Check
-```python
-@router.post("/complex-operation")
-async def complex_operation(
-    current_user: TokenUser = Depends(require_pos_permissions([
-        {"module": "sales", "action": "create"},
-        {"module": "customers", "action": "update"}
-    ]))
-):
-    # Requires both permissions
-    pass
-```
-
-## Role Configuration
-
-### Sample POS Role in `scm_access_roles` Collection
-```json
-{
-  "role_id": "role_pos_manager",
-  "role_name": "POS Manager",
-  "is_active": true,
-  "permissions": {
-    "sales": ["view", "create", "update"],
-    "customers": ["view", "create", "update", "delete"],
-    "staff": ["view", "create", "update"],
-    "appointments": ["view", "create", "update"],
-    "catalogue": ["view", "create", "update"]
-  }
-}
-```
-
-### Sample POS Cashier Role
-```json
-{
-  "role_id": "role_pos_cashier",
-  "role_name": "POS Cashier",
-  "is_active": true,
-  "permissions": {
-    "sales": ["view", "create"],
-    "customers": ["view", "create"],
-    "appointments": ["view", "create", "update"]
-  }
-}
-```
-
-## Security Benefits
-
-1. **Role-based Access Control**: Fine-grained permissions per module and action
-2. **JWT Token Validation**: Secure token-based authentication
-3. **Consistent Pattern**: Same security model as SCM microservice
-4. **Audit Trail**: Permission checks are logged for security monitoring
-5. **Flexible Permissions**: Easy to add new modules and actions
-
-## Error Responses
-
-### 401 Unauthorized
-```json
-{
-  "detail": "Could not validate credentials"
-}
-```
-
-### 403 Forbidden
-```json
-{
-  "detail": "Access denied. Required permission: sales.create"
-}
-```
-
-## Implementation Status
-
-✅ **Completed:**
-- POS permissions system created
-- All existing endpoints protected
-- List endpoints converted to POST method
-- Consistent permission naming
-
-🔄 **Next Steps:**
-- Add projection_list support to schemas and services
-- Create POS-specific role definitions
-- Add integration tests for permission system
-- Document role setup procedures
-
-## Maintenance
-
-### Adding New Endpoints
-1. Import required dependencies
-2. Add permission check to endpoint
-3. Use appropriate module and action names
-4. Follow POST method for list endpoints
-
-### Adding New Modules
-1. Define module name (e.g., "inventory", "reports")
-2. Add to role permissions in database
-3. Use in `require_pos_permission()` calls
-4. Document in this file

POS_SYNC_IMPLEMENTATION.md

@@ -1,310 +0,0 @@
-# POS MongoDB to PostgreSQL Sync Implementation
-
-## Overview
-
-This document describes the MongoDB to PostgreSQL synchronization utility implemented for the POS microservice. The sync system ensures that critical POS data is available in both MongoDB (primary) and PostgreSQL (reference) for improved query performance and reporting capabilities.
-
-## Architecture
-
-### Components
-
-1. **Sync Models** (`app/sync/models.py`)
-   - PostgreSQL table definitions for reference data
-   - Currently supports: `CustomerRef`, `StaffRef`
-
-2. **Sync Services** 
-   - `CustomerSyncService` - Handles customer data synchronization
-   - `StaffSyncService` - Handles staff data synchronization
-   - `POSSyncService` - Main coordinator service
-
-3. **Sync Handler** (`app/sync/handler.py`)
-   - Event-driven sync operations
-   - Automatic session management
-   - Error handling and logging
-
-4. **Utilities** (`app/sync/utils.py`)
-   - Migration tools for existing data
-   - Consistency verification
-   - Cleanup operations
-
-## Supported Entities
-
-### Customers
-- **MongoDB Collection**: `customers`
-- **PostgreSQL Table**: `pos.customer_ref`
-- **Key Fields**: customer_id, merchant_id, name, phone, email, notes, status
-
-### Staff
-- **MongoDB Collection**: `staff`
-- **PostgreSQL Table**: `pos.staff_ref`
-- **Key Fields**: staff_id, merchant_id, name, phone, email, role, specializations, status
-
-### Catalogue Services
-- **MongoDB Collection**: `catalogue_services`
-- **PostgreSQL Table**: `pos.catalogue_service_ref`
-- **Key Fields**: service_id, merchant_id, service_name, service_code, category, duration_mins, price, gst_rate, status
-
-## Usage
-
-### 1. Basic Sync Operations
-
-```python
-from app.sync.handler import POSSyncHandler
-from app.customers.models.model import CustomerModel
-
-# Customer operations
-customer = CustomerModel(...)
-await POSSyncHandler.handle_customer_created(customer)
-await POSSyncHandler.handle_customer_updated(customer_id, customer)
-await POSSyncHandler.handle_customer_deleted(customer_id)
-await POSSyncHandler.handle_customer_status_changed(customer_id, "inactive")
-
-# Staff operations
-staff_data = {...}
-await POSSyncHandler.handle_staff_created(staff_data)
-await POSSyncHandler.handle_staff_updated(staff_id, staff_data)
-await POSSyncHandler.handle_staff_deleted(staff_id)
-await POSSyncHandler.handle_staff_status_changed(staff_id, "inactive")
-
-# Catalogue service operations
-service_data = {...}
-await POSSyncHandler.handle_catalogue_service_created(service_data)
-await POSSyncHandler.handle_catalogue_service_updated(service_id, service_data)
-await POSSyncHandler.handle_catalogue_service_deleted(service_id)
-await POSSyncHandler.handle_catalogue_service_status_changed(service_id, "archived")
-```
-
-### 2. Integration with Service Layer
-
-Add sync calls to your service methods:
-
-```python
-# In customers/services/service.py
-from app.sync.handler import POSSyncHandler
-
-async def create_customer(self, customer_data: dict):
-    # Create in MongoDB
-    customer = await self.mongo_collection.insert_one(customer_data)
-    
-    # Sync to PostgreSQL
-    customer_model = CustomerModel(**customer_data)
-    await POSSyncHandler.handle_customer_created(customer_model)
-    
-    return customer
-
-async def update_customer(self, customer_id: str, update_data: dict):
-    # Update in MongoDB
-    result = await self.mongo_collection.update_one(
-        {"customer_id": customer_id}, 
-        {"$set": update_data}
-    )
-    
-    # Sync to PostgreSQL
-    updated_customer = await self.get_customer(customer_id)
-    await POSSyncHandler.handle_customer_updated(customer_id, updated_customer)
-    
-    return result
-```
-
-### 3. Migration and Maintenance
-
-```bash
-# Migrate existing customers
-python -m app.sync.utils migrate_customers merchant_123 1000
-
-# Migrate existing staff
-python -m app.sync.utils migrate_staff merchant_123 1000
-
-# Verify consistency
-python -m app.sync.utils verify_consistency merchant_123 customers
-
-# Clean up orphaned records
-python -m app.sync.utils cleanup_orphaned merchant_123 customers
-```
-
-### 4. Health Monitoring
-
-```python
-from app.sync.handler import POSSyncHandler
-
-# Check sync system health
-health = await POSSyncHandler.check_sync_health()
-print(health)
-# {
-#   "overall_health": True,
-#   "customer_sync": True,
-#   "staff_sync": True,
-#   "timestamp": "..."
-# }
-```
-
-## Database Schema
-
-### PostgreSQL Tables
-
-```sql
--- Customer reference table
-CREATE TABLE pos.customer_ref (
-    customer_id VARCHAR PRIMARY KEY,
-    merchant_id VARCHAR NOT NULL,
-    name VARCHAR(150) NOT NULL,
-    phone VARCHAR(20),
-    email VARCHAR(255),
-    notes TEXT,
-    status VARCHAR(20) NOT NULL DEFAULT 'active',
-    created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
-);
-
--- Staff reference table
-CREATE TABLE pos.staff_ref (
-    staff_id VARCHAR PRIMARY KEY,
-    merchant_id VARCHAR NOT NULL,
-    name VARCHAR(150) NOT NULL,
-    phone VARCHAR(20),
-    email VARCHAR(255),
-    role VARCHAR(50),
-    specializations TEXT[],
-    status VARCHAR(20) NOT NULL DEFAULT 'active',
-    created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
-    updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
-);
-```
-
-### Indexes
-
-```sql
--- Customer indexes
-CREATE INDEX idx_customer_ref_merchant_id ON pos.customer_ref(merchant_id);
-CREATE INDEX idx_customer_ref_phone ON pos.customer_ref(phone);
-CREATE INDEX idx_customer_ref_email ON pos.customer_ref(email);
-
--- Staff indexes
-CREATE INDEX idx_staff_ref_merchant_id ON pos.staff_ref(merchant_id);
-CREATE INDEX idx_staff_ref_status ON pos.staff_ref(status);
-```
-
-## Testing
-
-Run the test suite to verify sync functionality:
-
-```bash
-cd cuatrolabs-pos-ms
-python test_pos_sync.py
-```
-
-The test suite covers:
-- Customer CRUD sync operations
-- Staff CRUD sync operations
-- Bulk sync operations
-- Health checks
-- Error handling
-
-## Error Handling
-
-The sync system includes comprehensive error handling:
-
-1. **Graceful Degradation**: MongoDB operations continue even if PostgreSQL sync fails
-2. **Logging**: All sync operations are logged with appropriate levels
-3. **Retry Logic**: Can be implemented at the service layer for critical operations
-4. **Health Monitoring**: Regular health checks to detect sync issues
-
-## Performance Considerations
-
-1. **Async Operations**: All sync operations are asynchronous
-2. **Bulk Operations**: Support for bulk sync to reduce overhead
-3. **Selective Sync**: Only sync essential fields to PostgreSQL
-4. **Connection Pooling**: Uses SQLAlchemy async sessions with connection pooling
-
-## Best Practices
-
-1. **Always sync after MongoDB operations**: Ensure data consistency
-2. **Handle sync failures gracefully**: Don't fail the main operation if sync fails
-3. **Monitor sync health**: Regular health checks and alerting
-4. **Use bulk operations**: For large data migrations or batch updates
-5. **Verify consistency**: Regular consistency checks between MongoDB and PostgreSQL
-
-## Future Enhancements
-
-1. **Event-Driven Sync**: Implement MongoDB change streams for real-time sync
-2. **Conflict Resolution**: Handle conflicts when data differs between systems
-3. **Selective Field Sync**: Configure which fields to sync per entity
-4. **Audit Trail**: Track sync operations for debugging and compliance
-5. **Performance Metrics**: Add metrics for sync operation performance
-
-## Troubleshooting
-
-### Common Issues
-
-1. **Schema Creation Fails**
-   - Check PostgreSQL permissions
-   - Verify connection string
-   - Ensure schema exists
-
-2. **Sync Operations Fail**
-   - Check PostgreSQL connectivity
-   - Verify table structure
-   - Review error logs
-
-3. **Data Inconsistency**
-   - Run consistency verification
-   - Check for failed sync operations
-   - Review application logs
-
-### Debug Commands
-
-```python
-# Check sync health
-from app.sync.handler import POSSyncHandler
-health = await POSSyncHandler.check_sync_health()
-
-# Verify specific merchant data
-from app.sync.utils import verify_sync_consistency
-result = await verify_sync_consistency("merchant_123", "customers")
-
-# Manual sync test
-from app.sync.sync_service import POSSyncService
-from app.sql import get_postgres_session
-
-async with get_postgres_session() as session:
-    sync_service = POSSyncService(session)
-    # Test operations...
-```
-
-## Integration with Existing Code
-
-To integrate with existing POS services:
-
-1. Import the sync handler in your service files
-2. Add sync calls after successful MongoDB operations
-3. Handle sync failures appropriately (log but don't fail main operation)
-4. Add health checks to your monitoring system
-5. Set up regular consistency verification jobs
-
-Example integration:
-
-```python
-# In your existing service
-from app.sync.handler import POSSyncHandler
-import logging
-
-logger = logging.getLogger(__name__)
-
-async def create_customer(self, customer_data):
-    try:
-        # Existing MongoDB operation
-        result = await self.collection.insert_one(customer_data)
-        
-        # Add sync operation
-        try:
-            customer_model = CustomerModel(**customer_data)
-            await POSSyncHandler.handle_customer_created(customer_model)
-        except Exception as sync_error:
-            # Log but don't fail the main operation
-            logger.error(f"Customer sync failed: {sync_error}")
-        
-        return result
-    except Exception as e:
-        logger.error(f"Customer creation failed: {e}")
-        raise
-```

app/main.py

@@ -2,11 +2,17 @@
 Main FastAPI application for POS Microservice.
 """
 import logging
-from fastapi import FastAPI
+from fastapi import FastAPI, Request, status
+from fastapi.exceptions import RequestValidationError
+from starlette.exceptions import HTTPException as StarletteHTTPException
+from fastapi.responses import JSONResponse
 from fastapi.middleware.cors import CORSMiddleware
-# # from insightfy_utils.logging import get_logger  # TODO: Uncomment when package is available
 import logging  # TODO: Uncomment when package is available
+from pymongo.errors import PyMongoError, ConnectionFailure
+
 from app.core.config import settings
+from app.utils.response import error_response
+from app.middleware.logging_middleware import RequestLoggingMiddleware
 
 from app.nosql import connect_to_mongo, close_mongo_connection
 from app.sql import connect_to_database, disconnect_from_database
@@ -29,6 +35,9 @@ app = FastAPI(
     redoc_url="/redoc",
 )
 
+# Request logging middleware
+app.add_middleware(RequestLoggingMiddleware)
+
 # CORS middleware
 app.add_middleware(
     CORSMiddleware,
@@ -39,6 +48,118 @@ app.add_middleware(
 )
 
 
+# Global Exception Handlers
+@app.exception_handler(RequestValidationError)
+async def validation_exception_handler(request: Request, exc: RequestValidationError):
+    """Handle validation errors"""
+    errors = []
+    for e in exc.errors():
+        # Get field name from loc (last element), default to "body" or "unknown"
+        field = str(e["loc"][-1]) if e["loc"] else "unknown"
+        errors.append({
+            "field": field,
+            "message": e["msg"],
+            "type": e["type"]
+        })
+        
+    return JSONResponse(
+        status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
+        content=error_response(
+            error="Validation Error",
+            detail="The request contains invalid data",
+            errors=errors,
+            request_id=getattr(request.state, "request_id", None)
+        )
+    )
+
+
+@app.exception_handler(status.HTTP_404_NOT_FOUND)
+async def not_found_exception_handler(request: Request, exc: Exception):
+    """Handle 404 errors"""
+    return JSONResponse(
+        status_code=status.HTTP_404_NOT_FOUND,
+        content=error_response(
+            error="Not Found",
+            detail="Resource not found",
+            request_id=getattr(request.state, "request_id", None)
+        )
+    )
+
+
+@app.exception_handler(StarletteHTTPException)
+async def http_exception_handler(request: Request, exc: StarletteHTTPException):
+    """Handle standard HTTP exceptions"""
+    error_title = "Error"
+    headers = None
+    
+    if exc.status_code == 401:
+        error_title = "Authentication Error"
+        headers = {"WWW-Authenticate": "Bearer"}
+    elif exc.status_code == 403:
+        error_title = "Permission Denied"
+    elif exc.status_code == 404:
+        error_title = "Not Found"
+    elif exc.status_code == 422:
+        error_title = "Validation Error"
+        
+    return JSONResponse(
+        status_code=exc.status_code,
+        content=error_response(
+            error=error_title,
+            detail=str(exc.detail),
+            request_id=getattr(request.state, "request_id", None),
+            headers=headers
+        )
+    )
+
+
+@app.exception_handler(ConnectionFailure)
+async def mongo_connection_exception_handler(request: Request, exc: ConnectionFailure):
+    """Handle MongoDB connection errors"""
+    logger.error(f"Database connection error: {exc}", exc_info=True)
+    return JSONResponse(
+        status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+        content=error_response(
+            error="Database Connection Error",
+            detail="Unable to connect to the database. Please try again later.",
+            request_id=getattr(request.state, "request_id", None)
+        )
+    )
+
+
+@app.exception_handler(PyMongoError)
+async def mongo_exception_handler(request: Request, exc: PyMongoError):
+    """Handle general MongoDB errors"""
+    logger.error(f"Database error: {exc}", exc_info=True)
+    return JSONResponse(
+        status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+        content=error_response(
+            error="Database Error",
+            detail="An unexpected database error occurred.",
+            request_id=getattr(request.state, "request_id", None)
+        )
+    )
+
+
+# Handle generic HTTPException (catch-all for other HTTP exceptions)
+@app.exception_handler(Exception)
+async def generic_exception_handler(request: Request, exc: Exception):
+    """
+    Handle all unhandled exceptions.
+    
+    In production, we should not expose internal error details.
+    """
+    logger.error(f"Unhandled exception: {exc}", exc_info=True)
+    return JSONResponse(
+        status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
+        content=error_response(
+            error="Internal Server Error",
+            detail="An unexpected error occurred. Please try again later.",
+            request_id=getattr(request.state, "request_id", None)
+        )
+    )
+
+
 # Startup and shutdown events
 @app.on_event("startup")
 async def startup_event():

app/middleware/logging_middleware.py

@@ -0,0 +1,68 @@
+import time
+import logging
+from uuid import uuid4
+from fastapi import Request
+from starlette.middleware.base import BaseHTTPMiddleware
+
+logger = logging.getLogger("middleware")
+
+class RequestLoggingMiddleware(BaseHTTPMiddleware):
+    """
+    Middleware to log request start and completion details.
+    """
+    async def dispatch(self, request: Request, call_next):
+        request_id = request.headers.get("X-Request-ID", str(uuid4()))
+        # Store request_id in request state for access in routers/error handlers
+        request.state.request_id = request_id
+        
+        start_time = time.time()
+        
+        client_host = request.client.host if request.client else "unknown"
+        user_agent = request.headers.get("user-agent", "unknown")
+        
+        # Log request start
+        logger.info(
+            f"Request started: {request.method} {request.url.path}",
+            extra={
+                "request_id": request_id,
+                "method": request.method,
+                "path": request.url.path,
+                "client": client_host,
+                "user_agent": user_agent
+            }
+        )
+        
+        try:
+            response = await call_next(request)
+            
+            process_time = time.time() - start_time
+            
+            # Log request completion
+            logger.info(
+                f"Request completed: {request.method} {request.url.path} - Status: {response.status_code}",
+                extra={
+                    "request_id": request_id,
+                    "method": request.method,
+                    "path": request.url.path,
+                    "status_code": response.status_code,
+                    "process_time": f"{process_time:.3f}s"
+                }
+            )
+            
+            response.headers["X-Request-ID"] = request_id
+            return response
+            
+        except Exception as e:
+            process_time = time.time() - start_time
+            logger.error(
+                f"Request failed: {request.method} {request.url.path}",
+                extra={
+                    "request_id": request_id,
+                    "method": request.method,
+                    "path": request.url.path,
+                    "status_code": 500,
+                    "process_time": f"{process_time:.3f}s",
+                    "error": str(e)
+                }
+            )
+            raise e

app/staff/controllers/router.py

@@ -11,6 +11,7 @@ from app.staff.schemas.staff_schema import (
     StaffCreateSchema,
     StaffUpdateSchema,
     StaffResponseSchema,
+    StaffListRequest,
     StaffListResponse,
     EmployeeUpdate,
     EmployeeResponse
@@ -198,63 +199,49 @@ async def update_employee(
     return await StaffService.update_employee(user_id, payload, x_user_id)
 
 
-@router.get(
-    "",
-    response_model=List[EmployeeResponse],
+@router.post(
+    "/list",
+    response_model=StaffListResponse,
     summary="List staff",
     description="List staff with optional filters and pagination"
 )
 async def list_staff(
-    designation: Optional[Designation] = Query(
-        None,
-        description="Filter by designation/role"
-    ),
-    manager_id: Optional[str] = Query(
-        None,
-        description="Filter by manager's user_id (direct reports)"
-    ),
-    status_filter: Optional[stafftatus] = Query(
-        None,
-        alias="status",
-        description="Filter by employee status"
-    ),
-    region: Optional[str] = Query(
-        None,
-        description="Filter by region"
-    ),
-    skip: int = Query(
-        0,
-        ge=0,
-        description="Number of records to skip (pagination)"
-    ),
-    limit: int = Query(
-        100,
-        ge=1,
-        le=500,
-        description="Maximum number of records to return"
-    )
-) -> List[EmployeeResponse]:
+    payload: StaffListRequest
+) -> StaffListResponse:
     """
     List staff with filters.
     
-    **Query Parameters:**
+    **Filters:**
     - `designation`: Filter by role (RSM, ASM, BDE, Trainer, etc.)
     - `manager_id`: Show only direct reports of specific manager
     - `status`: Filter by status (onboarding, active, inactive, suspended, terminated)
     - `region`: Filter by assigned region
+    
+    **Pagination:**
     - `skip`: Pagination offset (default: 0)
     - `limit`: Page size (default: 100, max: 500)
     
+    **Projection:**
+    - `projection_list`: List of fields to include in response
+    
     Returns:
-        List of staff matching the filters
+        List of staff matching the filters and total count
     """
-    return await StaffService.list_staff(
-        designation=designation,
-        manager_id=manager_id,
-        status_filter=status_filter,
-        region=region,
-        skip=skip,
-        limit=limit
+    items, total = await StaffService.list_staff(
+        designation=payload.designation,
+        manager_id=payload.manager_id,
+        status_filter=payload.status,
+        region=payload.region,
+        skip=payload.skip,
+        limit=payload.limit,
+        projection_list=payload.projection_list
+    )
+    
+    return StaffListResponse(
+        items=items,
+        total=total,
+        skip=payload.skip,
+        limit=payload.limit
     )
 
 
@@ -319,12 +306,13 @@ async def get_employee_reports(
     # First verify employee exists
     await StaffService.get_employee(user_id)
     
-    return await StaffService.list_staff(
+    items, _ = await StaffService.list_staff(
         manager_id=user_id,
         status_filter=status_filter,
         skip=skip,
         limit=limit
     )
+    return items
 
 
 @router.get(

app/staff/schemas/staff_schema.py

@@ -103,9 +103,9 @@ class StaffResponseSchema(BaseModel):
 class StaffListRequest(BaseModel):
     """Schema for staff list request with projection support."""
     merchant_id: Optional[str] = Field(None, description="Merchant identifier (optional, defaults to token merchant)")
-    designation: Optional[str] = Field(None, description="Filter by designation/role")
+    designation: Optional[Designation] = Field(None, description="Filter by designation/role")
     manager_id: Optional[str] = Field(None, description="Filter by manager's ID")
-    status: Optional[str] = Field(None, description="Filter by status")
+    status: Optional[Union[stafftatus, List[stafftatus]]] = Field(None, description="Filter by status")
     role: Optional[str] = Field(None, description="Filter by role")
     region: Optional[str] = Field(None, description="Filter by region")
     skip: int = Field(0, ge=0, description="Number of records to skip")
@@ -117,7 +117,7 @@ class StaffListRequest(BaseModel):
 
 class StaffListResponse(BaseModel):
     """Schema for staff list response."""
-    staff: List[StaffResponseSchema]  # Always StaffResponseSchema (fields are optional to support projection)
+    items: List[Union[EmployeeResponse, dict]]  # Can be full response or projected dict
     total: int
     skip: int
     limit: int

app/staff/services/staff_service.py

@@ -3,7 +3,7 @@ Staff service layer - business logic and database operations.
 Syncs staff data to both MongoDB and PostgreSQL (trans.pos_staff_ref).
 """
 from datetime import datetime
-from typing import Optional, List, Dict, Any
+from typing import Optional, List, Dict, Any, Union
 from fastapi import HTTPException, status
 import logging
 import secrets
@@ -190,54 +190,7 @@ class StaffService:
                 detail=f"Error updating staff: {str(e)}"
             )
 
-    @staticmethod
-    async def list_staff(
-        merchant_id: Optional[str] = None,
-        status: Optional[str] = None,
-        role: Optional[str] = None,
-        skip: int = 0,
-        limit: int = 100
-    ) -> tuple[List[StaffResponseSchema], int]:
-        """
-        List staff with optional filters.
-        
-        Args:
-            merchant_id: Filter by merchant
-            status: Filter by status
-            role: Filter by role
-            skip: Number of records to skip
-            limit: Maximum records to return
-            
-        Returns:
-            Tuple of (staff list, total count)
-        """
-        try:
-            # Build query
-            query = {}
-            if merchant_id:
-                query["merchant_id"] = merchant_id
-            if status:
-                query["status"] = status
-            if role:
-                query["role"] = role
 
-            # Get total count
-            total = await get_database()[POS_STAFF_COLLECTION].count_documents(query)
-
-            # Fetch staff
-            cursor = get_database()[POS_STAFF_COLLECTION].find(query).skip(skip).limit(limit)
-            staff_list = await cursor.to_list(length=limit)
-            
-            staff_responses = [StaffResponseSchema(**staff) for staff in staff_list]
-            
-            return staff_responses, total
-            
-        except Exception as e:
-            logger.error(f"Error listing staff: {e}")
-            raise HTTPException(
-                status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
-                detail="Error listing staff"
-            )
 
     @staticmethod
     async def delete_staff(staff_id: str) -> Dict[str, str]:
@@ -415,11 +368,12 @@ class StaffService:
     async def list_staff(
         designation: Optional[Designation] = None,
         manager_id: Optional[str] = None,
-        status_filter: Optional[stafftatus] = None,
+        status_filter: Optional[Union[stafftatus, List[stafftatus]]] = None,
         region: Optional[str] = None,
         skip: int = 0,
-        limit: int = 100
-    ) -> List[EmployeeResponse]:
+        limit: int = 100,
+        projection_list: Optional[List[str]] = None
+    ) -> tuple[List[Union[EmployeeResponse, dict]], int]:
         """List staff with filters (employee version)."""
         try:
             # Build query
@@ -428,16 +382,34 @@ class StaffService:
                 query["designation"] = designation
             if manager_id:
                 query["manager_id"] = manager_id
+            
             if status_filter:
-                query["status"] = status_filter
+                if isinstance(status_filter, list):
+                    query["status"] = {"$in": status_filter}
+                else:
+                    query["status"] = status_filter
+                    
             if region:
                 query["region"] = region
 
+            # Get total count
+            total = await get_database()[POS_STAFF_COLLECTION].count_documents(query)
+            
+            # Build projection
+            projection = None
+            if projection_list:
+                projection = {field: 1 for field in projection_list}
+                if "_id" not in projection_list:
+                    projection["_id"] = 0
+
             # Fetch staff
-            cursor = get_database()[POS_STAFF_COLLECTION].find(query).skip(skip).limit(limit)
+            cursor = get_database()[POS_STAFF_COLLECTION].find(query, projection).skip(skip).limit(limit)
             staff_list = await cursor.to_list(length=limit)
             
-            return [EmployeeResponse(**staff) for staff in staff_list]
+            if projection_list:
+                return staff_list, total
+            else:
+                return [EmployeeResponse(**staff) for staff in staff_list], total
             
         except Exception as e:
             logger.error(f"Error listing staff: {e}")

app/utils/response.py

@@ -1,7 +1,7 @@
 """
 Response formatting utilities for standardized API responses.
 """
-from typing import Any, Dict, Optional
+from typing import Any, Dict, Optional, List
 from datetime import datetime
 from uuid import uuid4
 
@@ -23,7 +23,7 @@ def success_response(
         Standardized success response dictionary
     """
     return {
-        "status": True,
+        "success": True,
         "data": data,
         "message": message,
         "correlation_id": correlation_id or str(uuid4()),
@@ -32,33 +32,45 @@ def success_response(
 
 
 def error_response(
-    code: str,
-    message: str,
-    details: Optional[Dict[str, Any]] = None,
-    correlation_id: Optional[str] = None
+    error: str,
+    detail: str,
+    errors: Optional[List[Dict[str, Any]]] = None,
+    code: Optional[str] = None, # Kept for backward compatibility if needed, but mapped to error title usually
+    request_id: Optional[str] = None,
+    headers: Optional[Dict[str, str]] = None
 ) -> Dict[str, Any]:
     """
-    Format an error API response.
+    Format an error API response according to the Error Handling Guide.
     
     Args:
-        code: Error code
-        message: Human-readable error message
-        details: Optional additional error details
-        correlation_id: Optional correlation ID for request tracing
+        error: Error title (e.g., "Validation Error", "Authentication Error")
+        detail: Human-readable error message
+        errors: Optional list of specific errors (e.g., validation fields)
+        code: Optional error code (legacy support, can be ignored or used as error title)
+        request_id: Optional request ID
+        headers: Optional headers (included in body as per guide example)
         
     Returns:
         Standardized error response dictionary
     """
-    return {
-        "status": False,
-        "error": {
-            "code": code,
-            "message": message,
-            "details": details or {}
-        },
-        "correlation_id": correlation_id or str(uuid4()),
-        "timestamp": datetime.utcnow().isoformat() + "Z"
+    response = {
+        "success": False,
+        "error": error or code or "Error",
+        "detail": detail
     }
+    
+    if errors:
+        response["errors"] = errors
+        
+    if request_id:
+        response["request_id"] = request_id
+        
+    if headers:
+        response["headers"] = headers
+        
+    response["timestamp"] = datetime.utcnow().isoformat() + "Z"
+    
+    return response
 
 
 def paginated_response(

requirements.txt

@@ -28,3 +28,4 @@ twilio==8.10.3
 aiosmtplib==3.0.1
 
 python-json-logger==2.0.7
+

tests/test_error_handling.py

@@ -0,0 +1,34 @@
+from unittest.mock import patch, AsyncMock
+from fastapi.testclient import TestClient
+
+# Mock database connections before importing app
+with patch("app.main.connect_to_mongo", new_callable=AsyncMock), \
+     patch("app.main.connect_to_database", new_callable=AsyncMock):
+    from app.main import app
+
+client = TestClient(app)
+
+def test_404_error_handling():
+    response = client.get("/non-existent-route")
+    assert response.status_code == 404
+    data = response.json()
+    assert data["success"] is False
+    assert data["error"] == "Not Found"
+    assert data["detail"] == "Resource not found"
+    assert "request_id" in data
+    assert "timestamp" in data
+
+def test_validation_error_handling():
+    # token query param is required for this endpoint
+    response = client.post("/debug/verify-token")
+    assert response.status_code == 422
+    data = response.json()
+    assert data["success"] is False
+    assert data["error"] == "Validation Error"
+    assert data["detail"] == "The request contains invalid data"
+    assert "errors" in data
+    assert isinstance(data["errors"], list)
+    assert len(data["errors"]) > 0
+    assert "field" in data["errors"][0]
+    assert "message" in data["errors"][0]
+    assert "type" in data["errors"][0]

tests/test_staff_list.py

@@ -0,0 +1,72 @@
+from unittest.mock import patch, AsyncMock
+from fastapi.testclient import TestClient
+from app.constants.staff_types import Designation, stafftatus
+
+# Mock database connections before importing app
+with patch("app.main.connect_to_mongo", new_callable=AsyncMock), \
+     patch("app.main.connect_to_database", new_callable=AsyncMock):
+    from app.main import app
+
+client = TestClient(app)
+
+@patch("app.staff.services.staff_service.StaffService.list_staff")
+def test_staff_list_endpoint(mock_list_staff):
+    # Setup mock return value
+    mock_items = [
+        {
+            "user_id": "u1",
+            "name": "Test User",
+            "designation": "Stylist",
+            "status": "active",
+            "email": "test@example.com",
+            "phone": "1234567890",
+            "role": "Stylist",
+            "merchant_id": "m1"
+        }
+    ]
+    mock_list_staff.return_value = (mock_items, 1)
+
+    # Test payload
+    payload = {
+        "status": "active",
+        "skip": 0,
+        "limit": 10,
+        "projection_list": ["name", "email"]
+    }
+
+    response = client.post("/api/v1/staff/list", json=payload)
+    
+    # Assertions
+    assert response.status_code == 200
+    data = response.json()
+    assert data["total"] == 1
+    assert len(data["items"]) == 1
+    assert data["items"][0]["name"] == "Test User"
+    
+    # Verify mock call
+    mock_list_staff.assert_called_once()
+    call_kwargs = mock_list_staff.call_args.kwargs
+    assert call_kwargs["status_filter"] == "active"
+    assert call_kwargs["projection_list"] == ["name", "email"]
+
+@patch("app.staff.services.staff_service.StaffService.list_staff")
+def test_staff_list_with_list_status(mock_list_staff):
+    # Setup mock return value
+    mock_list_staff.return_value = ([], 0)
+
+    # Test payload with list of statuses
+    payload = {
+        "status": ["active", "inactive"],
+        "skip": 0,
+        "limit": 10
+    }
+
+    response = client.post("/api/v1/staff/list", json=payload)
+    
+    # Assertions
+    assert response.status_code == 200
+    
+    # Verify mock call
+    mock_list_staff.assert_called_once()
+    call_kwargs = mock_list_staff.call_args.kwargs
+    assert call_kwargs["status_filter"] == ["active", "inactive"]