CATALOGUE_SERVICES_META_IMPLEMENTATION.md

Catalogue Services: Meta Schema Implementation

Overview

This document describes the implementation of the Meta Schema Pattern for the Catalogue Services module in the POS microservice. This implementation follows the same pattern used in Customers and Staff modules.

Implementation Date

December 2024

Module Updated

Catalogue Services Module (app/catalogue_services/)

Changes Made

1. Schema (app/catalogue_services/schemas/schema.py)

Added Import

from typing import Optional, List, Union, Dict, Any

Updated CreateServiceRequest

Added username tracking field:

class CreateServiceRequest(BaseModel):
    merchant_id: Optional[str] = None
    name: str = Field(..., min_length=1, max_length=200)
    code: str = Field(..., min_length=1, max_length=50)
    # ... other fields ...
    created_by_username: Optional[str] = Field(None, description="Username who created - will be set from token")

Updated UpdateServiceRequest

Added username tracking field:

class UpdateServiceRequest(BaseModel):
    # ... existing fields ...
    updated_by_username: Optional[str] = Field(None, description="Username who updated - will be set from token")

Updated ServiceResponse

Replaced timestamp fields with meta:

class ServiceResponse(BaseModel):
    service_id: str
    merchant_id: str
    name: str
    code: str
    category: Optional[dict] = None
    description: Optional[str]
    duration_mins: int
    pricing: dict
    status: str
    sort_order: int = 0
    meta: Dict[str, Any] = Field(..., description="Audit information (created_by, created_at, updated_by, updated_at)")

2. Service Layer (app/catalogue_services/services/service.py)

Added Imports

from app.core.utils import format_meta_field, extract_audit_fields

Updated create_service

• Added user_id and username parameters
• Removed manual timestamp creation
• Uses extract_audit_fields(is_update=False) to add audit fields
• Returns document with audit fields

Before:

async def create_service(
    merchant_id,
    name: str,
    code: str,
    # ... other params ...
) -> dict:
    now = datetime.utcnow()
    doc = {
        # ... fields ...
        "created_at": now,
        "updated_at": now
    }

After:

async def create_service(
    merchant_id,
    name: str,
    code: str,
    # ... other params ...
    user_id: Optional[str] = None,
    username: Optional[str] = None,
) -> dict:
    doc = {
        # ... fields ...
    }
    
    # Add audit fields
    audit_fields = extract_audit_fields(
        user_id=user_id or "system",
        username=username or "system",
        is_update=False
    )
    doc.update(audit_fields)

Updated update_service

• Added user_id and username parameters
• Removed manual updated_at timestamp
• Uses extract_audit_fields(is_update=True) to add update audit fields

Before:

async def update_service(
    service_id: str,
    # ... params ...
) -> dict:
    update_data = {"updated_at": datetime.utcnow()}
    # ... build update ...

After:

async def update_service(
    service_id: str,
    # ... params ...
    user_id: Optional[str] = None,
    username: Optional[str] = None,
) -> dict:
    update_data = {}
    # ... build update ...
    
    # Add audit fields for update
    audit_fields = extract_audit_fields(
        user_id=user_id or "system",
        username=username or "system",
        is_update=True
    )
    update_data.update(audit_fields)

Updated update_status

• Added user_id and username parameters
• Uses extract_audit_fields(is_update=True) for status changes

Updated delete_service

• Added user_id and username parameters
• Passes them to update_status (soft delete via archive)

Updated list_services

When no projection is used, formats each item with meta:

# No projection - convert to response format with meta
item_copy = item.copy()
item_copy["service_id"] = item["_id"]
formatted = format_meta_field(item_copy)

result_item = {
    "service_id": item["_id"],
    "merchant_id": item["merchant_id"],
    # ... other fields ...
    "meta": formatted["meta"],
}

3. Controller (app/catalogue_services/controllers/router.py)

Added Import

from app.core.utils import format_meta_field

Updated create_service_endpoint

Passes user info from token:

@router.post("", response_model=ServiceResponse, status_code=status.HTTP_201_CREATED)
async def create_service_endpoint(
    req: CreateServiceRequest,
    current_user: TokenUser = Depends(require_pos_permission("retail_catalogue", "create"))
):
    # Set username for audit trail
    req.created_by_username = current_user.username
    
    doc = await create_service(
        # ... params ...
        user_id=current_user.user_id,
        username=current_user.username
    )

Updated update_service_endpoint

Passes user info from token:

@router.put("/{service_id}", response_model=ServiceResponse)
async def update_service_endpoint(
    service_id: str,
    req: UpdateServiceRequest,
    current_user: TokenUser = Depends(require_pos_permission("retail_catalogue", "update"))
):
    # Set username for audit trail
    req.updated_by_username = current_user.username
    
    doc = await update_service(
        service_id=service_id,
        # ... params ...
        user_id=current_user.user_id,
        username=current_user.username
    )

Updated update_status_endpoint

Passes user info:

doc = await update_status(
    service_id, 
    req.status,
    user_id=current_user.user_id,
    username=current_user.username
)

Updated delete_service_endpoint

Passes user info:

doc = await delete_service(
    service_id,
    user_id=current_user.user_id,
    username=current_user.username
)

Updated _to_response Helper

Uses format_meta_field to transform document:

def _to_response(doc: dict) -> ServiceResponse:
    # Ensure service_id field is set for format_meta_field
    doc_copy = doc.copy()
    doc_copy["service_id"] = doc["_id"]
    
    # Format with meta
    formatted = format_meta_field(doc_copy)
    
    return ServiceResponse(
        service_id=doc["_id"],
        merchant_id=doc["merchant_id"],
        # ... other fields ...
        meta=formatted["meta"],
    )

API Response Changes

Before (Old Format)

{
  "service_id": "srv_123",
  "merchant_id": "merch_456",
  "name": "Haircut - Premium",
  "code": "HAIRCUT_PREMIUM",
  "category": {
    "id": "cat_789",
    "name": "Hair Services"
  },
  "description": "Premium haircut service",
  "duration_mins": 45,
  "pricing": {
    "price": 500.0,
    "currency": "INR",
    "gst_rate": 18.0
  },
  "status": "active",
  "sort_order": 0,
  "created_at": "2024-01-15T10:30:00Z",
  "updated_at": "2024-12-01T14:20:00Z"
}

After (Meta Format)

{
  "service_id": "srv_123",
  "merchant_id": "merch_456",
  "name": "Haircut - Premium",
  "code": "HAIRCUT_PREMIUM",
  "category": {
    "id": "cat_789",
    "name": "Hair Services"
  },
  "description": "Premium haircut service",
  "duration_mins": 45,
  "pricing": {
    "price": 500.0,
    "currency": "INR",
    "gst_rate": 18.0
  },
  "status": "active",
  "sort_order": 0,
  "meta": {
    "created_by": "admin",
    "created_by_id": "usr_01HZQX5K3N2P8R6T4V9W",
    "created_at": "2024-01-15T10:30:00Z",
    "updated_by": "manager",
    "updated_by_id": "usr_01HZQX5K3N2P8R6T4V9X",
    "updated_at": "2024-12-01T14:20:00Z"
  }
}

Database Structure

The MongoDB documents maintain a flat structure with audit fields:

{
  "_id": "srv_123",
  "merchant_id": "merch_456",
  "name": "Haircut - Premium",
  "code": "HAIRCUT_PREMIUM",
  "category": {
    "id": "cat_789",
    "name": "Hair Services"
  },
  "description": "Premium haircut service",
  "duration_mins": 45,
  "pricing": {
    "price": 500.0,
    "currency": "INR",
    "gst_rate": 18.0
  },
  "status": "active",
  "sort_order": 0,
  "created_by": "usr_01HZQX5K3N2P8R6T4V9W",
  "created_by_username": "admin",
  "created_at": "2024-01-15T10:30:00Z",
  "updated_by": "usr_01HZQX5K3N2P8R6T4V9X",
  "updated_by_username": "manager",
  "updated_at": "2024-12-01T14:20:00Z"
}

The format_meta_field utility transforms this into the API response format with nested meta object.

Benefits

1. Audit Trail

Complete tracking of who created and modified each service:

• Human-readable usernames (created_by, updated_by)
• System IDs for lookups (created_by_id, updated_by_id)
• Timestamps for both creation and updates

2. Consistency

Same meta structure as Customers and Staff modules, providing uniform API responses across all POS entities.

3. Clean Responses

Audit information is grouped under 'meta', keeping the main response body focused on service data.

4. Query Performance

Database documents maintain flat structure, allowing efficient queries on audit fields when needed.

5. Backwards Compatible Storage

Existing documents in MongoDB will work (audit fields are optional in format_meta_field).

PostgreSQL Sync

The catalogue service syncs to trans.catalogue_service_ref table. The sync function uses the MongoDB document's audit fields but doesn't currently sync audit information to PostgreSQL (only business data: service name, price, category, etc.).

If PostgreSQL audit tracking is needed in the future, the _sync_to_postgres function can be updated to include:

await session.execute(text(
    """
    INSERT INTO trans.catalogue_service_ref (
      service_id, merchant_id, service_name, category,
      duration_mins, price, gst_rate, status, 
      created_by, created_at, updated_by, updated_at
    ) VALUES (
      :sid, :mid, :name, :cat, :dur, :price, :gst, :status,
      :created_by, :created_at, :updated_by, :updated_at
    )
    ...
    """
), {
    # ... existing params ...
    "created_by": service_doc.get("created_by_username"),
    "created_at": service_doc.get("created_at"),
    "updated_by": service_doc.get("updated_by_username"),
    "updated_at": service_doc.get("updated_at"),
})

List Endpoint Behavior

Without Projection

Returns full service objects with meta:

{
  "items": [
    {
      "service_id": "srv_123",
      "name": "Haircut",
      "meta": {
        "created_by": "admin",
        "created_at": "2024-01-15T10:30:00Z",
        ...
      }
    }
  ],
  "total": 10
}

With Projection

Returns only requested fields (raw format, no meta transformation):

{
  "items": [
    {
      "service_id": "srv_123",
      "name": "Haircut",
      "price": 500.0
    }
  ],
  "total": 10
}

This optimization allows clients to request minimal data for performance when audit information isn't needed.

Testing

Test Create Endpoint

POST /pos/catalogue/services
{
  "name": "Test Service",
  "code": "TEST_001",
  "duration_mins": 30,
  "pricing": {
    "price": 100.0,
    "gst_rate": 18.0
  }
}

# Response should include meta.created_by with username from JWT

Test Update Endpoint

PUT /pos/catalogue/services/{service_id}
{
  "name": "Updated Service Name"
}

# Response should include meta.updated_by with username from JWT

Test Status Update

PATCH /pos/catalogue/services/{service_id}/status
{
  "status": "inactive"
}

# Response should include meta.updated_by and meta.updated_at

Test Delete (Archive)

DELETE /pos/catalogue/services/{service_id}

# Response should show status="archived" and updated meta

Migration Notes

Existing Data

Existing services in MongoDB without audit fields will still work. The format_meta_field utility handles missing fields gracefully:

• Missing created_by/username → omitted from meta
• Missing updated_by/username → omitted from meta
• Missing timestamps → omitted from meta

New Services

All new services created after this implementation will have complete audit information.

Gradual Migration

If you want to add audit fields to existing services, you can run a migration script:

from datetime import datetime

async def migrate_existing_services():
    db = get_database()
    cursor = db[CATALOGUE_SERVICES_COLLECTION].find({
        "created_by": {"$exists": False}
    })
    
    async for doc in cursor:
        await db[CATALOGUE_SERVICES_COLLECTION].update_one(
            {"_id": doc["_id"]},
            {"$set": {
                "created_by": "system",
                "created_by_username": "migration",
                "created_at": doc.get("created_at", datetime.utcnow()),
                "updated_at": doc.get("updated_at", datetime.utcnow())
            }}
        )

Related Documentation

• POS Meta Schema Implementation - Overall POS microservice meta implementation
• Core README - Usage guide for core schemas and utilities
• SCM Meta Implementation - Similar implementation in SCM microservice

Summary

The Catalogue Services module now follows the same meta schema pattern as Customers and Staff modules, providing:

• ✅ Consistent audit trail across all POS entities
• ✅ Human-readable usernames in audit logs
• ✅ Clean, grouped meta structure in API responses
• ✅ Backwards compatible with existing data
• ✅ Efficient database queries with flat structure
• ✅ Reusable utilities from core module

All CRUD operations (create, read, update, delete/archive, list) now properly track and return audit information through the meta object.