swiftops-backend / docs /dev /orgs /PROJECT_SUMMARY.md
kamau1's picture
chore: migrate to useast organize the docs, delete redundant migrations
c4f7e3e

SwiftOps Backend - Project Summary

πŸ“‹ Overview

SwiftOps is a comprehensive field service management platform designed for telecom operators and contractors in Kenya and East Africa. The backend is built with FastAPI, following clean architecture principles for maximum maintainability and scalability.


🎯 Core Business Domains

1. Ticket Management

  • Create tickets from sales orders, incidents, or infrastructure tasks
  • Assign tickets to field agents with business rule validation
  • Track ticket lifecycle (open β†’ assigned β†’ in_progress β†’ completed)
  • GPS-based arrival verification
  • SLA monitoring and deadline tracking

2. Payroll Processing

  • Automated weekly payroll generation
  • Multiple compensation models (flat rate, commission, hybrid, hourly)
  • Deduction handling (advances, penalties)
  • Payment tracking and reconciliation

3. Inventory Management

  • Multi-level inventory distribution (main office β†’ regional hubs β†’ field agents)
  • Serial number tracking
  • Optimistic locking for concurrent updates
  • Equipment vs tools vs consumables handling

4. Financial Management

  • Project-level cash flow tracking
  • Approval workflows (pending β†’ approved β†’ paid)
  • Payment gateway integration (M-Pesa, bank transfers)
  • Expense tracking and reconciliation

5. Customer & Subscription Management

  • Customer onboarding from sales orders
  • Subscription activation with dynamic requirements
  • Service lifecycle management
  • Support incident tracking

6. Location Tracking

  • Real-time GPS tracking via WebSocket
  • Journey tracking (breadcrumb trail)
  • Distance calculations
  • Fraud detection (impossible travel speeds)

πŸ—οΈ Technical Architecture

Architecture Pattern: Clean Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                    Presentation Layer                        β”‚
β”‚              (FastAPI Routes, WebSockets)                    β”‚
β”‚  - Input validation (Pydantic)                              β”‚
β”‚  - Authentication/Authorization                              β”‚
β”‚  - Response formatting                                       β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                            ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                      Service Layer                           β”‚
β”‚           (Business Logic, Orchestration)                    β”‚
β”‚  - Business rule validation                                  β”‚
β”‚  - Transaction coordination                                  β”‚
β”‚  - Side effect triggering (notifications, webhooks)         β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                            ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                    Repository Layer                          β”‚
β”‚              (Data Access, Queries)                          β”‚
β”‚  - CRUD operations                                           β”‚
β”‚  - Complex queries                                           β”‚
β”‚  - Soft delete handling                                      β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                            ↓
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                      Database Layer                          β”‚
β”‚            (PostgreSQL via Supabase)                         β”‚
β”‚  - Data persistence                                          β”‚
β”‚  - Row-level security                                        β”‚
β”‚  - Constraints and indexes                                   β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Technology Stack

Component Technology Purpose
API Framework FastAPI High-performance async API
Database PostgreSQL (Supabase) Primary data store
ORM SQLAlchemy 2.0 Database abstraction
Caching Redis Session storage, rate limiting
Background Jobs Celery + Redis Async task processing
Authentication Supabase Auth + JWT User authentication
File Storage Supabase Storage / S3 Document uploads
Payment Gateway M-Pesa API Mobile money payments
SMS Provider Africa's Talking SMS notifications
Email Provider SendGrid Email notifications
Maps API Google Maps Geocoding, distance calculation
Monitoring Sentry Error tracking
Testing Pytest Unit/integration tests

πŸ“‚ Folder Structure Philosophy

Why This Structure?

  1. Separation of Concerns: Each layer has a single responsibility
  2. Testability: Easy to mock dependencies and test in isolation
  3. Scalability: Can add new features without touching existing code
  4. Maintainability: New developers can quickly understand the codebase
  5. Flexibility: Can swap implementations (e.g., change payment gateway)

Key Folders

app/
β”œβ”€β”€ api/              # HTTP endpoints (thin layer)
β”œβ”€β”€ services/         # Business logic (thick layer)
β”œβ”€β”€ repositories/     # Data access (thin layer)
β”œβ”€β”€ models/           # Database models (SQLAlchemy)
β”œβ”€β”€ schemas/          # Request/response schemas (Pydantic)
β”œβ”€β”€ tasks/            # Background tasks (Celery)
β”œβ”€β”€ integrations/     # External services (M-Pesa, SMS, etc.)
└── utils/            # Helper functions

πŸ”„ Key Workflows

Workflow 1: Ticket Assignment

1. API receives POST /api/v1/tickets/{id}/assign
2. Route validates JWT token and request body
3. Route calls TicketService.assign_ticket()
4. Service validates business rules:
   - User has < 3 active assignments
   - User is in project team
   - User's region matches ticket region
5. Service calls TicketRepository.get_by_id()
6. Service calls AssignmentRepository.create()
7. Service calls TicketRepository.update_status()
8. Service calls SLAService.calculate_deadline()
9. Service calls NotificationService.send_notification()
10. Service returns result to route
11. Route formats response and returns to client

Workflow 2: Weekly Payroll Generation

1. Celery Beat triggers task (Friday 6 PM)
2. Task calls PayrollService.generate_weekly_payroll()
3. Service gets all active projects
4. For each project:
   a. Get project team members
   b. For each member:
      - Get tickets closed in period
      - Get timesheet hours
      - Calculate earnings based on compensation type
      - Apply deductions
      - Create payroll record
      - Create finance transaction
      - Send notification to user
5. Task logs completion

Workflow 3: Payment Processing

1. Manager approves expense in UI
2. API receives POST /api/v1/finance/{id}/approve
3. Service validates approver has permission
4. Service updates transaction status to 'approved'
5. Service calls PaymentService.initiate_payment()
6. PaymentService calls M-Pesa API
7. M-Pesa returns transaction ID
8. Service creates PaymentLog entry
9. M-Pesa sends webhook callback (async)
10. Webhook handler updates transaction status
11. Service sends notification to recipient

πŸ” Security Architecture

Multi-Layer Security

  1. Authentication Layer

    • Supabase Auth for user authentication
    • JWT tokens for API access
    • Token refresh mechanism
  2. Authorization Layer

    • Role-based access control (RBAC)
    • Row-level security (RLS) at database level
    • Multi-tenancy isolation (client/contractor scoping)
  3. Data Protection

    • Password hashing (bcrypt)
    • Sensitive data encryption
    • Audit logging for financial transactions
  4. API Security

    • Rate limiting (per user/client)
    • CORS configuration
    • Input validation (Pydantic)
    • SQL injection prevention (ORM)

Multi-Tenancy Model

Platform
β”œβ”€β”€ Client A (Safaricom)
β”‚   β”œβ”€β”€ Project 1
β”‚   β”‚   β”œβ”€β”€ Contractor X
β”‚   β”‚   β”‚   └── Workers
β”‚   β”‚   └── Contractor Y
β”‚   β”‚       └── Workers
β”‚   └── Project 2
β”‚       └── Contractor Z
β”‚           └── Workers
└── Client B (Airtel)
    └── Project 3
        └── Contractor X
            └── Workers

Isolation Strategy:

  • Users belong to either a Client OR a Contractor
  • Projects link Client + Contractor
  • All data scoped through Projects
  • RLS policies enforce tenant boundaries

πŸ“Š Data Flow Patterns

Pattern 1: Repository Pattern

Purpose: Separate data access from business logic

# Repository (data access)
class TicketRepository:
    def get_by_id(self, ticket_id: UUID) -> Ticket:
        return db.query(Ticket).filter(Ticket.id == ticket_id).first()

# Service (business logic)
class TicketService:
    def assign_ticket(self, ticket_id: UUID, user_id: UUID):
        ticket = self.ticket_repo.get_by_id(ticket_id)
        # Business logic here

Pattern 2: Dependency Injection

Purpose: Loose coupling, easy testing

# Define dependencies
def get_ticket_service(
    ticket_repo: TicketRepository = Depends(get_ticket_repo),
    notification_service: NotificationService = Depends(get_notification_service)
) -> TicketService:
    return TicketService(ticket_repo, notification_service)

# Use in route
@router.post("/tickets/{id}/assign")
async def assign_ticket(
    ticket_id: UUID,
    service: TicketService = Depends(get_ticket_service)
):
    return await service.assign_ticket(ticket_id)

Pattern 3: Service Layer Orchestration

Purpose: Coordinate multiple operations, enforce business rules

class TicketService:
    async def assign_ticket(self, ticket_id: UUID, user_id: UUID):
        # 1. Validate
        ticket = await self._validate_ticket(ticket_id)
        await self._validate_user(user_id, ticket.project_id)
        
        # 2. Execute
        assignment = await self._create_assignment(ticket_id, user_id)
        await self._update_ticket_status(ticket_id, 'assigned')
        
        # 3. Side effects
        await self._calculate_sla(ticket_id)
        await self._send_notification(user_id, ticket)
        
        return assignment

πŸ§ͺ Testing Strategy

Test Pyramid

        E2E (5%)
      Integration (25%)
    Unit Tests (70%)

Test Coverage Goals

  • Unit Tests: 80%+ coverage
  • Integration Tests: Critical paths covered
  • E2E Tests: Main user workflows

Testing Tools

  • pytest: Test framework
  • pytest-asyncio: Async test support
  • pytest-cov: Coverage reporting
  • faker: Test data generation
  • factory-boy: Model factories

πŸš€ Deployment Strategy

Environments

  1. Development (Local)

    • Docker Compose
    • Hot reload enabled
    • Debug logging
  2. Staging

    • Kubernetes/Docker Swarm
    • Production-like setup
    • E2E tests run here
  3. Production

    • Kubernetes/Docker Swarm
    • Auto-scaling enabled
    • Monitoring and alerts

CI/CD Pipeline

1. Code Push β†’ GitHub
2. Run Tests (pytest)
3. Check Coverage (> 80%)
4. Run Linters (black, flake8, mypy)
5. Build Docker Image
6. Push to Registry
7. Deploy to Staging
8. Run E2E Tests
9. Deploy to Production (manual approval)
10. Health Check

πŸ“ˆ Scalability Considerations

Horizontal Scaling

  • Stateless API: Can run multiple instances
  • Load Balancer: Distribute traffic across instances
  • Session Storage: Redis (shared across instances)

Database Optimization

  • Connection Pooling: SQLAlchemy pool (20 connections)
  • Read Replicas: For reporting queries
  • Indexes: Strategic indexes on all foreign keys
  • Query Optimization: Avoid N+1 queries

Caching Strategy

  • L1 Cache: In-memory (per instance)
  • L2 Cache: Redis (shared)
  • Cache Invalidation: On data updates

Background Jobs

  • Celery Workers: Scale independently
  • Task Queues: Separate queues for priority
  • Retry Logic: Exponential backoff

πŸ”§ Development Workflow

Getting Started

  1. Clone repository
  2. Copy .env.example to .env
  3. Run docker-compose up -d
  4. Run migrations: alembic upgrade head
  5. Access API docs: http://localhost:8000/docs

Adding a New Feature

  1. Create feature branch
  2. Add models (if needed)
  3. Create migration: alembic revision --autogenerate
  4. Add repository methods
  5. Add service methods
  6. Add API routes
  7. Write tests
  8. Submit PR

Code Quality Checks

# Format code
black app/

# Sort imports
isort app/

# Lint
flake8 app/

# Type check
mypy app/

# Run tests
pytest --cov=app

πŸ“š Documentation

Available Docs

  1. README.md: Project overview and setup
  2. ARCHITECTURE.md: Detailed architecture guide
  3. QUICKSTART.md: 5-minute setup guide
  4. API Docs: http://localhost:8000/docs (auto-generated)

Code Documentation

  • Docstrings for all public methods
  • Type hints for all functions
  • Comments for complex business logic

🀝 Team Collaboration

Code Review Checklist

  • Tests added/updated
  • Documentation updated
  • No hardcoded secrets
  • Error handling implemented
  • Logging added
  • Performance considered
  • Security reviewed

Git Workflow

main (production)
  ↑
develop (staging)
  ↑
feature/ticket-assignment
feature/payroll-generation

πŸ“ž Support & Resources

Getting Help

  • Documentation: See README.md and ARCHITECTURE.md
  • API Docs: http://localhost:8000/docs
  • Team Chat: [Your communication channel]
  • Issues: GitHub Issues

External Resources


βœ… Project Status

Current Phase: Development

Completed

  • βœ… Database schema design
  • βœ… Architecture documentation
  • βœ… Project structure setup
  • βœ… Docker configuration

In Progress

  • πŸ”„ Core API implementation
  • πŸ”„ Service layer development
  • πŸ”„ Integration with Supabase
  • πŸ”„ Payment gateway integration

Upcoming

  • ⏳ Background task implementation
  • ⏳ Testing suite
  • ⏳ Deployment setup
  • ⏳ Monitoring and logging

🎯 Success Metrics

Technical Metrics

  • API response time < 200ms (p95)
  • Test coverage > 80%
  • Zero critical security vulnerabilities
  • 99.9% uptime

Business Metrics

  • Ticket completion time
  • SLA compliance rate
  • Payroll processing accuracy
  • Agent productivity

Last Updated: January 2025
Version: 1.0.0
Maintainers: [Your Team]