# Deployment Module

## Abstract

The `deploy/` directory contains the infrastructure-as-code definitions required to deploy the Portfolio Engine as a production-grade, containerised microservice on Kubernetes. The deployment architecture follows a stateless application pattern backed by managed PostgreSQL and Redis services, orchestrated via Helm charts and Docker Compose for local development parity.

---

## 1. Architectural Overview

The production deployment comprises three services:

```
┌─────────────────────────────────────────────────────────────────┐
│                     Kubernetes Cluster                          │
│                                                                 │
│  ┌──────────────────┐  ┌──────────────┐  ┌──────────────────┐  │
│  │  portfolio-engine │  │  PostgreSQL  │  │      Redis       │  │
│  │  (Python 3.11)   │──│  (v15)       │  │  (v7)            │  │
│  │  Uvicorn / FastAPI│  │  Persistent  │  │  Ephemeral Cache │  │
│  │  Port 8080       │  │  Port 5432   │  │  Port 6379       │  │
│  └──────────────────┘  └──────────────┘  └──────────────────┘  │
└─────────────────────────────────────────────────────────────────┘
```

- **Portfolio Engine:** Stateless Python application serving the FastAPI-based REST API via Uvicorn. Handles all optimisation, backtesting, and reporting logic.
- **PostgreSQL 15:** Persistent storage for daily price data, portfolio state, and transaction history. The engine writes to the `daily_prices` table and reads during the data-loading phase.
- **Redis 7:** Optional caching layer for rate-limited API responses and inter-process synchronisation in multi-replica deployments.

---

## 2. Container Image — `Dockerfile`

```dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 8080
CMD ["python", "core_engine.py", "--headless"]
```

**Design Decisions:**

- **Base image:** `python:3.11-slim` minimises attack surface and image size while retaining the `manylinux` wheels required by NumPy, SciPy, and CVXPY.
- **Dependency installation before source copy:** Exploits Docker layer caching. Dependencies change infrequently; source code changes do not invalidate the pip install layer.
- **Headless mode:** The default entrypoint runs the engine in non-interactive mode, suitable for scheduled batch jobs. For API serving, override with `uvicorn api:app --host 0.0.0.0 --port 8080`.

---

## 3. Local Development — `docker-compose.yml`

The Compose file provisions all three services for local development with production-equivalent topology.

### Service Configuration

| Service   | Image          | Ports    | Health Check                                          |
|-----------|----------------|----------|-------------------------------------------------------|
| `db`      | `postgres:15`  | 5432     | `pg_isready -U engine_user -d portfolio_db`           |
| `redis`   | `redis:7`      | 6379     | `redis-cli ping`                                      |
| `engine`  | Build from `.` | 8080     | `curl -f http://localhost:8080/health`                |

### Dependency Ordering

The engine container specifies `depends_on` with `condition: service_healthy`, ensuring that PostgreSQL and Redis are fully operational before the application starts. This prevents race conditions during cold-start data loading.

### Environment Variables

| Variable       | Value                                                            |
|----------------|------------------------------------------------------------------|
| `DATABASE_URL` | `postgresql://engine_user:engine_password@db:5432/portfolio_db`  |
| `REDIS_URL`    | `redis://redis:6379/0`                                           |

**Security Note.** The Compose file uses plaintext credentials for local development only. Production deployments must inject secrets via Kubernetes Secrets, HashiCorp Vault, or an equivalent secrets manager.

### Usage

```bash
# Build and start all services
docker-compose up --build

# Run in detached mode
docker-compose up -d

# View logs
docker-compose logs -f engine

# Tear down
docker-compose down -v
```

---

## 4. Kubernetes Deployment — Helm Chart

### 4.1 Chart Metadata — `Chart.yaml`

| Field          | Value                                  |
|----------------|----------------------------------------|
| `apiVersion`   | `v2` (Helm 3)                          |
| `name`         | `portfolio-engine`                     |
| `description`  | Quantitative Portfolio Allocation Engine |
| `type`         | `application`                          |
| `version`      | `0.1.0` (chart version)               |
| `appVersion`   | `1.0` (application version)            |

### 4.2 Configurable Values — `values.yaml`

| Parameter               | Default              | Description                                    |
|-------------------------|----------------------|------------------------------------------------|
| `replicaCount`          | `2`                  | Number of engine pod replicas                  |
| `image.repository`      | `portfolio-engine`   | Container registry path                        |
| `image.pullPolicy`      | `IfNotPresent`       | Image pull policy                              |
| `image.tag`             | `latest`             | Image tag                                      |
| `service.type`          | `ClusterIP`          | Kubernetes service type                        |
| `service.port`          | `8080`               | Service port                                   |
| `env[0] (API_KEY)`      | `changeme`           | External data API key                          |
| `env[1] (DATABASE_URL)` | (empty)              | PostgreSQL connection string                   |
| `resources.limits.cpu`  | `2000m`              | CPU limit per pod                              |
| `resources.limits.memory` | `4Gi`              | Memory limit per pod                           |
| `resources.requests.cpu` | `500m`              | CPU request per pod                            |
| `resources.requests.memory` | `1Gi`            | Memory request per pod                         |

**Resource Sizing Rationale.** The 4Gi memory limit accommodates the in-memory covariance matrices and Monte Carlo simulation arrays for portfolios of up to ~200 assets. The 2-core CPU limit supports CVXPY's OSQP solver, which benefits from vectorised linear algebra but does not parallelise across cores.

### 4.3 Templates

| Template             | Purpose                                                              |
|----------------------|----------------------------------------------------------------------|
| `_helpers.tpl`       | Reusable Go template functions for generating consistent names and labels |
| `deployment.yaml`    | Kubernetes Deployment resource defining pod spec, container image, ports, environment variables, and resource constraints |
| `service.yaml`       | Kubernetes Service resource exposing the engine pods via ClusterIP    |

### 4.4 Deployment Commands

```bash
# Install the chart
helm install portfolio-engine ./deploy/helm/portfolio-engine \
  --set env[0].value="YOUR_API_KEY" \
  --set env[1].value="postgresql://user:pass@host:5432/db"

# Upgrade an existing release
helm upgrade portfolio-engine ./deploy/helm/portfolio-engine \
  --set image.tag="v2.0"

# Uninstall
helm uninstall portfolio-engine

# Dry-run to preview generated manifests
helm template portfolio-engine ./deploy/helm/portfolio-engine
```

---

## 5. CI/CD Pipeline — `.github/workflows/ci.yml`

The GitHub Actions workflow runs on every push and pull request to `main`:

| Step                  | Tool         | Purpose                                              |
|-----------------------|--------------|------------------------------------------------------|
| Checkout              | `actions/checkout@v3` | Clone repository                              |
| Python Setup          | `actions/setup-python@v4` | Install Python 3.11                       |
| Install Dependencies  | `pip install .[dev]`  | Install production and development dependencies |
| Lint                  | `black --check .`     | Enforce PEP 8 code formatting                |
| Type Check            | `mypy .`              | Static type analysis                         |
| Test                  | `pytest tests/`       | Run the full test suite                      |

**Pipeline Philosophy.** The CI pipeline enforces a "green trunk" policy: no merge is permitted unless all linting, typing, and testing gates pass. This is consistent with the engine's design principle of no overfitting—code quality gates prevent regressions that could silently corrupt optimisation results.

---

## 6. Production Considerations

### Scaling

The engine is designed for horizontal scaling behind a load balancer. Each pod is stateless; all persistent state resides in PostgreSQL. Redis can be used for distributed rate-limiting across replicas to honour external API quotas (max 2 requests/second globally).

### Observability

The engine emits structured JSON logs (see `output/engine.log`) compatible with ELK, Datadog, or Grafana Loki. Health check endpoints at `/health` support Kubernetes liveness and readiness probes.

### Security

- API keys must never be committed to version control. Use Kubernetes Secrets or a secrets manager.
- The `DATABASE_URL` should use TLS-encrypted connections (`sslmode=require`) in production.
- Container images should be scanned for CVEs before deployment (e.g., Trivy, Snyk Container).

---

## References

- Kubernetes Documentation. (2024). *Deployments*. https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
- Helm Documentation. (2024). *Chart Template Guide*. https://helm.sh/docs/chart_template_guide/
- Docker Documentation. (2024). *Best practices for writing Dockerfiles*. https://docs.docker.com/develop/develop-images/dockerfile_best-practices/