docs/TESTS.md

Test Suite

Abstract

The tests/ directory contains the automated verification suite for the Portfolio Engine. The suite is organised into unit tests, property-based tests, and integration tests, covering the optimisation solver, risk analytics, data pipelines, econometric validation, and end-to-end simulation. All tests are executed via pytest and are gated in the CI/CD pipeline to enforce a green-trunk development policy.

---

1. Design Principles

1.1 No Overfitting to Tests

The test suite is designed to verify mathematical invariants and physical constraints rather than to assert specific numerical outputs. This distinction is critical: portfolio optimisation is sensitive to floating-point precision, solver tolerances, and random seed variations. Tests that assert exact weight values would be brittle and misleading.

Instead, the suite verifies properties such as:

• Weights sum to 1.0 (simplex constraint).
• No single-asset weight exceeds its configured cap.
• Sector concentration limits are respected.
• Gross leverage does not breach the cap.
• The efficient frontier is monotonically non-decreasing (macro trend).
• Identical inputs produce deterministic outputs within solver tolerance.

1.2 Synthetic Data Generation

All tests use synthetically generated return series with controlled statistical properties. This eliminates external API dependencies and ensures reproducibility via numpy.random.default_rng(seed). The synthetic data is designed to exhibit realistic equity-like characteristics: mild positive drift, moderate volatility, and cross-sectional correlation structure.

---

2. Test Inventory

2.1 Constraint Logic & Regime Tests — test_optimize.py

Test Function	Verified Property
test_check_and_fix_bounds_min_exceeds_max	Impossible user bounds (min > max) are auto-corrected to min = 0
test_check_and_fix_bounds_hmm_leverage_disable	Crash regime (HMM severity ≥ 2.5) forces long-only, 1.0× leverage

These tests validate the constraint pre-processing layer (constraints.py), which sanitises user inputs before they reach the convex solver.

2.2 Mean-Variance & CVXPY Tests — test_optimize.py

Test Function	Verified Property
test_efficient_frontier_monotonicity	EF returns are non-decreasing with volatility (50bp tolerance for friction)
test_build_and_optimize_universal_bl_routing	Model 5 (ML Stacking + BL) routes correctly without solver crash
test_build_and_optimize_returns_physically_feasible_weights	Output weights satisfy all configured physical constraints
test_realistic_ml_tax_short_cvar_portfolio_is_feasible	Complex configuration (ML + tax + shorts + GARCH + CVaR) produces feasible output
test_optimizer_constraints_hold_across_random_seeds	Property test: constraints hold across 5 different random return samples
test_optimizer_is_deterministic_for_fixed_inputs	Same inputs → same weights within 1e-5 tolerance
test_jacobian_sensitivity_respects_bounds	10bp perturbation in returns does not cause >50pp weight swing
test_garch_cvar_combined_produces_feasible_portfolio	GARCH + CVaR combined scenario with vol-spike produces valid allocation

2.3 HRP & Tax Heuristic Tests — test_optimize.py

Test Function	Verified Property
test_hrp_with_tax_blending	Tax retention heuristic prevents full liquidation of high-gain positions
test_hrp_turnover_constraint_respected	Turnover budget ≤ 10% is respected by the HRP blending heuristic
test_hrp_property_symmetric_allocation	Identical assets receive symmetric weights under HRP

2.4 Multi-Period Optimisation — test_optimize.py

Test Function	Verified Property
test_multi_period_optimize_returns_valid_weights	MPC stochastic optimizer returns feasible simplex weights

2.5 End-to-End Differentiable Pipeline — test_e2e.py

Test Class / Function	Verified Property
TestDifferentiableLayer	Differentiable CVXPY layer produces valid simplex weights with gradient flow
TestForecastNetwork	Neural network output shapes and positivity of vol-scale
TestTrainer	E2E trainer runs for 3 epochs; predict returns valid Series
TestCache	Model save/load roundtrip; cache invalidation on parameter mismatch
TestSolverIntegration	E2E warm-start weights survive the full build_and_optimize path

2.6 Analytics & Validation — Other Test Files

File	Scope
test_analytics.py	Backtest engine, Sharpe ratio, Sortino, Calmar, drawdown calculations
test_models.py	Expected return model correctness (CAPM, BL, Fama-French, Bayesian)
test_data.py	Data fetching, missing-data handling, return frequency conversion
test_config.py	Configuration loading, defaults, and override logic
test_validation.py	Econometric tests: Christoffersen, DM, PSR, DSR statistical properties
test_regime_detection.py	HMM regime detection, VIX-based risk aversion adjustment
test_risk_attribution.py	Marginal VaR, CVaR attribution, stress correlation calculations
test_fixed_income.py	Bond duration, convexity, and yield calculations
test_overlay.py	Futures overlay optimisation and margin simulation
test_bl_multi_view.py	Black-Litterman with multiple investor views
test_bsts.py	Bayesian Structural Time Series model fitting
test_global.py	Global / multi-region portfolio integration
test_db_e2e.py	Database read/write integration with SQLite and PostgreSQL
test_advanced_paths.py	Edge cases: single-asset portfolio, zero-variance asset, etc.
test_reproducibility.py	Bit-exact reproducibility across sequential runs
test_uc.py	Utility / constraint helper function correctness
test_new_features.py	Transformer training/inference, options flow sentiment (put/call ratio, IV skew, missing chain fallback), and exact risk parity (equal marginal risk contributions)

2.7 End-to-End Simulation — test_simulate.py

This is the top-level integration test that exercises the entire pipeline end-to-end:

1. Configures a 5-asset portfolio (AAPL, MSFT, GOOGL, TLT, GLD) with $1M capital.
2. Mocks all external data sources with deterministic synthetic generators.
3. Invokes run_engine(overrides=...) in headless mode.
4. Asserts that the HTML report was generated and serve_report() was called.

This test validates that all modules compose correctly and that no integration seam is broken.

---

3. Test Execution

Running the Full Suite

pytest tests/ -v

Running a Specific Test File

pytest tests/test_optimize.py -v

Running a Specific Test

pytest tests/test_optimize.py::test_efficient_frontier_monotonicity -v

Running with Coverage

pytest tests/ --cov=. --cov-report=html

---

4. Mocking Strategy

External dependencies are mocked at the boundary:

Dependency	Mock Target	Behaviour
Database (PostgreSQL)	pandas.read_sql	Returns synthetic price DataFrames
Yahoo Finance API	core_engine.fetch_data	Returns ticker list without network
Risk-Free Rate API	core_engine.fetch_risk_free_rate	Returns constant 4%
Report Server	core_engine.serve_report	No-op (prevents browser launch)
CVXPY Solution Checker	cvxpy_engine._solution_violations	Returns empty list

This ensures tests are:

• Fast: No network I/O; typical suite execution < 30 seconds.
• Deterministic: No dependence on live market data.
• Isolated: Failures localise to the component under test, not external services.

---

5. Property-Based Testing Philosophy

Several tests implement a property-based approach (inspired by QuickCheck / Hypothesis), where the same assertion is verified across multiple randomly generated inputs:

• test_optimizer_constraints_hold_across_random_seeds runs 5 seeds.
• test_jacobian_sensitivity_respects_bounds perturbs returns and verifies bounded sensitivity.

This catches edge cases that single-example tests miss, including numerical instability near constraint boundaries.

---

References

• Claessen, K., & Hughes, J. (2000). QuickCheck: A lightweight tool for random testing of Haskell programs. ACM SIGPLAN Notices, 35(9), 268–279.
• pytest Documentation. (2024). https://docs.pytest.org/