docs/README_alpha_robustness.md

Alpha Robustness Runner

This runner is intentionally isolated from the stable backtest path.

Use:

• deploy/v2/jsonl_alpha_robustness.py
• backtest/robust_factor_executor.py
• backtest/robust_qlib_backtester.py

Do not modify the production backtest flow when running robustness experiments.

Purpose

Run robustness backtests for a fixed alpha pack, for example the top 10 alphas, while varying assumptions such as:

• TOP_K
• rebalance_freq
• buy/sell fee
• weighting mode
• score transform
• liquidity-derived universe filter
• execution limits

Input format

Pass a JSONL file where each row is at least:

{"name":"ALPHA_001","expr":"RANK($close)"}

Optional fields such as seed_name, factor_name, factor_expr, candidate_scope, proposal_rank, turn, and call_index are also accepted.

Minimal example

python deploy/v2/jsonl_alpha_robustness.py \
  --jsonl data/top10_alphas.jsonl \
  --period test \
  --backtest-engine custom \
  --top-k 5 \
  --rebalance-freq 5 \
  --custom-weight-mode alpha_score \
  --max-pos-each-stock 0.2 \
  --enforce-cash-limit \
  --capture-detail-artifacts \
  --output-dir outputs/robust/top10_baseline

Common robustness sweeps

1. TOP_K sensitivity

Run the same pack with:

• --top-k 3
• --top-k 5
• --top-k 10

2. Rebalance sensitivity

Run the same pack with:

• --rebalance-freq 5
• --rebalance-freq 10
• --rebalance-freq 20

3. Fee sensitivity

Run the same pack with:

• --buy-fee 0.0013 --sell-fee 0.0013
• --buy-fee 0.0020 --sell-fee 0.0020
• --buy-fee 0.0030 --sell-fee 0.0030

4. Weighting robustness

Examples:

• --custom-weight-mode equal --max-pos-each-stock 0.2
• --custom-weight-mode alpha_score --max-pos-each-stock 0.2
• --custom-weight-mode alpha_score --max-pos-each-stock 1.0

5. Score-transform robustness

Examples:

• --score-transform identity
• --score-transform rank
• --score-transform zscore
• --score-transform rank_zscore
• --score-transform clip_zscore --score-clip 3.0

6. Liquidity-derived universe robustness

Examples:

• --universe-filter top_amount --universe-top-n 70 --universe-lookback-days 20
• --universe-filter top_amount --universe-top-n 90 --universe-lookback-days 20
• --universe-filter top_volume --universe-top-n 70 --universe-lookback-days 20

This is an internal proxy universe derived from daily_pv.h5. It is not the same as an external market-cap or index-membership universe.

Outputs

The runner writes the same main artifacts as the standalone backtest runner, plus a robustness manifest:

• summary.csv
• summary_yearly.csv
• trials.csv
• trials_yearly.csv
• aggregate.json
• aggregate_yearly.csv
• portfolio_daily.csv
• trade_log.csv
• holdings_daily.csv
• signal_selection_daily.csv
• rebalance_plan.csv
• rebalance_window_returns.csv
• data_quality_report.json
• robust_manifest.json (or the custom name from --manifest-name)

Recommended workflow for 10 alphas

1. Prepare a top10_alphas.jsonl
2. Run one baseline bundle
3. Run one axis at a time:
   • top_k
   • rebalance_freq
   • fee
   • weighting
   • score transform
   • universe filter
4. Compare each bundle using:
   • summary.csv
   • aggregate.json
   • portfolio_daily.csv
   • trade_log.csv
   • signal_selection_daily.csv
   • rebalance_plan.csv

Important note

This path is separate on purpose. If a robustness experiment fails or needs extra knobs, patch the robust files above instead of the stable production backtest path.