docs/README_alpha_robustness.md

# Alpha Robustness Runner

This runner is intentionally isolated from the stable backtest path.

Use:

- [deploy/v2/jsonl_alpha_robustness.py](/Users/giaphu/Documents/Pylab/AAE-new/deploy/v2/jsonl_alpha_robustness.py)
- [backtest/robust_factor_executor.py](/Users/giaphu/Documents/Pylab/AAE-new/backtest/robust_factor_executor.py)
- [backtest/robust_qlib_backtester.py](/Users/giaphu/Documents/Pylab/AAE-new/backtest/robust_qlib_backtester.py)

Do **not** modify the production backtest flow when running robustness experiments.

## Purpose

Run robustness backtests for a fixed alpha pack, for example the top 10 alphas,
while varying assumptions such as:

- `TOP_K`
- `rebalance_freq`
- `buy/sell fee`
- weighting mode
- score transform
- liquidity-derived universe filter
- execution limits

## Input format

Pass a JSONL file where each row is at least:

```json
{"name":"ALPHA_001","expr":"RANK($close)"}
```

Optional fields such as `seed_name`, `factor_name`, `factor_expr`,
`candidate_scope`, `proposal_rank`, `turn`, and `call_index` are also accepted.

## Minimal example

```bash
python deploy/v2/jsonl_alpha_robustness.py \
  --jsonl data/top10_alphas.jsonl \
  --period test \
  --backtest-engine custom \
  --top-k 5 \
  --rebalance-freq 5 \
  --custom-weight-mode alpha_score \
  --max-pos-each-stock 0.2 \
  --enforce-cash-limit \
  --capture-detail-artifacts \
  --output-dir outputs/robust/top10_baseline
```

## Common robustness sweeps

### 1. `TOP_K` sensitivity

Run the same pack with:

- `--top-k 3`
- `--top-k 5`
- `--top-k 10`

### 2. Rebalance sensitivity

Run the same pack with:

- `--rebalance-freq 5`
- `--rebalance-freq 10`
- `--rebalance-freq 20`

### 3. Fee sensitivity

Run the same pack with:

- `--buy-fee 0.0013 --sell-fee 0.0013`
- `--buy-fee 0.0020 --sell-fee 0.0020`
- `--buy-fee 0.0030 --sell-fee 0.0030`

### 4. Weighting robustness

Examples:

- `--custom-weight-mode equal --max-pos-each-stock 0.2`
- `--custom-weight-mode alpha_score --max-pos-each-stock 0.2`
- `--custom-weight-mode alpha_score --max-pos-each-stock 1.0`

### 5. Score-transform robustness

Examples:

- `--score-transform identity`
- `--score-transform rank`
- `--score-transform zscore`
- `--score-transform rank_zscore`
- `--score-transform clip_zscore --score-clip 3.0`

### 6. Liquidity-derived universe robustness

Examples:

- `--universe-filter top_amount --universe-top-n 70 --universe-lookback-days 20`
- `--universe-filter top_amount --universe-top-n 90 --universe-lookback-days 20`
- `--universe-filter top_volume --universe-top-n 70 --universe-lookback-days 20`

This is an internal proxy universe derived from `daily_pv.h5`. It is **not**
the same as an external market-cap or index-membership universe.

## Outputs

The runner writes the same main artifacts as the standalone backtest runner,
plus a robustness manifest:

- `summary.csv`
- `summary_yearly.csv`
- `trials.csv`
- `trials_yearly.csv`
- `aggregate.json`
- `aggregate_yearly.csv`
- `portfolio_daily.csv`
- `trade_log.csv`
- `holdings_daily.csv`
- `signal_selection_daily.csv`
- `rebalance_plan.csv`
- `rebalance_window_returns.csv`
- `data_quality_report.json`
- `robust_manifest.json` (or the custom name from `--manifest-name`)

## Recommended workflow for 10 alphas

1. Prepare a `top10_alphas.jsonl`
2. Run one baseline bundle
3. Run one axis at a time:
   - `top_k`
   - `rebalance_freq`
   - fee
   - weighting
   - score transform
   - universe filter
4. Compare each bundle using:
   - `summary.csv`
   - `aggregate.json`
   - `portfolio_daily.csv`
   - `trade_log.csv`
   - `signal_selection_daily.csv`
   - `rebalance_plan.csv`

## Important note

This path is separate on purpose. If a robustness experiment fails or needs
extra knobs, patch the robust files above instead of the stable production
backtest path.