How to reproduce the AHD-CMA experiments

Every command assumes the repo root and an activated virtual env::

git clone <repo>
cd <repo>
python3.11 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
pip install torch==2.4.1 torchvision==0.19.1 --index-url https://download.pytorch.org/whl/cu121

Confirm GPU::

python -c "import torch; assert torch.cuda.is_available(); print(torch.cuda.get_device_name(0))"

1. Unit and integration tests

pytest tests/unit -q                              # ~1 minute
RUN_INTEGRATION=1 pytest tests/integration -v     # ~10 minutes (downloads CIFAR-10 + ViT-Tiny)

The integration test runs the LoRA fitness end-to-end on CIFAR-10 with a tiny ViT and asserts > 50 % accuracy in < 10 minutes.

2. CEC-2022 benchmark

Single run::

python -m ahdcma.cli.run_benchmark \
    --algo ahdcma --func F5_rastrigin --dim 10 --seed 0 \
    --max-evals 30000 --pop 30 --output outputs/runs/cec2022

Full sweep (7 algos x 12 funcs x {10D, 30D} x 30 seeds = 5040 runs, ~19 h on a single 24 GB GPU machine)::

python scripts/run_cec2022_full.py

Resume-safe: re-running picks up where it stopped via existing- result.json checks.

After the sweep, render figures and tables::

python -m ahdcma.cli.make_figures

Outputs land in outputs/figures/ and outputs/tables/.

3. LoRA hyperparameter sweep

The full pipeline is gated by user approval per CLAUDE.md §10. Recommended progression:

# Round 1 -- preliminary (3 algos x 3 tasks x 5 seeds, ~180 GPU-h)
bash scripts/run_full_pipeline.sh round 1

# Round 2 -- full sweep (14 algos x 7 tasks x 5 seeds, ~1960 GPU-h)
bash scripts/run_full_pipeline.sh round 2

# Round 3 -- 30-seed statistical confirmation (top-4 algos x 7 tasks)
bash scripts/run_full_pipeline.sh round 3

Single LoRA run::

python -m ahdcma.cli.run_task \
    --algo ahdcma --task cifar100_vit --seed 0 \
    --pop 8 --gens 10 --num-steps 100

4. Ablation study

python scripts/run_ablation.py \
    --funcs F1_bent_cigar F5_rastrigin F9_composite1 \
    --variants full no_chaotic no_adaptive no_ruggedness no_doa_de \
    --seeds 0 1 2 3 4 \
    --pop 30 --gens 100

5. Statistical analysis

The CEC-2022 figure generator computes per-dim Wilcoxon vs AHD-CMA and the Friedman omnibus. Use cli.make_figures for the canonical artifact bundle, or call the lower-level helpers directly:

from ahdcma.stats.tests import wilcoxon_pairwise, friedman_test, cliffs_delta

results = {"ahdcma": ..., "cmaes": ..., ...}  # algo -> per-seed array
df = wilcoxon_pairwise(results, baseline_name="ahdcma")
stat, p = friedman_test(results)
delta = cliffs_delta(results["ahdcma"], results["cmaes"])

6. Reproducibility checklist

All entry points call set_global_seed(seed) first thing after argparse.
Each run writes a config snapshot to its own result.json.
make_run_id includes algo, task, seed, dim and a timestamp.
The full 30-seed pool used for reported results is seeds=range(30).
Deviations from the original proposal are catalogued in docs/spec_deviations.md.