algorise/lion-conv-artifacts/repro-bundle-v0 - Storage Bucket

Reproduction: Convergence Analysis of the Lion Optimizer in Centralized and Distributed Settings

ICML 2026 reproduction of Jiang & Zhang, "Convergence Analysis of the Lion Optimizer in Centralized and Distributed Settings" (arXiv:2508.12327, OpenReview 32NvV5zixD).

The paper proves seven theorems bounding (1/T) sum_t E||grad f(x_t)||_1 for the Lion (evolved sign-momentum) optimizer and five variants: centralized, STORM variance-reduced, distributed (heterogeneous nodes), distributed+VR, and two communication-efficient (1-bit sign-compressed) variants. It ships no code and no numerical experiments -- it is a pure convergence-theory paper.

This repo verifies the seven rate claims numerically, from the paper's own algorithm statements (Algorithms 1-3), on a synthetic smooth-nonconvex stochastic objective (src/objective.py) sized to give clean log-log scaling fits on CPU: T up to 1.5e5, d up to 1000, n up to 64.

Layout

• paper.txt -- extracted full text of the paper (for reference).
• src/objective.py -- the test objective (smooth nonconvex "correntropy" robust-regression loss) + heterogeneous multi-node data generation.
• src/lion.py -- faithful implementations of Algorithms 1 (Lion / Lion-VR), 2 (distributed), and 3 (communication-efficient, 1-bit sign compression).
• src/rates.py -- the O(.) hyperparameter schedules from each theorem, with concrete constants (calibrated once in experiments/calibrate.py, then held fixed across every sweep).
• src/analysis.py -- shared log-log slope fitting / plotting helpers.
• experiments/claim{1..6}*.py -- one script per logbook claim; each writes results/claimN_*.csv/json/png.
• experiments/claim6b_discrimination_probe.py -- supplementary small-(d,n) probe for Claim 6's Theorem 6 vs 7 crossover.

Rerun

pip install -r requirements.txt
cd experiments
python3 calibrate.py                              # sanity: no NaN/divergence
python3 claim1_centralized_rate.py                 # ~100s
python3 claim2_variance_reduced_rate.py             # ~245s
python3 claim3_distributed_rate.py                  # ~65s
python3 claim4_distributed_vr_rate.py               # ~155s
python3 claim5_comm_efficient_floor.py              # ~330s
python3 claim6_comm_efficient_vr_improvement.py     # ~365s
python3 claim6b_discrimination_probe.py             # ~185s

All CPU-only, single-threaded numpy; no GPU used or required.