repro_bundle/README.md

Reproduction bundle — Rationality Measurement and Theory for RL Agents (ICML 2026)

Paper: Kejiang Qian, Amos Storkey, Fengxiang He. "Rationality Measurement and Theory for Reinforcement Learning Agents." arXiv:2602.04737 (OpenReview RlEPV6ec7P). Official code: https://github.com/EVIEHub/Rationality

Contents

• scripts/
  • theory_verification.py — numerical verification of Lemma 2, Theorem 1, Theorem 2 on Taxi-v3 (Claims 1-3).
  • run_sweep.py — fresh independent DQN sweep runner (clones+patches repo, runs all variables x seeds in parallel, uploads CSVs). Used for Claims 4-5 fresh runs on HF Jobs.
  • analyze_claims.py — quantitative claim verification from the authors' bundled logs (means, reductions, Pearson correlations).
  • make_figures.py — Plotly figure generation from bundled logs (Figures 2a/2b/3).
  • diag.py — minimal HF-environment diagnostic.
• figures/ — Plotly HTML figures + raw CSV data for Claims 4 & 5 (from authors' bundled logs), and claim_analysis.txt (the quantitative summary).
• tables/ — generated summary tables (reg intensity).

How to reproduce

# Theory claims (1-3): fast, local CPU
uv run --with torch numpy gymnasium scipy pandas scripts/theory_verification.py

# Empirical claims (4-5): figures from the authors' bundled logs (in the official repo "Datasets and plots/")
cd "Datasets and plots" && python ../repro_bundle/scripts/make_figures.py
python ../repro_bundle/scripts/analyze_claims.py

# Fresh independent runs (Claims 4-5) on HF Jobs (CPU):
hf jobs uv run ../repro_bundle/scripts/run_sweep.py --experiments reg,domain_rand,environment_level --workers 30 --flavor cpu-xl --timeout 4h
# (cliffwalking: add `-e RATIONALITY_ENV=cliffwalking`)

The only deviation from the official code is a version fallback (Taxi-v3→Taxi-v4, CliffWalking-v0→CliffWalking-v1) because newer gymnasium deprecates the old IDs; the transition dynamics are identical, so results are unaffected.