repro_bundle/README.md

# Reproduction bundle — Rationality Measurement and Theory for RL Agents (ICML 2026)

Paper: Kejiang Qian, Amos Storkey, Fengxiang He. "Rationality Measurement and Theory for Reinforcement Learning Agents." arXiv:2602.04737 (OpenReview RlEPV6ec7P).
Official code: https://github.com/EVIEHub/Rationality

## Contents

- `scripts/`
  - `theory_verification.py` — numerical verification of Lemma 2, Theorem 1, Theorem 2 on Taxi-v3 (Claims 1-3).
  - `run_sweep.py` — fresh independent DQN sweep runner (clones+patches repo, runs all variables x seeds in parallel, uploads CSVs). Used for Claims 4-5 fresh runs on HF Jobs.
  - `analyze_claims.py` — quantitative claim verification from the authors' bundled logs (means, reductions, Pearson correlations).
  - `make_figures.py` — Plotly figure generation from bundled logs (Figures 2a/2b/3).
  - `diag.py` — minimal HF-environment diagnostic.
- `figures/` — Plotly HTML figures + raw CSV data for Claims 4 & 5 (from authors' bundled logs), and `claim_analysis.txt` (the quantitative summary).
- `tables/` — generated summary tables (reg intensity).

## How to reproduce

```bash
# Theory claims (1-3): fast, local CPU
uv run --with torch numpy gymnasium scipy pandas scripts/theory_verification.py

# Empirical claims (4-5): figures from the authors' bundled logs (in the official repo "Datasets and plots/")
cd "Datasets and plots" && python ../repro_bundle/scripts/make_figures.py
python ../repro_bundle/scripts/analyze_claims.py

# Fresh independent runs (Claims 4-5) on HF Jobs (CPU):
hf jobs uv run ../repro_bundle/scripts/run_sweep.py --experiments reg,domain_rand,environment_level --workers 30 --flavor cpu-xl --timeout 4h
# (cliffwalking: add `-e RATIONALITY_ENV=cliffwalking`)
```

The only deviation from the official code is a version fallback (`Taxi-v3`→`Taxi-v4`, `CliffWalking-v0`→`CliffWalking-v1`) because newer gymnasium deprecates the old IDs; the transition dynamics are identical, so results are unaffected.