release/v1/docs/WORKFLOW.md

# Chebyshev Policies reproduction

This directory contains the independent, claim-by-claim reproduction for
ICML 2026 paper 12142, OpenReview `aNWIVNjocB`, arXiv `2605.22305`.

The driver fetches the authors' two repositories at immutable commits and
verifies both commit hashes before importing them. The upstream repositories
do not declare a license, so they and their bundled model weights are not
vendored or republished here. Only the reproduction driver, configs, raw
measurements, derived tables, figures, manifests, and newly trained
coefficients are part of this bundle.

## Pinned inputs

- Paper code: `JRC-ISIA/paper-2026-chebyshev-policies-low-dimensional-control-tasks`
  at `fad6ebb46f976db29eb5ba509622dc3c24b7ac5f`
- Polynomial SB3 code: `JRC-ISIA/polynomial-sb3-rl-agents`
  at `ed945331ce989592d75af9e8410823ee007e13a8`
- Paper: arXiv v4, `2605.22305v4`, dated 2026-07-08

## Workflow

```text
configs/*.json
  -> scripts/run_reproduction.py
  -> runs/raw/<batch-id>/
  -> scripts/build_tables.py
  -> runs/tables/
  -> scripts/make_figures.py
  -> runs/reports/
  -> scripts/analyze_claims.py
  -> runs/analysis/
```

Every raw batch is append-only: the runner refuses to reuse an existing
`batch_id`. Failed units remain in `records.jsonl`. Derived tables, analyses,
and figures are regenerated deterministically; their manifests carry input
hashes, the builder SHA-256, and the exact build command.

## Commands

Local smoke test:

```bash
uv run --python 3.10 scripts/run_reproduction.py \
  --config configs/smoke.json \
  --batch-id local-smoke-YYYYMMDD-HHMMSS \
  --output-root runs/raw
```

Full configurations are split so each empirical claim has a clear Hugging
Face Job URL and cost:

```bash
uv run --python 3.10 scripts/run_reproduction.py --config configs/claim23_full.json ...
uv run --python 3.10 scripts/run_reproduction.py --config configs/claim4_full.json ...
uv run --python 3.10 scripts/run_reproduction.py --config configs/claim5_pendulum_full.json ...
```

On Hugging Face Jobs, use `scripts/run_hf_job.sh CONFIG_FILENAME BATCH_ID`.
It stages append-heavy evidence on local storage, validates the completed
manifest, publishes only closed files to the Bucket mount, and requires a
byte-for-byte tree comparison before returning success.

The release allowlist contains exactly these three v4 batch identities.
Admission still requires each directory to pass its own closed-manifest and
byte-integrity validation; the batch name alone is not evidence of success.
Build canonical tables by enumerating the allowlist explicitly—never pass the
parent `runs/raw` directory:

```bash
uv run --python 3.10 scripts/build_tables.py \
  --raw-root runs/raw/hf-claim23-full-v4-20260715T1844Z \
  --raw-root runs/raw/hf-claim4-full-v4-20260715T1845Z \
  --raw-root runs/raw/hf-claim5-pendulum-full-v4-20260715T1846Z \
  --output-root runs/tables

uv run --python 3.10 scripts/make_figures.py \
  --tables-root runs/tables \
  --output-root runs/reports

uv run --python 3.10 scripts/analyze_claims.py \
  --tables-root runs/tables \
  --output-root runs/analysis
```

The release contract has four interactive figures:

1. Mountain Car per-start returns and method means.
2. All-seed robustness with the selected seed identified.
3. Parameter efficiency versus regret.
4. Pendulum paired reset sensitivity, including the author-exact and corrected
   reset protocols.

The policy microscope and claim-evidence map planned in the frozen spec are
not separate interactive panels. Claim-to-evidence links are instead published
as canonical CSV/Markdown outputs and summarized in the poster and Trackio
logbook; this presentation drift is disclosed below. Release construction is
fail-closed on accepted raw-batch identity and integrity, derived-manifest
provenance, the required four-figure set, and poster integrity gates.

After substituting the terminal Claim 4/5 results into the poster, rerunning
Posterly's strict gates, and creating `jobs/HF_JOBS.json` plus
`RELEASE_README.md`, build the curated public bundle with:

```bash
uv run --python 3.10 scripts/build_poster_manifest.py \
  --poster-root poster --replace

uv run --python 3.10 scripts/build_release.py \
  --project-root . \
  --release-readme RELEASE_README.md \
  --raw-root runs/raw/hf-claim23-full-v4-20260715T1844Z \
  --raw-root runs/raw/hf-claim4-full-v4-20260715T1845Z \
  --raw-root runs/raw/hf-claim5-pendulum-full-v4-20260715T1846Z \
  --jobs-root jobs \
  --tables-root runs/tables \
  --analysis-root runs/analysis \
  --reports-root runs/reports \
  --poster-root poster \
  --output-root release/v1

uv run --python 3.10 scripts/validate_logbook_release.py \
  --logbook-root ../.trackio/logbook \
  --project-root .. \
  --mode prepublish
```

The release builder recomputes every declared hash, requires the exact nine-Job
attempt chronology, distinguishes inspected scheduler commands from separate
hash-verified rerun recipes, and refuses raw records, third-party weight files,
secrets, caches, or unresolved poster placeholders in the curated bundle. The
full raw records and checkpoints remain hash-addressed in the public evidence
Bucket.

See `experiments/icml2026_chebyshev/SPEC.md` for the exact metrics, panels,
selection rules, and interpretation limits. Necessary post-freeze engineering
changes are disclosed in `experiments/icml2026_chebyshev/DRIFT.md`.
The bounded primary-source review of the paper's `first` and `36-year open
problem` language is in
`experiments/icml2026_chebyshev/HISTORICAL_PRIORITY.md`.