reverse_pass/README.md

# Compiler Phase-Ordering RLVR (Toy-IR)

Toy-IR compiler phase-ordering environment for RL with verifiable rewards (RLVR), including reverse passes to escape local optima.

## What Is Implemented

- Reverse passes are available in the pass library:
  - `expand_constant`
  - `duplicate_computation`
- Action routing and validation include reverse passes.
- Prompt guidance includes reverse-pass descriptions and usage intent.
- Reward shaping is terminal-weighted with RLVR hard gate:
  - non-equivalent -> `-1000.0`
  - terminal -> `((original_cycles - current_cycles) / original_cycles) * 100.0`
  - non-terminal -> `-0.1`
- Episode termination:
  - `STOP`/`done`, or
  - hard cap at 5 steps.
- Reverse-pass instrumentation logs every 50 episodes/completions:
  - `reverse_pass_episodes`
  - `expand_constant_count`
  - `duplicate_computation_count`

## Verified So Far

- Smoke test runs end-to-end without runtime errors.
- Rewards are scalar floats (not NaN) in tested rollouts.
- Forced reverse-pass episodes show expected behavior:
  - small negative intermediate rewards (`-0.1`)
  - terminal reward depends on final outcome (positive if chain beats baseline, negative if not)

## Pending (GPU Required)

Comparative training runs are still pending and require a GPU environment with:

- `torch`
- `trl`
- `unsloth`
- `wandb`

Required fair comparison:

1. Baseline run: reverse passes disabled (`baseline_no_reverse`)
2. Reverse run: reverse passes enabled (`with_reverse_passes`)
3. Same episode budget for both runs
4. Compare reward/cycle curves and reverse-pass usage metrics in WandB

## Team Handoff Status

- Reverse-pass feature: shipped
- Training wiring + reward shaping + instrumentation: shipped
- Comparative training: pending in GPU environment
- README: this document

## Suggested Final Pre-Training Sanity Check

Run one smoke/equivalence check on a real Role 3 generated curriculum program (not just mock IR) to confirm verifier behavior in-pipeline before long training.