reverse_pass/README.md

Compiler Phase-Ordering RLVR (Toy-IR)

Toy-IR compiler phase-ordering environment for RL with verifiable rewards (RLVR), including reverse passes to escape local optima.

What Is Implemented

• Reverse passes are available in the pass library:
  • expand_constant
  • duplicate_computation
• Action routing and validation include reverse passes.
• Prompt guidance includes reverse-pass descriptions and usage intent.
• Reward shaping is terminal-weighted with RLVR hard gate:
  • non-equivalent -> -1000.0
  • terminal -> ((original_cycles - current_cycles) / original_cycles) * 100.0
  • non-terminal -> -0.1
• Episode termination:
  • STOP/done, or
  • hard cap at 5 steps.
• Reverse-pass instrumentation logs every 50 episodes/completions:
  • reverse_pass_episodes
  • expand_constant_count
  • duplicate_computation_count

Verified So Far

• Smoke test runs end-to-end without runtime errors.
• Rewards are scalar floats (not NaN) in tested rollouts.
• Forced reverse-pass episodes show expected behavior:
  • small negative intermediate rewards (-0.1)
  • terminal reward depends on final outcome (positive if chain beats baseline, negative if not)

Pending (GPU Required)

Comparative training runs are still pending and require a GPU environment with:

• torch
• trl
• unsloth
• wandb

Required fair comparison:

1. Baseline run: reverse passes disabled (baseline_no_reverse)
2. Reverse run: reverse passes enabled (with_reverse_passes)
3. Same episode budget for both runs
4. Compare reward/cycle curves and reverse-pass usage metrics in WandB

Team Handoff Status

• Reverse-pass feature: shipped
• Training wiring + reward shaping + instrumentation: shipped
• Comparative training: pending in GPU environment
• README: this document

Suggested Final Pre-Training Sanity Check

Run one smoke/equivalence check on a real Role 3 generated curriculum program (not just mock IR) to confirm verifier behavior in-pipeline before long training.