pp-simulator/README.md

pp-simulator

pp-simulator is a downstream application for pp-scheduler.

It reads a pipeline-parallel schedule plan, samples compute operation durations, recomputes the actual execution timeline from dependency constraints, and exports both interactive web visualizations and trace files.

Features

• Read an in-memory pp_scheduler schedule object.
• Simulate compute operators: F, B, VF, VB, GF, GB.
• Sample operation durations from normal distributions.
• Recompute start/end/wait times from rank-order and semantic dependencies.
• Export Chrome Trace and Perfetto-compatible trace JSON.
• Run Monte Carlo simulations for makespan/utilization/wait-time statistics.
• Serve a lightweight web UI with an SVG pipeline timeline.

Communication operators are intentionally left for a later phase.

Layout

pp_simulator/              Core simulation library
examples/                  CLI demos
tests/                     Unit tests
web/                       Local web UI
pp-simulator-design/       Codex skill for simulator work

Requirements

This repository expects a sibling pp-scheduler checkout when running examples or the web UI from the BigMac workspace layout:

workspace/
  pp-scheduler/
  pp-simulator/

The core package itself only uses the Python standard library.

Single Simulation

python examples/simulate_demo.py

This constructs a UnifiedBigMacVPP schedule, runs one simulation, prints a summary, and writes:

simulate_demo_trace.json
simulate_demo_perfetto_trace.json

Monte Carlo

python examples/monte_carlo_demo.py

This writes:

monte_carlo_report.json

The report includes makespan mean/std/percentiles, rank utilization, rank wait time, rank compute time, and op type time.

Web UI

python web/app.py --host 127.0.0.1 --port 8765

Open the printed URL in a browser.

The UI supports:

• Scheduler selection.
• PP/VPP/microbatch/VE-limit configuration.
• Per-op mean/variance duration inputs.
• Single-run SVG timeline visualization.
• Wait gap rendering.
• Hover tooltip and selected-op dependency details.
• Chrome Trace and Perfetto-compatible trace downloads.

On macOS trackpads, pinch zoom works inside the timeline area.

API Sketch

from pp_scheduler import UnifiedBigMacVPP, OpType
from pp_simulator import OpDurationSpec, PipelineSimulator

scheduler = UnifiedBigMacVPP(
    pp_size=4,
    vpp_size=2,
    num_microbatches=8,
    ve_forward_limit=3,
)
scheduler.generate_schedule()

simulator = PipelineSimulator.from_scheduler(scheduler)
result = simulator.simulate(
    {
        OpType.FORWARD: OpDurationSpec(mean=1.0, variance=0.02),
        OpType.BACKWARD: OpDurationSpec(mean=2.0, variance=0.05),
        OpType.VEFORWARD: OpDurationSpec(mean=0.8, variance=0.01),
        OpType.VEBACKWARD: OpDurationSpec(mean=0.9, variance=0.01),
        OpType.GENFORWARD: OpDurationSpec(mean=0.6, variance=0.01),
        OpType.GENBACKWARD: OpDurationSpec(mean=0.7, variance=0.01),
    },
    seed=7,
)

result.to_chrome_trace("trace.json")
result.to_chrome_trace("perfetto_trace.json", perfetto_compat=True)

Monte Carlo:

report = simulator.monte_carlo(
    duration_specs,
    num_trials=100,
    seed=7,
    validate=True,
)
report.to_json("monte_carlo_report.json")

Validation

python -m unittest discover -s tests
python -m compileall pp_simulator examples tests web
node --check web/static/app.js

Author

寅虎 liuyang50@xiaohongshu.com