README.md

Reproduction: SPEED-Bench

Paper: SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding
OpenReview: https://openreview.net/forum?id=Rl2uQlCoQX
arXiv: https://arxiv.org/abs/2604.09557
Dataset: https://huggingface.co/datasets/nvidia/SPEED-Bench

Structure

• src/ - source code for reproduction experiments
• tests/ - unit tests
• configs/ - configuration files
• outputs/ - raw experimental outputs
• figures/ - generated figures
• artifacts/ - persistent artifacts for logbook
• logs/ - execution logs

Protocol

See protocol.md for detailed claim-by-claim reproduction plan.

Quick Start

# Dataset verification
python src/verify_dataset.py

# Semantic diversity analysis
python src/analyze_diversity.py

# Toy speculative decoding experiments
python src/minimal_sd.py