Speed Test Guide: HRM SRAM/DRAM Benchmarking
Use this guide to measure the hardware performance (latency, throughput, memory overhead) of the memory-tiered architecture.
This guide uses synthetic data. No external datasets or prior training are required. You can run this immediately after setup.
1. Environment Setup
Create a clean virtual environment and install dependencies:
python -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
pip install triton matplotlib
2. Run Hardware Comparison
This command compares the Baseline (Original) model vs. the Tiered (Optimized) model across multiple configurations.
python run_benchmark.py --mode compare --plot
What this does:
- Generates random tensors to simulate a reasoning workload.
- Compiles Triton kernels for the L-level (SRAM) and H-level (DRAM) paths.
- Records precise GPU timing using
torch.cuda.Event. - Saves charts to
benchmark_results/.
3. Deep Dive into Tiered Metrics
To get a detailed breakdown of memory tier behavior:
python run_benchmark.py --mode tiered --iterations 50 --batch-sizes 8 --seq-lens 128
Key Metrics to watch:
- L_lat(μs): Speed of the fast-updating L-level module in SRAM.
- H_lat(μs): Speed of the planning H-level module in DRAM.
- H/L Ratio: Shows the latency multiplier between memory tiers.
- Memory Efficiency: Percentage of time spent on math vs. memory transfer.
4. Custom Hardware Stress Test
Test the limits of your GPU by increasing batch sizes or sequence lengths:
python run_benchmark.py \
--mode compare \
--batch-sizes 64,128 \
--seq-lens 256,512 \
--hidden-size 1024