dataset_description.md

LLM-Prof Dataset Description

Dataset link: https://huggingface.co/buckets/zk521/LLM-Prof-Dataset/

Overview

The LLM-Prof dataset contains 207 GPU kernel traces collected for the paper "LLM-Prof: A Hierarchical Cross-Stack Performance Profiling Framework for Production LLM Inference Services". The dataset supports cross-layer analysis of LLM inference services at the service, model, and operator levels.

The traces cover three inference frameworks:

Framework	Cases	Source
RTP-LLM	51	Production MaaS services with real runtime telemetry
SGLang	73	Controlled benchmark traces across models, GPUs, and workloads
vLLM	83	Controlled benchmark traces across models, GPUs, and workloads
Total	207	Production plus controlled benchmark traces

Dataset Files

The released dataset is organized as three compressed archives:

Archive	Content
original_trace_RTP-LLM_51.tar.gz	RTP-LLM production traces and related runtime metadata
original_trace_SGLang_73.tar.gz	SGLang benchmark traces
original_trace_vLLM_83.tar.gz	vLLM benchmark traces

All traces are stored in Chrome Trace Event JSON format and compressed as .json.gz or framework-specific trace files. A typical trace contains GPU kernels, memory copies, CUDA runtime calls, CUDA driver calls, synchronization events, and related metadata.

Framework-Specific Data

RTP-LLM

The RTP-LLM subset contains traces from real production services. Each case may include:

• GPU kernel trace files collected from production inference execution.
• prefill_metrics_with_config.txt, which records model configuration, hardware information, QPS time series, GPU utilization, token-size information, and service metadata.
• Profiling metadata and optional analysis outputs.

This subset reflects real deployment diversity, including varying business workloads, request rates, token sizes, GPU utilization, model sizes, and hardware configurations.

SGLang and vLLM

The SGLang and vLLM subsets are controlled benchmark traces. They systematically vary:

• Model family and model size.
• Batch size.
• Input length and output length.
• GPU type.
• Inference framework.

These traces provide controlled comparisons across frameworks and hardware platforms, complementing the production diversity of the RTP-LLM subset.

Hardware and Model Coverage

The dataset covers six NVIDIA GPU types:

• A10
• A100
• A800
• H20
• H800
• L20

The evaluated services include 21 model variants from three major model families: Qwen, LLaMA, and BERT-family services. The model sizes range from sub-billion scale to 70B-scale deployments, including dense and MoE-style models.

Trace Format

Each trace follows the Chrome Trace Event format. The top-level structure usually contains:

{
  "schemaVersion": 1,
  "deviceProperties": [],
  "traceEvents": [],
  "traceName": "...",
  "displayTimeUnit": "ns"
}

The traceEvents array is the main analysis input. Important event categories include:

Event category	Description
kernel	GPU kernel execution events used for operator-level analysis
gpu_memcpy	Host-device and device-device memory copy events used for iteration anchoring
gpu_memset	GPU memory initialization events
cuda_runtime	CUDA Runtime API calls
cuda_driver	CUDA Driver API calls
synchronization	CUDA synchronization events
cuda_event	CUDA event records

For GPU kernel events, important fields include:

Field	Description
name	Compiled CUDA kernel name
ts	Start timestamp
dur	Duration
tid	CUDA stream or thread identifier
args.grid	Grid configuration
args.block	Block configuration
args.shared memory	Shared memory usage
args.registers per thread	Register count per thread
args.est. achieved occupancy %	Estimated occupancy

Analysis Layers Supported by the Dataset

The dataset supports the three LLM-Prof analysis layers:

Layer	Purpose	Key metrics
SEA	Service-level hotspot detection	QPS, FPR, token size, GPU utilization
MEA	Model-level iteration analysis	IIPS, MIE, iteration duration
OEA	Operator-level bottleneck analysis	operator efficiency, BottleScore, time proportion, Roofline position

Notes

• RTP-LLM traces reflect real production workloads and may contain more heterogeneous runtime behavior than controlled benchmarks.
• SGLang and vLLM traces are better suited for controlled model-framework-hardware comparisons.
• Fine-grained operator attribution may be affected by framework-specific kernel fusion, custom kernels, and CUDA Graph-based execution.
• Current profiling support is based on NVIDIA/CUPTI traces.