metadata
title: Organization Card
emoji: 🚀
colorFrom: blue
colorTo: green
sdk: static
pinned: false
license: mit
Precise Debugging Benchmarking (PDB)
💻 Code · 🌐 Project page · 🏆 Leaderboard
Anonymous release for NeurIPS 2026 Datasets & Benchmarks review.
PDB is an automatic pipeline that turns any coding dataset into a debugging benchmark with fine-grained metrics. Beyond binary unit-test scores, PDB evaluates a debugger with edit-level precision (did the model touch only the lines it had to?) and bug-level recall (did it fix every fault?). This rewards targeted fixes and penalizes the regeneration behavior frontier LLMs often fall back on.
Released datasets
| Dataset | Size | Bug granularity | Notes |
|---|---|---|---|
| PDB-Single | 5,751 | single line | hard subset of single-line bug examples |
| PDB-Wild | 484 | line blocks | multi-line synthesized bugs, real-world repository bugs |
All datasets ship a unified schema with source_dataset provenance, ground-truth
edit scripts (gt_diff), and Orthogonal Defect Classification labels.