| --- |
| title: Organization Card |
| emoji: 馃殌 |
| colorFrom: blue |
| colorTo: green |
| sdk: static |
| pinned: false |
| license: mit |
| --- |
| |
| # Precise Debugging Benchmarking (PDB) |
|
|
| 馃捇 [Code](https://github.com/anon-pdb-2026/anon-pdb-code) 路 |
| 馃寪 [Project page](https://anon-pdb-2026.github.io) 路 |
| 馃弳 [Leaderboard](https://anon-pdb-2026.github.io/leaderboard.html) |
|
|
| > *Anonymous release for NeurIPS 2026 Datasets & Benchmarks review.* |
|
|
| **PDB** is an automatic pipeline that turns any coding dataset into a |
| *debugging* benchmark with fine-grained metrics. Beyond binary unit-test |
| scores, PDB evaluates a debugger with **edit-level precision** (did the model |
| touch only the lines it had to?) and **bug-level recall** (did it fix every |
| fault?). This rewards targeted fixes and penalizes the regeneration behavior |
| frontier LLMs often fall back on. |
|
|
| ## Released datasets |
|
|
| | Dataset | Size | Bug granularity | Notes | |
| |---|---|---|---| |
| | [PDB-Single](https://huggingface.co/datasets/anon-pdb/PDB-Single) | 5,751 | single line | hard subset of single-line bug examples | |
| | [PDB-Wild](https://huggingface.co/datasets/anon-pdb/PDB-Wild) | 484 | line blocks | multi-line synthesized bugs, real-world repository bugs | |
|
|
| All datasets ship a unified schema with `source_dataset` provenance, ground-truth |
| edit scripts (`gt_diff`), and Orthogonal Defect Classification labels. |
