Haaardikkkk/bugagent-localizer

BugAgent bug localizer

A binary classifier (clean or buggy) over single functions in Python, Java and C/C++, used by BugAgent. Built on answerdotai/ModernBERT-base, with 512 tokens per function.

Results

AUC: 0.50 is chance and 1.00 is perfect.

Test	Held-out projects	Benchmarks never used in training or selection
Buggy vs its own fix	0.49	0.50
Buggy vs fixed and unchanged code	0.67	0.55
Buggy vs unchanged code	0.75	0.58
Buggy ranked above its fix	42% of pairs	47% of pairs

This model can't tell a bug from its own fix: that score is chance level. BugAgent gives its scores zero weight when ranking problems. Don't rely on it to decide whether code is correct.

Data

Trained on: nebius/SWE-rebench, princeton-nlp/SWE-bench (train), nebius/SWE-bench-extra, SWE-Gym/SWE-Gym, ByteDance-Seed/Multi-SWE-RL, bigcode/commitpackft, ASSERT-KTH/megadiff-single-function, bstee615/bigvul.

Benchmarks: princeton-nlp/SWE-bench_Verified, ByteDance-Seed/Multi-SWE-bench.