---
title: TestEvo-Bench
emoji: 🧪
colorFrom: indigo
colorTo: green
sdk: static
pinned: false
short_description: Live benchmark for test generation and test update.
thumbnail: >-
  https://cdn-uploads.huggingface.co/production/uploads/69fa059362e7e8f47d7c5aa2/14h690a494OmjiPpt6elN.png
---

# TestEvo-Bench

<div align="center">

## A Live Benchmark for Test Generation & Test Update

Evaluating how AI agents understand and adapt tests to real-world software evolution.

🌐 https://www.testevo-bench.com/

</div>

---

TestEvo-Bench is a live benchmark for evaluating AI software engineering agents on realistic software test evolution tasks mined from open-source repositories.

Unlike traditional benchmarks that isolate tests from production changes, TestEvo-Bench models real software co-evolution between production code and test suites.

The benchmark contains two complementary tracks:

- 🟠 **Test Generation** — generate new tests for newly introduced behavior
- 🟣 **Test Update** — repair or adapt outdated tests after code changes

Each task is execution-grounded with runnable environments and evaluated using metrics such as pass rate, coverage, and mutation score.

## Datasets

- Test Generation — https://huggingface.co/datasets/TestEvo-Bench/teb-generation

- Test Update — https://huggingface.co/datasets/TestEvo-Bench/teb-update

## Links

- 🌐 Website — https://www.testevo-bench.com/

- 🤗 Hugging Face Space — https://huggingface.co/spaces/TestEvo-Bench/

-  💻 Code — https://anonymous.4open.science/r/testevo-bench-1150/README.md


---

<div align="center">

Real-world • Execution-grounded • Live software evolution benchmark

</div>