---
title: ML4SE Benchmark Viewer
emoji: 📊
colorFrom: blue
colorTo: green
sdk: docker
pinned: false
---

# ML4SE Benchmark Viewer

A web-based interface for browsing and manually inspecting individual datapoints from popular ML4SE (Machine Learning for Software Engineering) benchmark datasets.

## Supported Datasets (28)

### Code Generation
| Dataset | Source | View Type |
|---------|--------|-----------|
| **HumanEval+** | evalplus/humanevalplus | Simple |
| **MBPP+** | evalplus/mbppplus | Simple |
| **MBPP** | google-research-datasets/mbpp | Simple |
| **ClassEval** | FudanSELab/ClassEval | Simple |
| **LiveCodeBench** | livecodebench/code_generation_lite | Simple |
| **APPS** | codeparrot/apps | Multi-solution |
| **CodeContests** | deepmind/code_contests | Multi-solution |
| **BigOBench** | facebook/BigOBench | Complexity badges |
| **BigCodeBench** | bigcode/bigcodebench | Simple |
| **EffiBench** | DONG19/EffiBench | Simple |

### Code Reasoning & Evaluation
| Dataset | Source | View Type |
|---------|--------|-----------|
| **REval** | JetBrains-Research/REval | Interactive (coverage, arrows, ground truth) |
| **CRUXEval** | cruxeval-org/cruxeval | Given/Predict task selector |
| **HumanEvalPack** | bigcode/humanevalpack | Multi-language + buggy/canonical |

### Code Editing & Debugging
| Dataset | Source | View Type |
|---------|--------|-----------|
| **SWE-bench Lite** | princeton-nlp/SWE-bench_Lite | Unified diff |
| **SWE-bench Verified** | princeton-nlp/SWE-bench_Verified | Unified diff |
| **SWE-bench** | princeton-nlp/SWE-bench | Unified diff |
| **DebugBench** | Rtian/DebugBench | Before/After |
| **CanItEdit** | nuprl/CanItEdit | Before/After |
| **CodeEditorBench** | m-a-p/CodeEditorBench | Before/After |
| **CodeXGLUE Refinement** | google/code_x_glue_cc_code_refinement | Before/After |
| **CommitBench** | Maxscha/commitbench | Unified diff |

### Code Completion & Translation
| Dataset | Source | View Type |
|---------|--------|-----------|
| **SAFIM** | gonglinyuan/safim | Fill-in-the-Middle |
| **HumanEval-X** | THUDM/humaneval-x | Multi-language tabs |
| **CodeSearchNet** | code-search-net/code_search_net | Simple |

### Vulnerability Detection
| Dataset | Source | View Type |
|---------|--------|-----------|
| **BigVul** | bstee615/bigvul | Vulnerability (CWE badges) |
| **DiverseVul** | claudios/DiverseVul | Vulnerability |
| **PrimeVul** | starsofchance/PrimeVul | Vulnerability |
| **Devign** | google/code_x_glue_cc_defect_detection | Vulnerability |

## Installation & Usage

```bash
# Install dependencies
uv sync

# Run the server (default port: 7860)
uv run python app.py

# Development mode with auto-reload
FLASK_DEBUG=true uv run python app.py
```

Then open http://localhost:7860.

## Development

```bash
# Lint and format
uv run ruff check .
uv run ruff format .
```

### Adding a New Dataset

1. Create an adapter class in the appropriate `adapters/` submodule inheriting from `DatasetAdapter`
2. Implement required methods: `problem_count()`, `get_problem_summary()`, `get_problem_detail()`
3. Register the adapter in `adapters/registration.py`
4. Test: `/api/<slug>/problems` and `/api/<slug>/problem/<idx>`