README.md

metadata

title: OpenHands Index
emoji: 🤖
colorFrom: yellow
colorTo: yellow
sdk: docker
app_file: app.py
pinned: true
license: apache-2.0
hf_oauth: true
app_port: 7860
failure_strategy: none
tags:
  - leaderboard
thumbnail: >-
  https://huggingface.co/spaces/OpenHands/openhands-index/resolve/main/assets/openhands-index-preview.jpg
short_description: A Holistic Benchmark for Software Engineering

OpenHands Index Leaderboard

This leaderboard tracks agent performance across multiple software engineering and AI benchmarks.

Development

The leaderboard is built using the HuggingFace Datasets library, which provides a convenient way to manage and query datasets. Results are sourced from the OpenHands Index Results repository.

To run the leaderboard locally first make sure to set this env variable:

export IS_INTERNAL=true

You can then start it up with the following command:

python app.py

This will start a local server that you can access in your web browser at http://localhost:7860.

Hugging Face Integration

The repo backs the Hugging Face space at https://huggingface.co/spaces/OpenHands/openhands-index

Please follow the steps below to push changes to the leaderboard on Hugging Face.

Before pushing, make sure to merge your changes to the main branch of this repository (following the standard GitHub workflow of creating a branch, making changes, and then merging it back to main).

Then, to push the changes to the Hugging Face leaderboard:

git push origin main