Replace all AstaBench references with OpenHands Index

- Updated branding across all files (about.py, content.py, etc.)
- Rewrote about.py to focus on software engineering benchmarks
- Updated descriptions to reflect OpenHands' focus on coding agents
- Maintained original design and layout structure

about.py

@@ -3,50 +3,52 @@ import gradio as gr
 
 def build_page():
     with gr.Column(elem_id="about-page-content-wrapper"):
-        # --- Section 1: About AstaBench ---
+        # --- Section 1: About OpenHands Index ---
         gr.HTML(
             """
-            <h2>About AstaBench</h2>
+            <h2>About OpenHands Index</h2>
             <p>
-                AstaBench is a novel AI agents evaluation framework, providing a challenging new test for AI agents: the first benchmark challenge that evaluates agents’ scientific abilities on a broad spectrum of research skills, including literature understanding, data analysis, planning, tool use, coding, and search. Asta’s set of standard tools makes it easy to build general-purpose science agents and to compare their performance in an apples-to-apples manner.
+                OpenHands Index is a comprehensive leaderboard that tracks the performance of AI coding agents across multiple software engineering benchmarks. It provides a unified view of agent capabilities in areas like code generation, bug fixing, repository-level tasks, and complex reasoning challenges. The index makes it easy to compare agents' performance in an apples-to-apples manner across diverse evaluation scenarios.
             </p>
             """
         )
         gr.Markdown("---", elem_classes="divider-line")
 
-        # --- Section 2: Why AstaBench? ---
+        # --- Section 2: Why OpenHands Index? ---
         gr.HTML(
             """
-            <h2>Why AstaBench?</h2>
+            <h2>Why OpenHands Index?</h2>
             <p>
-                Most current benchmarks test agentic AI and isolated aspects of scientific reasoning, but rarely evaluate AI agentic behavior rigorously or capture the full skill set scientific research requires. Agents can appear effective despite inconsistent results and high compute use, often outperforming others by consuming more resources. Advancing scientific AI requires evaluations that emphasize reproducibility, efficiency, and the real complexity of research.
+                Software engineering benchmarks are scattered across different platforms and evaluation frameworks, making it difficult to compare agent performance holistically. Agents may excel at one type of task but struggle with others. Understanding the true capabilities of coding agents requires comprehensive evaluation across multiple dimensions.
             </p>
             <br>
             <p>
-                AstaBench fills this gap: an agents evaluation framework and suite of open benchmarks for evaluating scientific AI assistants on core scientific tasks that require novel reasoning. AstaBench helps scientists identify which agents best support their needs through task-relevant leaderboards, while giving AI developers a standard execution environment and tools to test the scientific reasoning capabilities of their agents compared to well-known baselines from the literature, including both open and closed LLM foundation models.
+                OpenHands Index fills this gap by providing a unified leaderboard aggregating results from diverse software engineering benchmarks. It helps developers and researchers identify which agents best suit their needs, while providing standardized metrics for comparing agent performance across tasks like repository-level editing, multimodal understanding, and commit message generation.
             </p>
             """
         )
         gr.Markdown("---", elem_classes="divider-line")
 
-        # --- Section 3: What Does AstaBench Include? ---
+        # --- Section 3: What Does OpenHands Index Include? ---
         gr.HTML(
             """
-            <h2>What Does AstaBench Include?</h2>
+            <h2>What Does OpenHands Index Include?</h2>
             <p>
-                AstaBench includes a rigorous agents evaluation framework and a suite of benchmarks consisting of over 2,400 problems across 11 benchmarks, organized into four core categories:
+                OpenHands Index aggregates results from 6 key benchmarks for evaluating AI coding agents:
             </p>
             <ul class="info-list">
-                <li>Literature Understanding</li>
-                <li>Code & Execution</li>
-                <li>Data Analysis</li>
-                <li>End-to-End Discovery</li>
+                <li><strong>SWE-bench</strong>: Repository-level bug fixing from real GitHub issues</li>
+                <li><strong>Multi-SWE-bench</strong>: Multi-repository software engineering tasks</li>
+                <li><strong>SWE-bench Multimodal</strong>: Bug fixing with visual context</li>
+                <li><strong>SWT-bench</strong>: Web development and testing tasks</li>
+                <li><strong>Commit0</strong>: Commit message generation and code understanding</li>
+                <li><strong>GAIA</strong>: General AI assistant tasks requiring reasoning and tool use</li>
             </ul>
             <p>
-                Plus: a large suite of integrated agents and leaderboards with results from extensive evaluation of agents and models.
+                Plus: comprehensive leaderboards showing performance across models, agents, and configurations.
             </p>
             <p>
-                🔍 Learn more in the <a href="https://allenai.org/blog/astabench" target="_blank" class="primary-link-button">AstaBench technical blog post</a>
+                🔍 Learn more at <a href="https://github.com/OpenHands/OpenHands" target="_blank" class="primary-link-button">github.com/OpenHands/OpenHands</a>
             </p>
             """
         )
@@ -57,18 +59,19 @@ def build_page():
             """
             <h2>Understanding the Leaderboards</h2>
             <p>
-                The AstaBench Overall Leaderboard provides a high-level view of overall agent performance and efficiency:
+                The OpenHands Index Overall Leaderboard provides a high-level view of agent performance and efficiency:
             </p>
             <ul class="info-list">
-                <li>Overall score: A macro-average of the four category-level averages (equal weighting)</li>
-                <li>Overall cost: Average cost per task, aggregated only across benchmarks with reported cost</li>
+                <li><strong>Overall score</strong>: A macro-average across all benchmarks (equal weighting)</li>
+                <li><strong>Overall cost</strong>: Average cost per task in USD, aggregated across benchmarks with reported cost</li>
             </ul>
             <p>
-                Each category leaderboard provides:
+                Individual benchmark pages provide:
             </p>
             <ul class="info-list">
-                <li>Average score and cost for that category (macro-averaged across the benchmarks in the category)</li>
-                <li>A breakdown by individual benchmarks</li>
+                <li>Detailed scores and metrics for that specific benchmark</li>
+                <li>Cost breakdowns per agent</li>
+                <li>Links to submission details and logs</li>
             </ul>
             """
         )
@@ -79,66 +82,43 @@ def build_page():
             """
             <h2>Scoring & Aggregation</h2>
             <p>
-                AstaBench encourages careful, transparent evaluation. Here's how we handle scoring, cost, and partial results:
+                OpenHands Index provides transparent, standardized evaluation metrics:
             </p>
             
             <h3>Scores</h3>
             <ul class="info-list">
-                <li>Each benchmark returns an average score based on per-problem scores</li>
-                <li>All scores are aggregated upward using macro-averaging</li>
-                <li>Partial completions are included (even with poor performance)</li>
+                <li>Each benchmark returns an average score based on per-task performance</li>
+                <li>All scores are aggregated using macro-averaging (equal weight per benchmark)</li>
+                <li>Metrics vary by benchmark (e.g., resolve rate, pass@1, accuracy)</li>
             </ul>
             
             <h3>Cost</h3>
             <ul class="info-list">
-                <li>Costs are reported in USD per task.</li>
+                <li>Costs are reported in USD per task</li>
                 <li>Benchmarks without cost data are excluded from cost averages</li>
-                <li>In scatter plots, agents without cost are plotted to the far right and clearly marked.</li>
+                <li>In scatter plots, agents without cost data are clearly marked</li>
             </ul>
             
             <p>
-                <em>Note: Cost values reflect pricing and infrastructure conditions at a fixed point in time. We recognize that compute costs may change over time and vary by provider, and are actively working on methods to keep costs up-to-date and normalized for fair comparisons.</em>
-            </p>
-            
-            <h3>Coverage</h3>
-            <ul class="info-list">
-                <li>Main leaderboard: category coverage (X/4)</li>
-                <li>Category view: benchmark coverage (X/Y)</li>
-                <li>Incomplete coverage is flagged visually</li>
-            </ul>
-            
-            <p>
-                These design choices ensure fair comparison while penalizing cherry-picking and omissions.
+                <em>Note: Cost values reflect API pricing at evaluation time and may vary based on provider, infrastructure, and usage patterns.</em>
             </p>
             """
         )
         gr.Markdown("---", elem_classes="divider-line")
 
-        # --- Section 6: Learn More ---
+        # --- Section 6: Citation ---
         gr.HTML(
             """
-            <div class="learn-more-section">
-                <h2>Learn More</h2>
-                <div class="link-buttons-container">
-                    
-                    <a href="https://allenai.org/blog/astabench" target="_blank" class="link-button">
-                        <span style="color:#0fcb8c;">AstaBench technical blog post</span>
-                        <span class="external-link-icon">↗</span>
-                    </a>
-                    
-                    <a href="/submit" target="_blank" class="link-button">
-                        <span style="color:#0fcb8c;">Submit an agent for evaluation</span>
-                        <span class="external-link-icon">↗</span>
-                    </a>
-                    
-                </div>
-            </div>
+            <h2>Citation</h2>
+            <p>
+                If you use OpenHands or reference the OpenHands Index in your work, please cite:
+            </p>
+            <pre class="citation-block">
+@misc{openhands2024,
+    title={OpenHands: An Open Platform for AI Software Developers as Generalist Agents},
+    author={OpenHands Team},
+    year={2024},
+    howpublished={https://github.com/OpenHands/OpenHands}
+}</pre>
             """
         )
-        # Floating feedback button
-        floating_feedback_button_html = """
-        <div>
-            <a id="feedback-button" href="https://docs.google.com/forms/d/e/1FAIpQLSfJdVkD62aPYh8XehN2FrSeHUWt488Ejc-QdtuZn5NZ3eNoxA/viewform">Have feedback?</a>
-        </div>
-        """
-        gr.HTML(floating_feedback_button_html)

category_page_builder.py

@@ -17,7 +17,7 @@ def build_category_page(CATEGORY_NAME, PAGE_DESCRIPTION):
         with gr.Row(elem_id="intro-row"):
 
             with gr.Column(scale=1):
-                gr.HTML(f'<h2>AstaBench {CATEGORY_NAME} Leaderboard <span style="font-weight: normal; color: inherit;">(Aggregate)</span></h2>', elem_id="main-header")
+                gr.HTML(f'<h2>OpenHands Index {CATEGORY_NAME} Leaderboard <span style="font-weight: normal; color: inherit;">(Aggregate)</span></h2>', elem_id="main-header")
                 with gr.Column(elem_id="validation_nav_container", visible=False) as validation_nav_container:
                     create_sub_navigation_bar(validation_tag_map, CATEGORY_NAME, validation=True)
 

content.py

@@ -13,11 +13,11 @@ def create_gradio_anchor_id(text: str, validation) -> str:
     return f"h-{text}-leaderboard"
 
 
-TITLE = """<h1 align="left" id="space-title">AstaBench Leaderboard</h1>"""
+TITLE = """<h1 align="left" id="space-title">OpenHands Index</h1>"""
 
 INTRO_PARAGRAPH = """
 <p>
-    <strong>AstaBench</strong> provides an aggregated view of agent performance and efficiency across all benchmarks in all four categories. We report:
+    <strong>OpenHands Index</strong> provides an aggregated view of agent performance and efficiency across all benchmarks in all categories. We report:
 </p>
 
 <ul class="info-list">
@@ -48,7 +48,7 @@ For detailed results, use the links above to explore individual benchmarks.
 <br>
 """
 CODE_EXECUTION_DESCRIPTION = """
-The **Code & Execution** category in AstaBench includes tasks that evaluate an agent’s ability to write, modify, and run code in realistic research scenarios. Unlike literature tasks—which only require read-only tools and can sometimes even be solved by a language model alone—these problems often require the agent to manipulate a machine environment with tools: reading input files, executing code, and writing outputs to specific files in the required format.
+The **Code & Execution** category in OpenHands Index includes tasks that evaluate an agent’s ability to write, modify, and run code in realistic research scenarios. Unlike literature tasks—which only require read-only tools and can sometimes even be solved by a language model alone—these problems often require the agent to manipulate a machine environment with tools: reading input files, executing code, and writing outputs to specific files in the required format.
 <br><br>
 The scores in this category are aggregated from three distinct benchmarks, each targeting different facets of scientific coding and execution. Together, these benchmarks evaluate whether an agent can function as a hands-on scientific assistant—not just by reasoning about code, but by running it in real-world contexts.
 <br><br>
@@ -68,7 +68,7 @@ Scores in this category are aggregated from two benchmarks, providing the first
 <br>
 """
 SUBMISSION_CONFIRMATION = """
-**Your agent has been submitted to AstaBench for evaluation.**
+**Your agent has been submitted to OpenHands Index for evaluation.**
 <br><br>
 🙏 Thanks for contributing!
 <br><br>
@@ -172,8 +172,8 @@ def get_benchmark_description(benchmark_name, validation):
 
 CITATION_BUTTON_LABEL = "Copy the following snippet to cite these results"
 CITATION_BUTTON_TEXT = r"""@article{asta-bench,
-    title={AstaBench},
-    author={AstaBench folks},
+    title={OpenHands Index},
+    author={OpenHands Index folks},
     year={2025},
     eprint={TBD.TBD},
     archivePrefix={arXiv},
@@ -184,11 +184,11 @@ CITATION_BUTTON_TEXT = r"""@article{asta-bench,
 LEGAL_DISCLAIMER_TEXT = """
 <h2>Terms and Conditions</h2>
 <p>
-    The Allen Institute for Artificial Intelligence (Ai2) maintains this repository for agent evaluation submissions to AstaBench. To keep AstaBench fair and auditable, all evaluation logs and associated submission files will be made publicly available. This includes your benchmark inputs, model output responses, and other data and information related to your submission as needed to verify the results.
+    The Allen Institute for Artificial Intelligence (Ai2) maintains this repository for agent evaluation submissions to OpenHands Index. To keep OpenHands Index fair and auditable, all evaluation logs and associated submission files will be made publicly available. This includes your benchmark inputs, model output responses, and other data and information related to your submission as needed to verify the results.
 </p>
 <br>
 <p>
-    Your submissions to AstaBench will be posted, scored, and ranked on the leaderboard at <a href="https://huggingface.co/spaces/allenai/asta-bench-leaderboard" target="_blank" rel="noopener noreferrer">https://huggingface.co/spaces/allenai/asta-bench-leaderboard</a>. You agree you have the rights to the materials you submit and that you will not share any personal, sensitive, proprietary, or confidential information.
+    Your submissions to OpenHands Index will be posted, scored, and ranked on the leaderboard at <a href="https://huggingface.co/spaces/allenai/asta-bench-leaderboard" target="_blank" rel="noopener noreferrer">https://huggingface.co/spaces/allenai/asta-bench-leaderboard</a>. You agree you have the rights to the materials you submit and that you will not share any personal, sensitive, proprietary, or confidential information.
 </p>
 """
 

leaderboard_transformer.py

@@ -535,7 +535,7 @@ def _plot_scatter_plotly(
 
     fig.update_layout(
         template="plotly_white",
-        title=f"AstaBench {name} Leaderboard",
+        title=f"OpenHands Index {name} Leaderboard",
         xaxis=xaxis_config, # Use the updated config
         yaxis=dict(title="Average (mean) score", range=[-0.2, None]),
         legend=dict(

main_page.py

@@ -34,7 +34,7 @@ def build_page():
     # --- Leaderboard Display Section ---
     gr.Markdown("---")
     CATEGORY_NAME = "Overall"
-    gr.HTML(f'<h2>AstaBench {CATEGORY_NAME} Leaderboard <span style="font-weight: normal; color: inherit;">(Aggregate)</span></h2>', elem_id="main-header")
+    gr.HTML(f'<h2>OpenHands Index {CATEGORY_NAME} Leaderboard <span style="font-weight: normal; color: inherit;">(Aggregate)</span></h2>', elem_id="main-header")
 
     with gr.Tabs() as tabs:
         with gr.Tab("Results: Test Set") as test_tab:

submission.py

@@ -357,7 +357,7 @@ agent_tooling_label_html = f"""<div>
 
 heading_html = """
 <h2>🚀 Submit an agent for evaluation</h2>
-<p>Submit your agent to AstaBench for evaluation on real-world scientific tasks. Once submitted, your run will be reviewed by our team. If there are any issues, we’ll reach out within 5–7 business days. We’re working toward full automation, but in the meantime, human review helps ensure quality and trust.</p>
+<p>Submit your agent to OpenHands Index for evaluation on real-world scientific tasks. Once submitted, your run will be reviewed by our team. If there are any issues, we’ll reach out within 5–7 business days. We’re working toward full automation, but in the meantime, human review helps ensure quality and trust.</p>
 <h3>How to run an evaluation</h3>
 <p>Please follow the steps in our <a href="https://github.com/allenai/asta-bench?tab=readme-ov-file#usage" target="_blank">README</a>. You’ll upload your run file at the end of this form.</p>
 """
@@ -372,7 +372,7 @@ def build_page():
             gr.HTML(value="""<h3>Username</h3>""", elem_classes="form-label")
             username_tb = gr.Textbox(label="This will show on the leaderboard. By default, we’ll use your Hugging Face username; but you can enter your organization name instead (e.g., university, company, or lab).")
             gr.HTML(value="""<h3>Role</h3>""", elem_classes="form-label")
-            role = gr.Dropdown(label="Please select the role that most closely matches your current position. Helps us improve AstaBench for different user types. Not displayed on the leaderboard.",
+            role = gr.Dropdown(label="Please select the role that most closely matches your current position. Helps us improve OpenHands Index for different user types. Not displayed on the leaderboard.",
                 interactive=True,
                 choices=[
                     "Undergraduate Student",