Spaces:
Sleeping
Sleeping
File size: 3,413 Bytes
751b697 9fff15f 3c73799 9fff15f 3c73799 f52a0a7 19b8f85 3c73799 f52a0a7 3270d98 19b8f85 b8ea1f1 3c73799 b8ea1f1 19b8f85 f52a0a7 19b8f85 9fff15f 3c73799 9fff15f 3c73799 9fff15f 3c73799 9fff15f 751b697 3c73799 fc8d954 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 | from .formatting import styled_warning
# ================================
# = ABOUT =
# ================================
INTRODUCTION_TITLE = """<h1 align="center">🏟️ BenchName </h1>"""
INTRODUCTION_TEXT = """🏟️ **BenchName** is a suite of benchmarks for code-related tasks with large contexts, up to a whole code repository.
It currently spans six different tasks and contains six datasets:
* 🤗 [Library-based code generation](https://huggingface.co/datasets/icmlbenchname/library-based-code-generation)
* 🤗 [CI builds repair](https://huggingface.co/datasets/icmlbenchname/ci-builds-repair)
* 🤗 [Project-level code completion](https://huggingface.co/datasets/icmlbenchname/project-level-code-completion)
* 🤗 [Commit message generation](https://huggingface.co/datasets/icmlbenchname/commit-message-generation)
* 🤗 [Bug localization](https://huggingface.co/datasets/icmlbenchname/bug-localization)
* 🤗 [Module summarization](https://huggingface.co/datasets/icmlbenchname/module-summarization)
We are excited to invite you to participate in solving our benchmarks! To submit your results, please send the following materials to our 📩 email (icmlbenchname@gmail.com):
* **Results**: Include the summary of your benchmark outcomes.
* **Reproduction Package**: To ensure the integrity and reproducibility of your results, please include the code for context collection (if any), generation of predictions, and evaluating. You can follow [baselines](https://anonymous.4open.science/r/icml-benchname-2025/README.md) as a reference.
* **Metadata**: Model information, organization name, licence of your model, context size, and other information you find relevant.
We look forward to reviewing your innovative solutions!
"""
# ================================
# = LEADERBOARD =
# ================================
LEADERBOARD_TITLE = '<h2 align="center">🏅Leaderboard</h2>'
LEADERBOARD_TEXT = """The raw results from the leaderboard are available in 🤗 [icmlbenchname/results](https://huggingface.co/datasets/icmlbenchname/results)."""
# ================================
# = SUBMISSION =
# ================================
SUBMISSION_TITLE = '<h2 align="center">📩 Make A Submission</h2>'
SUBMISSION_TEXT_INTRO = """Use the form below to submit new results to 🏟️ BenchName. If any problems arise, don't hesitate to contact us by email `TODO` or open a discussion 💛"""
SUBMISSION_TEXT_TASK = """1. Select a task you want to submit results for."""
SUBMISSION_TEXT_METADATA = """2. Fill in some metadata about your submission."""
SUBMISSION_TEXT_FILES = """3. Attach one or more files with your model's predictions.
* If several files are attached, they will be treated as separate runs of the submitted model (e.g., with different seeds), and the metrics will be averaged across runs. For baselines provided by 🏟️ BenchName Team, the results are averaged across 3 runs.
"""
SUBMISSION_TEXT_SUBMIT = """All set! A new PR to 🤗 [icmlbenchname/results](https://huggingface.co/datasets/icmlbenchname/results) should be opened when you press "Submit" button. 🏟️ BenchName Team will review it shortly, and the results will appear in the leaderboard.
⏳ **Note:** It might take some time (up to 40 minutes) for PR to get created, since it involves computing metrics for your submission."""
|