Season opening dates, Season 1 ties, and how resubmissions are scored

#1
by 1Dmeds - opened

Hello organizers, thank you for running the Open Quantum Challenge.

  1. Opening dates. The Space lists both seasons as opening on 2026-10-11, while the harness README lists 2026-10-20 for Season 1 and 2027-02-10 for Season 2. Which dates govern prize eligibility, and do entries submitted now count toward the final ranking?

  2. Season 1 ties. Several entries already have MAE = 0.0 using exact diagonalization, which the quickstart itself uses. Are exact-diagonalization entries eligible for prizes, and how are equal scores ranked (submission time, or another criterion)?

  3. Resubmissions. When a participant submits more than once on a board, does the leaderboard keep the best score or the latest one?

Thank you for clarifying.

FINAL_Bench org

Thanks for the clear questions. Answers below.

  1. Opening dates. Both seasons are open as of 2026-10-11, which is what the Space shows. The README has been corrected to match, so there is no longer a discrepancy. Every submission from 2026-10-11 onward counts toward the final rankings. Deadlines are 2027-02-28 (Season 1) and 2027-03-31 (Season 2).

  2. Season 1 ties and exact diagonalization. You are right. For the current small Season 1 systems, exact diagonalization is a valid solution, so an MAE of 0.0 is reachable and multiple entries will tie there. Season 1 is therefore provisional: we are revising it to add larger systems that are beyond exact reach and to rank by achieved energy, so that method quality separates entries rather than copying the exact answer. Season 2 (QEC) is unaffected and runs as is.

  3. Resubmissions. The leaderboard shows your best score, not the most recent, so entering multiple times is fine.

We will post the Season 1 revision in this thread when it lands. Thanks for helping tighten the rules.

FINAL_Bench org

Follow-up: Season 1 has been revised to remove the exact-diagonalization tie problem, and the changes are live now.

What changed:

  • Ranking is by the lowest total ground-state energy you achieve across all systems, not MAE versus the exact answer. By the variational principle any valid state gives an energy at or above the true ground state, so the task is to get as low as you can. A reported energy below the exact ground state is rejected as unphysical.
  • You must report every system. Incomplete submissions are not ranked.
  • Larger systems were added (2x4 and 3x3 Hubbard); the set now has 9 systems.
  • Ties are broken by submission time, earliest first.
  • The known floor, the sum of exact ground-state energies, is shown on the board as a reference anchor.

Because the system set and metric changed, the Season 1 board was reset. Please resubmit for the new 9-system set. Season 2 (QEC) is unchanged. Thanks again for catching this.

Sign up or log in to comment