QEC Decoder: 4 submissions from sdoygb, the board shows only the 2nd (0.0127) — later entries missing

#2
by sdoygb - opened

Hi organizers — I want to flag a leaderboard inconsistency on the Season 2 QEC board.

Participant: sdoygb. I have made four submissions on this board. The leaderboard currently shows one entry:
0.0127, method "Tesseract decoder (det_beam=5, merge_errors) on the public decomposed DEM", rank 7.
That corresponds to my second submission.

The four submissions, in order (times approximate, KST):

# what was submitted note
1 MWPM (pymatching 2.4.0) on the public DEM 0.0127 — equal to the MWPM baseline anchor
2 Tesseract decode displayed now as 0.0127
3 Tesseract (det_beam=5, det_penalty=0.5, merge_errors off) file final2_qec_predictions.json, 150,001 B, sha256 08b5678cb5c17f54; local self-test 0.00770 (20,000 shots) / 0.00750 (8,000 shots)
4 same 10,000 predictions as #3, pretty-printed file final3_qec_predictions_pretty.json, 180,002 B, sha256 413e1e3abfc30d44 (parses fine, 10,000 keys)

Two things look wrong to me:

  1. #2 is labelled "Tesseract" but scores exactly 0.0127, which is the MWPM baseline value. A Tesseract decode of this DEM gives ~0.0077 in our local tests, so the scored entry and the method label do not seem to come from the same file.
  2. #3 and #4 are not shown at all. Under the stated rule ("the leaderboard shows your best score, not the most recent"), a 0.0077 entry should have replaced 0.0127.

Could you please:

  • check the submission log for participant sdoygb on the QEC board: which of the four were received, and what LER each scored;
  • if #3/#4 were received, re-score them against the private ground truth and display the best;
  • tell me whether a later, worse-scoring submission is supposed to leave an earlier better one in place (that would explain #2 being shown, but not why #3/#4 are absent).

The queue counter shows {"submitted": 16, "ranked": 9} for the QEC board right now, which suggests several received entries are not ranked — mine may be among them.

I can provide the exact files (same sha256) on request.

Thanks!

Sign up or log in to comment