Clarification on Pre-Hackathon Preparation, Reproduction Rules, and Judging Criteria

#1
by Gro97 - opened

Hello organizers,

Thank you for putting together the NeurIPS 2026 ML Intern Reproduction Hackathon. I'm looking forward to participating and contributing a scientifically meaningful, independently reproducible submission.

I would appreciate clarification on a few points before the official kickoff:

  1. Pre-hackathon preparation: Can participants select papers, prepare paper-specific implementations, and develop experimental pipelines before November 30, provided the actual reproduction experiments are performed after kickoff? Or must all paper-specific work begin when the competition officially starts?

  2. Judging criteria: Beyond reproducing the reported results, how will submissions be evaluated? In particular, how much emphasis will be placed on independent validation, additional experiments, statistical rigor, and original scientific findings?

  3. Submission strategy: Is there an advantage to reproducing multiple papers, or can a single exceptionally thorough reproduction compete equally for the main awards?

  4. Tools and compute: Can participants use external GPU infrastructure, their own GitHub repositories, and coding agents such as Codex alongside ML Intern? Are there any requirements regarding agent traces or documenting human intervention?

  5. Reproducibility expectations: Will judges independently execute submitted experiments, and are there compute, runtime, or documentation requirements that participants should plan for?

Any guidance on these points would be very helpful as we prepare while ensuring full compliance with the competition rules.

Thank you for your time and for organizing this opportunity.

Neurips 2026 ML Intern Reproductions org

Great questions! We actually have a dedicated Discord for these discussions, so I'd recommend asking there whenever possible. You'll likely get answers much faster, and it'll help others who might have similar questions too: https://discord.gg/PPKUsJ9s9J

  1. We trust that participants will work on their submissions during the official hackathon period.
  2. We're working on publishing an evaluation rubric soon, so you'll have a better idea of what we're looking for and can assess your submission before the final deadline.
  3. That's entirely up to you! I'd recommend aiming for multiple award categories where relevant to improve your chances of winning.
  4. Yes, all coding agents are allowed! We're also offering $20 in Hub credits, which can be useful for trying out ML Intern. It provides a Hub-native experience, making it easier to work with Hub artifacts and push your results directly.
  5. We'll primarily use agents to help review submissions, along with manual checks to make sure submissions are complete and meet the requirements.

Hope this helps! Super excited to have you joining us for the hackathon. Can't wait to see what you come up with! ๐Ÿ™Œ

Sign up or log in to comment