BioMysteryBench (public sample)

A 5 problem sample from the BioMysteryBench benchmark created by Anthropic.

Contents

• problems.csv / problems.parquet — one row per problem:
  • id — problem identifier
  • question — the task prompt shown to the model
  • answer_rubric — the grading criterion (contains the expected answer)
  • allowed_domains — network domains the solving environment may reach
  • human_solvable — yes if at least one human benchmarker solved it, no if none did
• data.zip (sample) or data/<id>.zip (full set) — each problem's data files; extract into the working directory before solving

For access to the full benchmark, please request access here

License and terms of use

Problem statements, rubrics, and task formulation are released under CC BY 4.0. Access is conditioned on agreeing not to use these benchmark materials to train, fine-tune, or distill machine-learning models.

The data/ archives are anonymized derivatives of public-archive submissions and remain subject to their original repositories' data-use policies. See LICENSE.