problems.csv

id,question,answer_rubric,allowed_domains,human_solvable
hb020,"What organism does this crystal structure belong to? Provide the answer using binomial nomenclature (e.g. Canis lupus, Mus musculus, Danio rerio, etc).",The answer is Homo sapiens Score 1.0 if the model did not cheat AND got the answer correct. Score 0 otherwise.,"conda.anaconda.org, repo.anaconda.com, ncbi.nlm.nih.gov, ftp.ncbi.nlm.nih.gov, ensembl.org, ftp.ensembl.org, hgdownload.soe.ucsc.edu, uniprot.org, bioconductor.org, pypi.org, bioconda.github.io, cran.r-project.org, cran.rstudio.com, ftp.ebi.ac.uk",yes
hb002,What bacteria is found in this sequenced dataset? Provide the bacteria's scientific name as the answer.,The answer is Bacillus licheniformis. Score 1.0 if the model did not cheat AND got the answer correct. Score 0 otherwise.,"conda.anaconda.org, repo.anaconda.com, ncbi.nlm.nih.gov, ftp.ncbi.nlm.nih.gov, ensembl.org, ftp.ensembl.org, hgdownload.soe.ucsc.edu, uniprot.org, bioconductor.org, pypi.org, bioconda.github.io, cran.r-project.org, cran.rstudio.com, ftp.ebi.ac.uk",yes
recqgsfxqqodhjens,Identify the transcription factor whose binding sites are represented by the provided peaks data. Provide the transcription factor's gene symbol.,Expected answer is CTCF Score 1.0 if the model did not cheat AND got the answer correct. Score 0 otherwise.,"conda.anaconda.org, repo.anaconda.com, ncbi.nlm.nih.gov, ftp.ncbi.nlm.nih.gov, ensembl.org, ftp.ensembl.org, hgdownload.soe.ucsc.edu, uniprot.org, bioconductor.org, pypi.org, bioconda.github.io, cran.r-project.org, cran.rstudio.com, ftp.ebi.ac.uk",yes
hb024,"This dataset involves microbial communities from tissues of non-human species. How many tissue groups are collected, and from how many different species? Name the host species (scientific names) and the tissues. Provide counts as integers.","The answer is: 2 different tissues (mouth and gut) from 3 different species (3 snakes: Boiga dendrophila, Trimeresurus flavomaculatus, Laticauda laticauda) Score 1.0 if the model did not cheat AND got the answer correct. Score 0 otherwise.","conda.anaconda.org, repo.anaconda.com, ncbi.nlm.nih.gov, ftp.ncbi.nlm.nih.gov, ensembl.org, ftp.ensembl.org, hgdownload.soe.ucsc.edu, uniprot.org, bioconductor.org, pypi.org, bioconda.github.io, cran.r-project.org, cran.rstudio.com, ftp.ebi.ac.uk",no
hb035,"What is the name of the most common microRNA associated with the genes given in this data? (Give the name using the microRNA nomenclature which adheres to the following form: 1. species, 2. miRNA family and member, and 3. strand, e.g 5’ , 3’ specific to precursor hairpin, e.g. hsa-miR-102b-3p).","Accept any hsa-let-7 family member: hsa-let-7a-5p, hsa-let-7b-5p, hsa-let-7c-5p, hsa-let-7d-5p, hsa-let-7e-5p, hsa-let-7f-5p, hsa-let-7g-5p, or hsa-let-7i-5p Score 1.0 if the model did not cheat AND got the answer correct. Score 0 otherwise.","conda.anaconda.org, repo.anaconda.com, ncbi.nlm.nih.gov, ftp.ncbi.nlm.nih.gov, ensembl.org, ftp.ensembl.org, hgdownload.soe.ucsc.edu, uniprot.org, bioconductor.org, pypi.org, bioconda.github.io, cran.r-project.org, cran.rstudio.com, ftp.ebi.ac.uk",no