Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation
Abstract
A benchmark for scientific lineage reasoning and idea generation is introduced, organizing scientific works as genetic-like Idea Genome objects and evaluating both reasoning and generation capabilities.
Scientific ideas rarely start from a blank page. They inherit mechanisms, repair known limitations, and recombine pieces of earlier work, much like biological genomes. Current benchmarks still say little about whether AI systems can follow this inheritance structure. We present IdeaGene-Bench (IG-Bench), a benchmark for scientific lineage reasoning and lineage-grounded idea generation. IG-Bench is organized around the IdeaGene framework: each paper or proposal is represented as a set of minimal, typed, evidence-grounded Idea Genome objects, and a GenomeDiff aligns these objects to record inheritance, mutation, loss, external import, and novel insertion under six operational evolutionary dynamics. The benchmark contains 1,961 golden lineage traces, 1,085 curated Idea Genome objects, and 920 pairwise GenomeDiff records across 10 scientific domains. It supports two evaluations. IG-Exam (42 task types, 1,029 instances) tests closed-form lineage reasoning across Idea Genome abstraction, inheritance tracing, evolutionary reasoning, and lineage verification. IG-Arena evaluates generation with a lineage-conditioned Population-Evolution Score(PES), asking whether a proposal can be inserted as a coherent descendant of a given lineage population: it should inherit the right Idea Genome objects, vary meaningfully from nearby work, and offer selection value for future research. Experiments on 14 LLM-based scientists expose a compositional bottleneck. The strongest system reaches only 27.3% exact accuracy on lineage reasoning, and structured lineage context reshuffles system rankings rather than helping every participant uniformly.
Community
Step length seems underappreciated from the image, can be measured as bits of fuzzing needed, maybe as a "removal of idea from trained model direction", and then probably from a positive direction as well, though that one seems more initialized and absolute rather than the relative removal tactic.
Great point. In this version we focus mainly on characterizing the type of evolutionary move, e.g., mutation, radiation, hybridization, etc.
The “step length” of an idea transition is a very interesting direction for future work: measuring how much information/edit cost is needed to transform a parent Idea Genome into a child one.
I also like the removal/addition framing. Removal gives a relative, contrastive notion of distance, while positive construction may capture how much new structure is needed to instantiate the idea.
🚀 When Auto Research Meets Evolutionary Biology: Ideas Have Genomes
The real bottleneck is whether an AI Scientist can understand where scientific ideas come from, how they evolve, and why they are worth pursuing.✨
We release Ideas Have Genomes 🧬 and introduce IdeaGene-Bench, a benchmark for scientific lineage reasoning and lineage-grounded idea generation. The key idea is simple: scientific ideas are not isolated papers. They have genomes. An idea can inherit mechanisms, repair limitations, recombine with other lineages, and radiate into new problem niches.
IdeaGene-Bench contains two tracks:
📌 IdeaGene-Exam tests whether models can understand the lineage structure of ideas, including Idea Genome abstraction, inheritance tracing, evolutionary reasoning, and lineage verification.
📌 IdeaGene-Arena tests whether models can generate new research ideas grounded in an existing scientific lineage, evaluated by Heredity, Variation, and Selection.
Our experiments show that the strongest system still reaches only 27.3% accuracy on scientific lineage reasoning, suggesting that writing a plausible proposal is still far from understanding research lineages and producing ideas with real taste.
Project page: https://visionxlab.github.io/IdeasHaveGenomes/
arXiv: https://arxiv.org/abs/2607.08758
Discord: https://discord.gg/SrAvenJgg
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper