Published 2026-07-09 · Shanghai Jiao Tong University · Yifan Zhou, Qihao Yang, Yan Li et al.

Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation

Abstract (verbatim)

Scientific ideas rarely start from a blank page. They inherit mechanisms, repair known limitations, and recombine pieces of earlier work, much like biological genomes. Current benchmarks still say little about whether AI systems can follow this inheritance structure. We present IdeaGene-Bench (IG-Bench), a benchmark for scientific lineage reasoning and lineage-grounded idea generation. IG-Bench is organized around the IdeaGene framework: each paper or proposal is represented as a set of minimal, typed, evidence-grounded Idea Genome objects, and a GenomeDiff aligns these objects to record inheritance, mutation, loss, external import, and novel insertion under six operational evolutionary dynamics. The benchmark contains 1,961 golden lineage traces, 1,085 curated Idea Genome objects, and 920 pairwise GenomeDiff records across 10 scientific domains. It supports two evaluations. IG-Exam (42 task types, 1,029 instances) tests closed-form lineage reasoning across Idea Genome abstraction, inheritance tracing, evolutionary reasoning, and lineage verification. IG-Arena evaluates generation with a lineage-conditioned Population-Evolution Score(PES), asking whether a proposal can be inserted as a coherent descendant of a given lineage population: it should inherit the right Idea Genome objects, vary meaningfully from nearby work, and offer selection value for future research. Experiments on 14 LLM-based scientists expose a compositional bottleneck. The strongest system reaches only 27.3% exact accuracy on lineage reasoning, and structured lineage context reshuffles system rankings rather than helping every participant uniformly.

Background

Background Analysis

Scientific innovation does not emerge from a blank slate—new ideas often build on the "genetic" history of prior work: inheriting core mechanisms, fixing known limitations, or recombining past findings, much like biological genomes evolve through variation. However, current AI systems struggle to understand this "scientific lineage." The core need for such technology is to help researchers trace research threads and validate the rationality of innovations—for example, assisting biologists in analyzing the historical dependencies of drug development or enabling materials scientists to quickly locate improvement paths for certain materials. Yet, existing benchmarks mostly focus on single tasks (e.g., document classification or summarization) and fail to capture the dynamic process of scientific idea evolution.

Previous methods were stuck at two levels: first, a lack of structured representation of scientific lineage, making it impossible to clearly distinguish key operations like "inheritance," "mutation," and "recombination"; second, oversimplified evaluation criteria, which typically only test whether models can identify text similarity rather than understand the logical chain of idea evolution. For instance, existing models might perform well in judging whether two papers are related but fail to explain "why this paper is an improvement over previous work." This limitation prevents AI from truly supporting research activities requiring deep context, such as designing coherent research projects or predicting the potential value of new ideas.

The solution proposed in this paper is to build a benchmark framework called IdeaGene-Bench (IG-Bench), which models scientific ideas as "genome" objects (Idea Genome) and records their evolutionary dynamics (e.g., inheritance, mutation, or external introduction) through "genome diffs." The framework includes two types of tasks: lineage reasoning (e.g., tracing the origin of a mechanism) and conditional generation (e.g., generating a reasonable new idea based on a given lineage). Experiments show that even advanced AI models (e.g., GPT-4) achieve only about 27% accuracy in complex lineage reasoning, and lineage context significantly alters model performance, indicating that current systems have not yet grasped the "genetic logic" of scientific innovation.

The key difference from previous work is that IG-Bench focuses not only on text matching but also on modeling structured evolutionary processes. For example, it evaluates whether a generated result provides "selection value" for future research using a "Population-Evolution Score" (PES), rather than just generating syntactically correct text. This perspective treats scientific innovation as a dynamically evolving system rather than a static knowledge base, thus setting a more realistic standard for evaluating AI systems in line with genuine research needs.