Search

Recent research on the decline in the paper disruption index (D-index) has sparked heated debates among scholars and garnered significant attention from policymakers and research institution leaders globally. To bridge the gap between policymakers' interest and scholars' skepticism about the D-index, we present this article summarizing key insights from our eight-year investigation, including interviews with scientists across nine disciplines and an analysis of 41 million papers over six decades. Our work confirms the decline in disruptive papers, addresses relevant technical concerns, and makes several original contributions: we clarify that the D-index measures how new ideas render old ones obsolete, suggesting 'Displacing' as an alternative interpretation for 'D'; we show that federal funding agencies like the NIH and NSF are less likely to support disruptive research; and we introduce the 'principle of functional equivalence' to explain the origins of recombining and displacing mechanisms in science, stressing that not all innovation problems are combinatorial, and challenging the belief that AI is the mature solution to scientific innovation. This article aims to promote broader and more accurate use of the D-index in research evaluation and to inspire new funding mechanisms for scientific breakthroughs., Comment: We submitted this paper as a summary of preliminary results. As our research has progressed, we have significantly broadened the scope of the original findings in various directions. We are now considering reorganizing the results into multiple separate papers. To avoid duplicate publication, we would like to withdraw the earlier version

The past half-century has seen a dramatic increase in the scale and complexity of scientific research, to which researchers have responded by dedicating more time to education and training, narrowing their areas of specialization, and collaborating in larger teams. A widely held view is that such collaborations, by fostering specialization and encouraging novel combinations of ideas, accelerate scientific innovation. However, recent research challenges this notion, suggesting that small teams and solo researchers consistently disrupt science and technology with fresh ideas and opportunities, while larger teams tend to refine existing ones (Wu et al. 2019). This study, along with other relevant research, has garnered attention for challenging the zeitgeist of our time that views collaboration as the inevitable path forward in scientific and technological advancement. Yet, few studies have re-evaluated its central finding: the innovative advantage of small teams over large ones, using alternative measures. We explore innovation by identifying papers proposing new scientific concepts and patents introducing new technology codes. We analyzed 88 million research articles spanning from 1800 to 2020 and 7 million patent applications from 1976 to 2020 worldwide. Our findings confirm that while large teams contribute to development, small teams play a critical role in innovation by propelling fresh, original ideas in science and technology.

Scholars are often categorized into two types: hedgehogs (specialists), who focus on working within a specific research field, and foxes (generalists), who actively contribute to a variety of fields. Despite the familiar anecdotes and popularity of this distinction, its empirical foundation has remained largely unexamined. We examine whether the research style of being a fox or a hedgehog is a stable personal trait or an evolving strategy over a scientist's career. Analyzing 2.3 million scholars' publication records over a century, we find that research styles exhibit remarkable stability. Notably, the proportion of fox-like scientists has dramatically declined in the past century, a phenomenon we term "the death of Renaissance scientists." This decline is particularly significant as science shifts toward team collaboration. Teams of foxes consistently outperform teams of hedgehogs in generating new ideas and directions, as confirmed by two emerging innovation metrics for papers: atypicality and disruption. Our research is the first to quantify the process and consequences of the decline of Renaissance scientists. By doing so, we establish a universal link between research styles, demographic shifts, and innovative output.

Theories of innovation emphasize the role of social networks and teams as facilitators of breakthrough discoveries. Around the world, scientists and inventors today are more plentiful and interconnected than ever before. But while there are more people making discoveries, and more ideas that can be reconfigured in novel ways, research suggests that new ideas are getting harder to find-contradicting recombinant growth theory. In this paper, we shed new light on this apparent puzzle. Analyzing 20 million research articles and 4 million patent applications across the globe over the past half-century, we begin by documenting the rise of remote collaboration across cities, underlining the growing interconnectedness of scientists and inventors globally. We further show that across all fields, periods, and team sizes, researchers in these remote teams are consistently less likely to make breakthrough discoveries relative to their onsite counterparts. Creating a dataset that allows us to explore the division of labor in knowledge production within teams and across space, we find that among distributed team members, collaboration centers on late-stage, technical tasks involving more codified knowledge. Yet they are less likely to join forces in conceptual tasks-such as conceiving new ideas and designing research-when knowledge is tacit. We conclude that despite striking improvements in digital technology in recent years, remote teams are less likely to integrate the knowledge of their members to produce new, disruptive ideas.

Science is built on the scholarly consensus that shifts with time. This raises the question of how new and revolutionary ideas are evaluated and become accepted into the canon of science. Using two recently proposed metrics, we identify papers with high atypicality, which models how research draws upon novel combinations of prior research, and evaluate disruption, which captures the degree to which a study creates a new direction by eclipsing its intellectual forebears. Atypical papers are nearly two times more likely to disrupt science than conventional papers, but this is a slow process taking ten years or longer for disruption scores to converge. We provide the first computational model reformulating atypicality as the distance across latent knowledge spaces learned by neural networks. The evolution of this knowledge space characterizes how yesterday's novelty forms today's scientific conventions, which condition the novelty--and surprise--of tomorrow's breakthroughs.