Search

Large language models (LLMs) demonstrate impressive zero-shot and few-shot reasoning capabilities. Some propose that such capabilities can be improved through self-reflection, i.e., letting LLMs reflect on their own output to identify and correct mistakes in the initial responses. However, despite some evidence showing the benefits of self-reflection, recent studies offer mixed results. Here, we aim to reconcile these conflicting findings by first demonstrating that the outcome of self-reflection is sensitive to prompt wording; e.g., LLMs are more likely to conclude that it has made a mistake when explicitly prompted to find mistakes. Consequently, idiosyncrasies in reflection prompts may lead LLMs to change correct responses unnecessarily. We show that most prompts used in the self-reflection literature are prone to this bias. We then propose different ways of constructing prompts that are conservative in identifying mistakes and show that self-reflection using such prompts results in higher accuracy. Our findings highlight the importance of prompt engineering in self-reflection tasks. We release our code at https://github.com/Michael98Liu/mixture-of-prompts.

Editors are crucial to the integrity of the scientific publishing process, yet they themselves could face conflicts of interest (COIs), whereby their personal interests interfere with their editorial duties. One such COI stems from the fact that, apart from a few exceptions, the vast majority of editors are research-active scientists with many collaborators. Each such editor could potentially handle submissions from their recent collaborators, allowing the editor to use their power, consciously or otherwise, to treat such submissions favourably, thereby jeopardizing the integrity of the editorial decision. Naturally, a number of policies have been put in place to govern such COI, but their effectiveness remains unknown. We fill this gap by analyzing half a million papers handled by 60,000 different editors and published in 500 journals by six publishers, namely Frontiers, Hindawi, IEEE, MDPI, PLOS, and PNAS. We find numerous papers handled by editors who collaborated recently with the authors; this happens despite policies explicitly prohibiting such behavior. Overall, nearly 3% of journals have a COI rate $\geq$ 10%, and nearly half of them have a COI rate $\geq$ 2%. Moreover, leveraging three quasi-experiments, we find that COI policies have a limited, if any, effect on regulating this phenomenon. Finally, we find that editors are faster to accept submissions from their collaborators, raising the possibility of favoritism. These findings highlight the need for policy reform to assure the scientific community that all submissions are treated equally.

Retracting academic papers is a fundamental tool of quality control when the validity of papers or the integrity of authors is questioned post-publication. While retractions do not eliminate papers from the record, they have far-reaching consequences for retracted authors and their careers, serving as a visible and permanent signal of potential transgressions. Previous studies have highlighted the adverse effects of retractions on citation counts and coauthors' citations; however, the broader impacts beyond these have not been fully explored. We address this gap leveraging Retraction Watch, the most extensive data set on retractions and link it to Microsoft Academic Graph, a comprehensive data set of scientific publications and their citation networks, and Altmetric that monitors online attention to scientific output. Our investigation focuses on: 1) the likelihood of authors exiting scientific publishing following a retraction, and 2) the evolution of collaboration networks among authors who continue publishing after a retraction. Our empirical analysis reveals that retracted authors, particularly those with less experience, tend to leave scientific publishing in the aftermath of retraction, particularly if their retractions attract widespread attention. We also uncover that retracted authors who remain active in publishing maintain and establish more collaborations compared to their similar non-retracted counterparts. Nevertheless, retracted authors with less than a decade of publishing experience retain less senior, less productive and less impactful coauthors, and gain less senior coauthors post-retraction. Taken together, notwithstanding the indispensable role of retractions in upholding the integrity of the academic community, our findings shed light on the disproportionate impact that retractions impose on early-career authors., Comment: 49 pages, 13 figures, 19 tables

The emergence of large language models has led to the development of powerful tools such as ChatGPT that can produce text indistinguishable from human-generated work. With the increasing accessibility of such technology, students across the globe may utilize it to help with their school work -- a possibility that has sparked discussions on the integrity of student evaluations in the age of artificial intelligence (AI). To date, it is unclear how such tools perform compared to students on university-level courses. Further, students' perspectives regarding the use of such tools, and educators' perspectives on treating their use as plagiarism, remain unknown. Here, we compare the performance of ChatGPT against students on 32 university-level courses. We also assess the degree to which its use can be detected by two classifiers designed specifically for this purpose. Additionally, we conduct a survey across five countries, as well as a more in-depth survey at the authors' institution, to discern students' and educators' perceptions of ChatGPT's use. We find that ChatGPT's performance is comparable, if not superior, to that of students in many courses. Moreover, current AI-text classifiers cannot reliably detect ChatGPT's use in school work, due to their propensity to classify human-written answers as AI-generated, as well as the ease with which AI-generated text can be edited to evade detection. Finally, we find an emerging consensus among students to use the tool, and among educators to treat this as plagiarism. Our findings offer insights that could guide policy discussions addressing the integration of AI into educational frameworks., Comment: 17 pages, 4 figures

Attribute inference - the process of analyzing publicly available data in order to uncover hidden information - has become a major threat to privacy, given the recent technological leap in machine learning. One way to tackle this threat is to strategically modify one's publicly available data in order to keep one's private information hidden from attribute inference. We evaluate people's ability to perform this task, and compare it against algorithms designed for this purpose. We focus on three attributes: the gender of the author of a piece of text, the country in which a set of photos was taken, and the link missing from a social network. For each of these attributes, we find that people's effectiveness is inferior to that of AI, especially when it comes to hiding the attribute in question. Moreover, when people are asked to modify the publicly available information in order to hide these attributes, they are less likely to make high-impact modifications compared to AI. This suggests that people are unable to recognize the aspects of the data that are critical to an inference algorithm. Taken together, our findings highlight the limitations of relying on human intuition to protect privacy in the age of AI, and emphasize the need for algorithmic support to protect private information from attribute inference., Comment: 25 pages, 4 figures

Artificial Intelligence (AI) has become a disruptive technology, promising to grant a significant economic and strategic advantage to the nations that harness its power. China, with its recent push towards AI adoption, is challenging the U.S.'s position as the global leader in this field. Given AI's massive potential, as well as the fierce geopolitical tensions between the two nations, a number of policies have been put in place that discourage AI scientists from migrating to, or collaborating with, the other country. However, the extents of such brain drain and cross-border collaboration are not fully understood. Here, we analyze a dataset of over 350,000 AI scientists and 5,000,000 AI papers. We find that, since the year 2000, China and the U.S. have been leading the field in terms of impact, novelty, productivity, and workforce. Most AI scientists who migrate to China come from the U.S., and most who migrate to the U.S. come from China, highlighting a notable brain drain in both directions. Upon migrating from one country to the other, scientists continue to collaborate frequently with the origin country. Although the number of collaborations between the two countries has been increasing since the dawn of the millennium, such collaborations continue to be relatively rare. A matching experiment reveals that the two countries have always been more impactful when collaborating than when each of them works without the other. These findings suggest that instead of suppressing cross-border migration and collaboration between the two nations, the field could benefit from promoting such activities., Comment: 23 pages, 5 figures, 4 tables

Academic publishing is the principal medium of documenting and disseminating scientific discoveries. At the heart of its daily operations are the editorial boards. Despite their activities and recruitment often being opaque to outside observers, they play a crucial role in promoting fair evaluations and gender parity. Literature on gender inequality lacks the connection between women as editors and as research-active scientists, thereby missing the comparison between the gender balances in these two academic roles. Literature on editorial fairness similarly lacks longitudinal studies on the conflicts of interest arising from editors being research active, which motivates them to expedite the publication of their papers. We fill these gaps using a dataset of 103,000 editors, 240 million authors, and 220 million publications spanning five decades and 15 disciplines. This unique dataset allows us to compare the proportion of female editors to that of female scientists in any given year or discipline. Although women are already underrepresented in science (26%), they are even more so among editors (14%) and editors-in-chief (8%); the lack of women with long-enough publishing careers explains the gender gap among editors, but not editors-in-chief, suggesting that other factors may be at play. Our dataset also allows us to study the self-publication patterns of editors, revealing that 8% of them double the rate at which they publish in their own journal soon after the editorship starts, and this behavior is accentuated in journals where the editors-in-chief self-publish excessively. Finally, men are more likely to engage in this behaviour than women.

Disinformation continues to attract attention due to its increasing threat to society. Nevertheless, a disinformation-based attack on critical infrastructure has never been studied to date. Here, we consider traffic networks and focus on fake information that manipulates drivers' decisions to create congestion. We study the optimization problem faced by the adversary when choosing which streets to target to maximize disruption. We prove that finding an optimal solution is computationally intractable, implying that the adversary has no choice but to settle for suboptimal heuristics. We analyze one such heuristic, and compare the cases when targets are spread across the city of Chicago vs. concentrated in its business district. Surprisingly, the latter results in more far-reaching disruption, with its impact felt as far as 2 kilometers from the closest target. Our findings demonstrate that vulnerabilities in critical infrastructure may arise not only from hardware and software, but also from behavioral manipulation., Comment: 25 pages, 5 figures

Inspired by the numerous benefits of mentorship in academia, we study "informal mentorship" in scientific collaborations, whereby a junior scientist is supported by multiple senior collaborators, without them necessarily having any formal supervisory roles. To this end, we analyze 2.5 million unique pairs of mentor-prot\'eg\'es spanning 9 disciplines and over a century of research, and we show that mentorship quality has a causal effect on the scientific impact of the papers written by the prot\'eg\'e post mentorship. This effect increases with the number of mentors, and persists over time, across disciplines and university ranks. The effect also increases with the academic age of the mentors until they reach 30 years of experience, after which it starts to decrease. Furthermore, we study how the gender of both the mentors and their prot\'eg\'e affect not only the impact of the prot\'eg\'e post mentorship, but also the citation gain of the mentors during the mentorship experience with their prot\'eg\'e. We find that increasing the proportion of female mentors decreases the impact of the prot\'eg\'e, while also compromising the gain of female mentors. While current policies that have been encouraging junior females to be mentored by senior females have been instrumental in retaining women in science, our findings suggest that the impact of women who remain in academia may increase by encouraging opposite-gender mentorships instead., Comment: Main: 12 pages and 3 figures. Supplementary: 16 pages, 3 figures, and 10 tables

Social technologies have made it possible to propagate disinformation and manipulate the masses at an unprecedented scale. This is particularly alarming from a security perspective, as humans have proven to be the weakest link when protecting critical infrastructure in general, and the power grid in particular. Here, we consider an attack in which an adversary attempts to manipulate the behavior of energy consumers by sending fake discount notifications encouraging them to shift their consumption into the peak-demand period. We conduct surveys to assess the propensity of people to follow-through on such notifications and forward them to their friends. This allows us to model how the disinformation propagates through social networks. Finally, using Greater London as a case study, we show that disinformation can indeed be used to orchestrate an attack wherein unwitting consumers synchronize their energy-usage patterns, resulting in blackouts on a city-scale. These findings demonstrate that in an era when disinformation can be weaponized, system vulnerabilities arise not only from the hardware and software of critical infrastructure, but also from the behavior of the consumers., Comment: 10 pages, 3 figures

Inspired by the social and economic benefits of diversity, we analyze over 9 million papers and 6 million scientists to study the relationship between research impact and five classes of diversity: ethnicity, discipline, gender, affiliation, and academic age. Using randomized baseline models, we establish the presence of homophily in ethnicity, gender and affiliation. We then study the effect of diversity on scientific impact, as reflected in citations. Remarkably, of the classes considered, ethnic diversity had the strongest correlation with scientific impact. To further isolate the effects of ethnic diversity, we used randomized baseline models and again found a clear link between diversity and impact. To further support these findings, we use coarsened exact matching to compare the scientific impact of ethnically diverse papers and scientists with closely-matched control groups. Here, we find that ethnic diversity resulted in an impact gain of 10.63% for papers, and 47.67% for scientists.