Search

The contribution to pushing the boundaries of knowledge is a critical metric for evaluating the research performance of countries and institutions, which in many cases is not revealed by common bibliometric indicators. The Rk-index was specifically designed to assess such contributions, and the Rn-index is a variant that corrects the weakness of the Rk-index, particularly in the evaluation of countries that produce a high proportion of global advancements. This is the case of the USA and China in many technological fields. Additionally, the Rn-index is simple to calculate and understand, as it involves only summing the ratios between the local and global ranks of papers, ordered by their citation count. Moreover, the Rn-index may also be fractionally counted., Comment: 6 pages; 2 figures; 4 tables

As events progress, news articles often update with new information: if we are not cautious, we risk propagating outdated facts. In this work, we hypothesize that linguistic features indicate factual fluidity, and that we can predict which facts in a news article will update using solely the text of a news article (i.e. not external resources like search engines). We test this hypothesis, first, by isolating fact-updates in large news revisions corpora. News articles may update for many reasons (e.g. factual, stylistic, narrative). We introduce the NewsEdits 2.0 taxonomy, an edit-intentions schema that separates fact updates from stylistic and narrative updates in news writing. We annotate over 9,200 pairs of sentence revisions and train high-scoring ensemble models to apply this schema. Then, taking a large dataset of silver-labeled pairs, we show that we can predict when facts will update in older article drafts with high precision. Finally, to demonstrate the usefulness of these findings, we construct a language model question asking (LLM-QA) abstention task. We wish the LLM to abstain from answering questions when information is likely to become outdated. Using our predictions, we show, LLM absention reaches near oracle levels of accuracy., Comment: 9 pages main body, 11 pages appendix

The multidisciplinary and socially anchored nature of Feminist Studies presents unique challenges for bibliometric analysis, as this research area transcends traditional disciplinary boundaries and reflects discussions from feminist and LGBTQIA+ social movements. This paper proposes a novel approach for identifying gender/sex related publications scattered across diverse scientific disciplines. Using the Dimensions database, we employ bibliometric techniques, natural language processing (NLP) and manual curation to compile a dataset of scientific publications that allows for the analysis of Gender Studies and its influence across different disciplines. This is achieved through a methodology that combines a core of specialized journals with a comprehensive keyword search over titles. These keywords are obtained by applying Topic Modeling (BERTopic) to the corpus of titles and abstracts from the core. This methodological strategy, divided into two stages, reflects the dynamic interaction between Gender Studies and its dialogue with different disciplines. This hybrid system surpasses basic keyword search by mitigating potential biases introduced through manual keyword enumeration. The resulting dataset comprises over 1.9 million scientific documents published between 1668 and 2023, spanning four languages. This dataset enables a characterization of Gender Studies in terms of addressed topics, citation and collaboration dynamics, and institutional and regional participation. By addressing the methodological challenges of studying "more-than-disciplinary" research areas, this approach could also be adapted to delineate other conversations where disciplinary boundaries are difficult to disentangle., Comment: 2 tables, 5 figures

Traditional archival practices for describing electronic theses and dissertations (ETDs) rely on broad, high-level metadata schemes that fail to capture the depth, complexity, and interdisciplinary nature of these long scholarly works. The lack of detailed, chapter-level content descriptions impedes researchers' ability to locate specific sections or themes, thereby reducing discoverability and overall accessibility. By providing chapter-level metadata information, we improve the effectiveness of ETDs as research resources. This makes it easier for scholars to navigate them efficiently and extract valuable insights. The absence of such metadata further obstructs interdisciplinary research by obscuring connections across fields, hindering new academic discoveries and collaboration. In this paper, we propose a machine learning and AI-driven solution to automatically categorize ETD chapters. This solution is intended to improve discoverability and promote understanding of chapters. Our approach enriches traditional archival practices by providing context-rich descriptions that facilitate targeted navigation and improved access. We aim to support interdisciplinary research and make ETDs more accessible. By providing chapter-level classification labels and using them to index in our developed prototype system, we make content in ETD chapters more discoverable and usable for a diverse range of scholarly needs. Implementing this AI-enhanced approach allows archives to serve researchers better, enabling efficient access to relevant information and supporting deeper engagement with ETDs. This will increase the impact of ETDs as research tools, foster interdisciplinary exploration, and reinforce the role of archives in scholarly communication within the data-intensive academic landscape.

The Virginia Tech University Libraries (VTUL) Digital Library Platform (DLP) hosts digital collections that offer our users access to a wide variety of documents of historical and cultural importance. These collections are not only of academic importance but also provide our users with a glance at local historical events. Our DLP contains collections comprising digital objects featuring complex layouts, faded imagery, and hard-to-read handwritten text, which makes providing online access to these materials challenging. To address these issues, we integrate AI into our DLP workflow and convert the text in the digital objects into a machine-readable format. To enhance the user experience with our historical collections, we use custom AI agents for handwriting recognition, text extraction, and large language models (LLMs) for summarization. This poster highlights three collections focusing on handwritten letters, newspapers, and digitized topographic maps. We discuss the challenges with each collection and detail our approaches to address them. Our proposed methods aim to enhance the user experience by making the contents in these collections easier to search and navigate.

As research institutions increasingly commit to supporting the United Nations' Sustainable Development Goals (SDGs), there is a pressing need to accurately assess their research output against these goals. Current approaches, primarily reliant on keyword-based Boolean search queries, conflate incidental keyword matches with genuine contributions, reducing retrieval precision and complicating benchmarking efforts. This study investigates the application of autoregressive Large Language Models (LLMs) as evaluation agents to identify relevant scholarly contributions to SDG targets in scholarly publications. Using a dataset of academic abstracts retrieved via SDG-specific keyword queries, we demonstrate that small, locally-hosted LLMs can differentiate semantically relevant contributions to SDG targets from documents retrieved due to incidental keyword matches, addressing the limitations of traditional methods. By leveraging the contextual understanding of LLMs, this approach provides a scalable framework for improving SDG-related research metrics and informing institutional reporting.

This article analyses the structural and institutional barriers hindering the development of scientific systems in transition economies, such as Kazakhstan. The main focus is on the concept of the "middle science trap," which is characterized by steady growth in quantitative indicators (publications, grants) but a lack of qualitative advancement. Excessive bureaucracy, weak integration into the international scientific community, and ineffective science management are key factors limiting development. This paper proposes an approach of "homeopathic modernization," which focuses on minimal yet strategically significant changes aimed at reducing bureaucratic barriers and enhancing the effectiveness of the scientific ecosystem. A comparative analysis of international experience (China, India, and the European Union) is provided, demonstrating how targeted reforms in the scientific sector can lead to significant results. Social and cultural aspects, including the influence of mentality and institutional structure, are also examined, and practical recommendations for reforming the scientific system in Kazakhstan and Central Asia are offered. The conclusions of the article could be useful for developing national science modernization programs, particularly in countries with high levels of bureaucracy and conservatism., Comment: 29 pages

Sharing and reusing research data can effectively reduce redundant efforts in data collection and curation, especially for small labs and research teams conducting human-centered system research, and enhance the replicability of evaluation experiments. Building a sustainable data reuse process and culture relies on frameworks that encompass policies, standards, roles, and responsibilities, all of which must address the diverse needs of data providers, curators, and reusers. To advance the knowledge and accumulate empirical understandings on data reuse, this study investigated the data reuse practices of experienced researchers from the area of Interactive Information Retrieval (IIR) studies, where data reuse has been strongly advocated but still remains a challenge. To enhance the knowledge on data reuse behavior and reusability assessment strategies within IIR community, we conducted 21 semi-structured in-depth interviews with IIR researchers from varying demographic backgrounds, institutions, and stages of careers on their motivations, experiences, and concerns over data reuse. We uncovered the reasons, strategies of reusability assessments, and challenges faced by data reusers within the field of IIR as they attempt to reuse researcher data in their studies. The empirical finding improves our understanding of researchers' motivations for reusing data, their approaches to discovering reusable research data, as well as their concerns and criteria for assessing data reusability, and also enriches the on-going discussions on evaluating user-generated data and research resources and promoting community-level data reuse culture and standards.

Despite the importance of Indo-US research collaboration, it is intriguing to note that measurement and characterization of dynamics of Indo-US research collaboration is relatively underexplored. Therefore, in this work, we investigate major patterns in Indo-US collaboration with respect to certain key aspects using suitable scientometric notions and indicators. The research publication data for the last three decades (1990-2020) is obtained from Web of Science and analysed for the purpose. Results indicate an increase in absolute number of Indo-US collaborated papers over time, with an impressive share of about 1/3rd of India's total internationally collaborated research output. However, the proportionate share of Indo-US collaborated papers in India's internationally collaborated papers has declined over the time, as Indian researchers find new collaborating partners. Nevertheless, the collaboration with US is found to be highly rewarding in terms of citations and boost measures. Important insights and recommendations that may be helpful for shaping up new perspective on Indo-US collaboration policy are presented in this work., Comment: Pre print

Science and scientific research activities, in addition to the involvement of the researchers, require resources like research infrastructure, materials and reagents, databases and computational tools, journal subscriptions and publication charges etc. In order to meet these requirements, researchers try to attract research funding from different funding sources, both intramural and extramural. Though some recent reports provide details of the amount of funding provided by different funding agencies in India, it is not known what quantum of research output resulted from such funding. This paper, therefore, attempts to quantify the research output produced with the funding provided by different funding agencies to Indian researchers. The major funding agencies that supported Indian research publications are identified and are further characterized in terms of being national or international, and public or private. The analytical results not only provide a quantitative estimate of funded research from India and the major funding agencies supporting the research, but also discusses the overall context of research funding in India, particularly in the context of upcoming operationalization of Anusandhan National Research Foundation (ANRF)., Comment: First Draft

The existence of an open access (OA) citation advantage, that is, whether OA increases citations, has been a topic of interest for many years. Although numerous previous studies have focused on whether OA increases citations, expectations for OA go beyond that. One such expectation is the promotion of knowledge transfer across various fields. This study aimed to clarify whether OA, especially gold OA, increases interdisciplinary citations in various natural science fields. Specifically, we measured the effect of OA on interdisciplinary and within-discipline citation counts by decomposing an existing metric of the OA citation advantage. The results revealed that OA increases both interdisciplinary and within-discipline citations in many fields and increases only interdisciplinary citations in chemistry, computer science, and clinical medicine. Among these fields, clinical medicine tends to obtain more interdisciplinary citations without being influenced by specific journals or papers. The findings indicate that OA fosters knowledge transfer to different fields, which extends our understanding of its effects.

Scientific progress depends on researchers' ability to synthesize the growing body of literature. Can large language models (LMs) assist scientists in this task? We introduce OpenScholar, a specialized retrieval-augmented LM that answers scientific queries by identifying relevant passages from 45 million open-access papers and synthesizing citation-backed responses. To evaluate OpenScholar, we develop ScholarQABench, the first large-scale multi-domain benchmark for literature search, comprising 2,967 expert-written queries and 208 long-form answers across computer science, physics, neuroscience, and biomedicine. On ScholarQABench, OpenScholar-8B outperforms GPT-4o by 5% and PaperQA2 by 7% in correctness, despite being a smaller, open model. While GPT4o hallucinates citations 78 to 90% of the time, OpenScholar achieves citation accuracy on par with human experts. OpenScholar's datastore, retriever, and self-feedback inference loop also improves off-the-shelf LMs: for instance, OpenScholar-GPT4o improves GPT-4o's correctness by 12%. In human evaluations, experts preferred OpenScholar-8B and OpenScholar-GPT4o responses over expert-written ones 51% and 70% of the time, respectively, compared to GPT4o's 32%. We open-source all of our code, models, datastore, data and a public demo.

Since generative artificial intelligence (AI) tools such as OpenAI's ChatGPT became widely available, researchers have used them in the writing process. The consensus of the academic publishing community is that such usage must be declared in the published article. Academ-AI documents examples of suspected undeclared AI usage in the academic literature, discernible primarily due to the appearance in research papers of idiosyncratic verbiage characteristic of large language model (LLM)-based chatbots. This analysis of the first 500 examples collected reveals that the problem is widespread, penetrating the journals and conference proceedings of highly respected publishers. Undeclared AI seems to appear in journals with higher citation metrics and higher article processing charges (APCs), precisely those outlets that should theoretically have the resources and expertise to avoid such oversights. An extremely small minority of cases are corrected post publication, and the corrections are often insufficient to rectify the problem. The 500 examples analyzed here likely represent a small fraction of the undeclared AI present in the academic literature, much of which may be undetectable. Publishers must enforce their policies against undeclared AI usage in cases that are detectable; this is the best defense currently available to the academic publishing community against the proliferation of undisclosed AI., Comment: 24 pages, 8 figures

A large host of scientific journals and conferences solicit peer reviews from multiple reviewers for the same submission, aiming to gather a broader range of perspectives and mitigate individual biases. In this work, we reflect on the role of diversity in the slate of reviewers assigned to evaluate a submitted paper as a factor in diversifying perspectives and improving the utility of the peer-review process. We propose two measures for assessing review utility: review coverage -- reviews should cover most contents of the paper -- and review redundancy -- reviews should add information not already present in other reviews. We hypothesize that reviews from diverse reviewers will exhibit high coverage and low redundancy. We conduct a causal study of different measures of reviewer diversity on review coverage and redundancy using observational data from a peer-reviewed conference with approximately 5,000 submitted papers. Our study reveals disparate effects of different diversity measures on review coverage and redundancy. Our study finds that assigning a group of reviewers that are topically diverse, have different seniority levels, or have distinct publication networks leads to broader coverage of the paper or review criteria, but we find no evidence of an increase in coverage for reviewer slates with reviewers from diverse organizations or geographical locations. Reviewers from different organizations, seniority levels, topics, or publications networks (all except geographical diversity) lead to a decrease in redundancy in reviews. Furthermore, publication network-based diversity alone also helps bring in varying perspectives (that is, low redundancy), even within specific review criteria. Our study adopts a group decision-making perspective for reviewer assignments in peer review and suggests dimensions of diversity that can help guide the reviewer assignment process.

Matthew effects, or the tendency for early achievements in science to lead to more recognition and opportunities, are a potential source of stratification and lost innovation when they draw unreasonable attention away from equally innovative but less celebrated scholars. Here, we analyze whether prizewinners produce more innovative works before and after being awarded a prize compared to equivalently impactful non-prizewinning contenders. Our data covers the careers of prizewinners and their dynamically matched non-prizewinners, a longitudinal, science-wide sample of 23,562 scholars and 5.7 million publications. We measured the innovativeness of prizewinners' and non-prizewinners' publications in terms of their novelty, convergent thinking, and interdisciplinarity. We find that prizewinners display distinctive forms of innovativeness relative to their non-prizewinning counterparts in terms of combining ideas in novel ways, bridging foundational and cutting-edge work on a topic, and formulating approaches to problems that leverage the strengths of interdisciplinarity. Further, prizewinners' innovativeness is strongly predicted by their type of network embeddedness. In contrast to matched non-prizewinners, prizewinners have shorter-term collaborations, their collaborators tend to focus their attention on topics that are new to the prizewinners, and their collaborators' collaborators have minimal overlap.

The chronological hierarchy and classification of psychological types of individuals are examined. The anomalous nature of psychological activity in individuals involved in scientific work is highlighted. Certain aspects of the introverted thinking type in scientific activities are analyzed. For the first time, psychological archetypes of scientists with pronounced introversion are postulated in the context of twelve hypotheses about the specifics of professional attributes of introverted scientific activities. A linear regression and Bayesian equation are proposed for quantitatively assessing the econometric degree of introversion in scientific employees, considering a wide range of characteristics inherent to introverts in scientific processing. Specifically, expressions for a comprehensive assessment of introversion in a linear model and the posterior probability of the econometric (scientometric) degree of introversion in a Bayesian model are formulated. The models are based on several econometric (scientometric) hypotheses regarding various aspects of professional activities of introverted scientists, such as a preference for solo publications, low social activity, narrow specialization, high research depth, and so forth. Empirical data and multiple linear regression methods can be used to calibrate the equations. The model can be applied to gain a deeper understanding of the psychological characteristics of scientific employees, which is particularly useful in ergonomics and the management of scientific teams and projects. The proposed method also provides scientists with pronounced introversion the opportunity to develop their careers, focusing on individual preferences and features., Comment: 30 pages, 2 table

The discovery of the Dead Sea Scrolls over 60 years ago is widely regarded as one of the greatest archaeological breakthroughs in modern history. Recent study of the scrolls presents ongoing computational challenges, including determining the provenance of fragments, clustering fragments based on their degree of similarity, and pairing fragments that originate from the same manuscript -- all tasks that require focusing on individual letter and fragment shapes. This paper presents a computational method for segmenting ink and parchment regions in multispectral images of Dead Sea Scroll fragments. Using the newly developed Qumran Segmentation Dataset (QSD) consisting of 20 fragments, we apply multispectral thresholding to isolate ink and parchment regions based on their unique spectral signatures. To refine segmentation accuracy, we introduce an energy minimization technique that leverages ink contours, which are more distinguishable from the background and less noisy than inner ink regions. Experimental results demonstrate that this Multispectral Thresholding and Energy Minimization (MTEM) method achieves significant improvements over traditional binarization approaches like Otsu and Sauvola in parchment segmentation and is successful at delineating ink borders, in distinction from holes and background regions., Comment: 17 pages, ICDAR-IJDAR-2025

We report here on the workflow that we needed to develop in order to integrate the growing range of openly licensed, born-digital and, increasingly, machine actionable publications. Our developmental work focused upon textual data for Ancient Greek, Latin, Old English, Classical Arabic and Classical Persian but the challenges that we have had to address are relevant to sources in a wide range of languages, ancient and modern. Perseus 6 was designed to be a publishing workflow that organizes complementary data into an integrated reading environment. This document focuses on the ways in which we have organized the data and describes the current state of ATLAS (Aligned Text and Linguistic Annotation Server) architecture. While this is the sixth version of the Perseus Digital Library, Perseus 6 represents a major step beyond its predecessors. Where Perseus 5 (described below) can represent and integrate digital versions of print editions (e.g., critical editions with interactive textual notes, links to lexicon and commentary entries), Perseus 6 was designed to bring together an expandable range of born-digital classes of annotation. An online ATLAS server with some initial functionality is now online and public services will expand during the rest of 2024. Most of the ATLAS data is, however, now available on Github and that data will be the focus on this paper in its current version.

Tenure is a cornerstone of the US academic system, yet its relationship to faculty research trajectories remains poorly understood. Conceptually, tenure systems may act as a selection mechanism, screening in high-output researchers; a dynamic incentive mechanism, encouraging high output prior to tenure but low output after tenure; and a creative search mechanism, encouraging tenured individuals to undertake high-risk work. Here, we integrate data from seven different sources to trace US tenure-line faculty and their research outputs at an unprecedented scale and scope, covering over 12,000 researchers across 15 disciplines. Our analysis reveals that faculty publication rates typically increase sharply during the tenure track and peak just before obtaining tenure. Post-tenure trends, however, vary across disciplines: in lab-based fields, such as biology and chemistry, research output typically remains high post-tenure, whereas in non-lab-based fields, such as mathematics and sociology, research output typically declines substantially post-tenure. Turning to creative search, faculty increasingly produce novel, high-risk research after securing tenure. However, this shift toward novelty and risk-taking comes with a decline in impact, with post-tenure research yielding fewer highly cited papers. Comparing outcomes across common career ages but different tenure years or comparing research trajectories in tenure-based and non-tenure-based research settings underscores that breaks in the research trajectories are sharply tied to the individual's tenure year. Overall, these findings provide a new empirical basis for understanding the tenure system, individual research trajectories, and the shape of scientific output.

Electronic health records (EHRs) contain important longitudinal information on individuals who have received medical care. Traditionally, EHRs have been used to support a wide range of administrative activities such as billing and clinical workflow, but, given the depth and breadth of clinical and demographic data they contain, they are increasingly being used to provide real-world data for research. Although EHR data have enormous research potential, the full realization of that potential requires a data management strategy that extracts from large EHR databases, that are collected from a range of care settings and time periods, well-documented research-relevant data that can be used by different researchers. Having a common well-documented data management strategy for EHR will support reproducible research and sharing documentation on research variables that are derived from EHR variables is important to open science. In this short paper, we describe the EHRs Data Harmonization Platform. The platform is based on an easy to use web app a publicly available at https://poxotn-arian-aminoleslami.shinyapps.io/Arian/ and as a standalone software package at https://github.com/ArianAminoleslami/EHRs-Data Harmonization-Platform, that is linked to an existing R library for data harmonization called recodeflow. The platform can be used to extract, document, and harmonize variables from EHR and it can also be used to document and share research variables that have been derived from those EHR data., Comment: 15 pages, 10 figures

This paper addresses how to assign a monetary value to scientific publications, particularly in the case of multi-author papers arising from large-scale research collaborations. Contemporary science increasingly relies on extensive and varied collaborations to tackle global challenges in fields such as life sciences, climate science, energy, high-energy physics, astronomy, and many others. We argue that existing literature fails to address the collaborative nature of research by overlooking the relationship between coauthorship and scientists productivity. Using the Marginal Cost of Production (MCP) approach, we first highlight the methodological limitations of ignoring this relationship, then propose a generalised MCP model to value co-authorship. As a case study, we examine High-Energy Physics (HEP) collaborations at the Large Hadron Collider (LHC) at CERN, analysing approximately half a million scientific outputs by over 50,000 authors from 1990 to 2021. Our findings indicate that collaborative adjustments yield monetary valuations for subsets of highly collaborative papers up to 3 orders of magnitude higher than previous estimates, with elevated values correlating with high research quality. This study contributes to the literature on research output evaluation, addressing debates in science policy around assessing research performance and impact. Our methodology is applicable to authorship valuation both within academia and in large-scale scientific collaborations, fitting diverse research impact assessment frameworks or as self-standing procedure. Additionally, we discuss the conditions under which this method may complement survey-based approaches., Comment: 38 pages, excluding annexes, 15 figures. Working paper to be submitted to "Scientometrics"

Purpose: Journal Impact Factors and other citation-based indicators are widely used and abused to help select journals to publish in or to estimate the value of a published article. Nevertheless, citation rates primarily reflect scholarly impact rather than other quality dimensions, including societal impact, originality, and rigour. In contrast, Journal Quality Factors (JQFs) are average quality score estimates given to a journal's articles by ChatGPT. Design: JQFs were compared with Polish, Norwegian and Finnish journal ranks and with journal citation rates for 1,300 journals with 130,000 articles from 2021 in large monodisciplinary journals in the 25 out of 27 Scopus broad fields of research for which it was possible. Outliers were also examined. Findings: JQFs correlated positively and mostly strongly (median correlation: 0.641) with journal ranks in 24 out of the 25 broad fields examined, indicating a nearly science-wide ability for ChatGPT to estimate journal quality. Journal citation rates had similarly high correlations with national journal ranks, however, so JQFs are not a universally better indicator. An examination of journals with JQFs not matching their journal ranks suggested that abstract styles may affect the result, such as whether the societal contexts of research are mentioned. Limitations: Different journal rankings may have given different findings because there is no agreed meaning for journal quality. Implications: The results suggest that JQFs are plausible as journal quality indicators in all fields and may be useful for the (few) research and evaluation contexts where journal quality is an acceptable proxy for article quality, and especially for fields like mathematics for which citations are not strong indicators of quality. Originality: This is the first attempt to estimate academic journal value with a Large Language Model.

This study aimed to identify and analyze the characteristics of highly cited publications in the field of artificial intelligence within the Science Citation Index Expanded from 1991 to 2022. The assessment focused on documents that garnered 100 citations or more from the Web of Science Core Collection, spanning from their publication date to the end of 2022. Various aspects of these documents were analyzed, encompassing document types, the distribution of annual production, the average number of citations per publication, Web of Science categories, and journals. Moreover, the publication performance of countries, institutions, and authors underwent evaluation through six publication indicators and associated citation metrics. To facilitate a comprehensive comparison of the authors research performance, the Y-index was employed. The outcomes of the analysis revealed that a majority of the highly cited articles were published within the Web of Science categories of "artificial intelligence computer science" and "electrical and electronic engineering". Notably, the United States exhibited dominance across all six publication indicators. Within the realm of average citations per publication, the United Kingdom emerged as a leader for independent articles, first-author articles, and corresponding-author articles. Exceptionally, the Chinese Academy of Sciences in China and the Massachusetts Institute of Technology (MIT) in the USA, contributed significantly. The significant impact of highly cited articles extended to the output of Stanford University in the USA. B.L. Bassler published the most highly cited articles. Upon employing the Y-index analysis, J.E.P. Santos was identified as having the highest potential for publication. In addition to the primary analysis, this study also presented nine classic articles that have left an indelible mark on artificial intelligence research.

Some research now suggests that ChatGPT can estimate the quality of journal articles from their titles and abstracts. This has created the possibility to use ChatGPT quality scores, perhaps alongside citation-based formulae, to support peer review for research evaluation. Nevertheless, ChatGPT's internal processes are effectively opaque, despite it writing a report to support its scores, and its biases are unknown. This article investigates whether publication date and field are biasing factors. Based on submitting a monodisciplinary journal-balanced set of 117,650 articles from 26 fields published in the years 2003, 2008, 2013, 2018 and 2023 to ChatGPT 4o-mini, the results show that average scores increased over time, and this was not due to author nationality or title and abstract length changes. The results also varied substantially between fields, and first author countries. In addition, articles with longer abstracts tended to receive higher scores, but plausibly due to such articles tending to be better rather than due to ChatGPT analysing more text. Thus, for the most accurate research quality evaluation results from ChatGPT, it is important to normalise ChatGPT scores for field and year and check for anomalies caused by sets of articles with short abstracts.

While previous studies have demonstrated that Large Language Models (LLMs) can predict peer review outcomes to some extent, this paper builds on that by introducing two new contexts and employing a more robust method - averaging multiple ChatGPT scores. The findings that averaging 30 ChatGPT predictions, based on reviewer guidelines and using only the submitted titles and abstracts, failed to predict peer review outcomes for F1000Research (Spearman's rho=0.00). However, it produced mostly weak positive correlations with the quality dimensions of SciPost Physics (rho=0.25 for validity, rho=0.25 for originality, rho=0.20 for significance, and rho = 0.08 for clarity) and a moderate positive correlation for papers from the International Conference on Learning Representations (ICLR) (rho=0.38). Including the full text of articles significantly increased the correlation for ICLR (rho=0.46) and slightly improved it for F1000Research (rho=0.09), while it had variable effects on the four quality dimension correlations for SciPost LaTeX files. The use of chain-of-thought system prompts slightly increased the correlation for F1000Research (rho=0.10), marginally reduced it for ICLR (rho=0.37), and further decreased it for SciPost Physics (rho=0.16 for validity, rho=0.18 for originality, rho=0.18 for significance, and rho=0.05 for clarity). Overall, the results suggest that in some contexts, ChatGPT can produce weak pre-publication quality assessments. However, the effectiveness of these assessments and the optimal strategies for employing them vary considerably across different platforms, journals, and conferences. Additionally, the most suitable inputs for ChatGPT appear to differ depending on the platform.

Several initiatives have been undertaken to conceptually model the domain of scholarly data using ontologies and to create respective Knowledge Graphs. Yet, the full potential seems unleashed, as automated means for automatic population of said ontologies are lacking, and respective initiatives from the Semantic Web community are not necessarily connected: we propose to make scholarly data more sustainably accessible by leveraging Wikidata's infrastructure and automating its population in a sustainable manner through LLMs by tapping into unstructured sources like conference Web sites and proceedings texts as well as already existing structured conference datasets. While an initial analysis shows that Semantic Web conferences are only minimally represented in Wikidata, we argue that our methodology can help to populate, evolve and maintain scholarly data as a community within Wikidata. Our main contributions include (a) an analysis of ontologies for representing scholarly data to identify gaps and relevant entities/properties in Wikidata, (b) semi-automated extraction -- requiring (minimal) manual validation -- of conference metadata (e.g., acceptance rates, organizer roles, programme committee members, best paper awards, keynotes, and sponsors) from websites and proceedings texts using LLMs. Finally, we discuss (c) extensions to visualization tools in the Wikidata context for data exploration of the generated scholarly data. Our study focuses on data from 105 Semantic Web-related conferences and extends/adds more than 6000 entities in Wikidata. It is important to note that the method can be more generally applicable beyond Semantic Web-related conferences for enhancing Wikidata's utility as a comprehensive scholarly resource. Source Repository: https://github.com/scholarly-wikidata/ DOI: https://doi.org/10.5281/zenodo.10989709 License: Creative Commons CC0 (Data), MIT (Code), Comment: 17 pages, accepted at EKAW-24

Peer review, as a widely used practice to ensure the quality and integrity of publications, lacks a well-defined and common mechanism to self-incentivize virtuous behavior across all the conferences and journals. This is because information about reviewer efforts and author feedback typically remains local to a single venue, while the same group of authors and reviewers participate in the publication process across many venues. Previous attempts to incentivize the reviewing process assume that the quality of reviews and papers authored correlate for the same person, or they assume that the reviewers can receive physical rewards for their work. In this paper, we aim to keep track of reviewing and authoring efforts by users (who review and author) across different venues while ensuring self-incentivization. We show that our system, DecentPeeR, incentivizes reviewers to behave according to the rules, i.e., it has a unique Nash equilibrium in which virtuous behavior is rewarded., Comment: This paper was part of 2024 IEEE International Conference on Blockchain and Cryptocurrency (ICBC)

Traditionally, recommendation algorithms have been designed for service developers. But recently, a new paradigm called user-side recommender systems has been proposed. User-side recommender systems are built and used by end users, in sharp contrast to traditional provider-side recommender systems. Even if the official recommender system offered by the provider is not fair, end users can create and enjoy their own user-side recommender systems by themselves. Although the concept of user-side recommender systems is attractive, the problem is they require tremendous communication costs between the user and the official system. Even the most efficient user-side recommender systems require about 5 times more costs than provider-side recommender systems. Such high costs hinder the adoption of user-side recommender systems. In this paper, we propose overhead-free user-side recommender systems, RecCycle, which realizes user-side recommender systems without any communication overhead. The main idea of RecCycle is to recycle past recommendation results offered by the provider's recommender systems. The ingredients of RecCycle can be retrieved ``for free,'' and it greatly reduces the cost of user-side recommendations. In the experiments, we confirm that RecCycle performs as well as state-of-the-art user-side recommendation algorithms while RecCycle reduces costs significantly., Comment: arXiv admin note: text overlap with arXiv:2208.09864, arXiv:2403.15757

This study seeks to identify the potential role of AI-driven assistive technologies in enhancing access to libraries for persons with varying degrees of challenges. Traditional libraries pose a problem to many users with vision and mobility, among other conditions related to physical and infirmities. This mixed-methods research approach will examine ways in which AI-powered assistive tools and applications associated with text-to-speech, navigation systems, and personalized assistants are revolutionizing library services through a literature review, survey methods, interviews, and case studies. Our findings suggest that these technologies greatly increase the autonomy and participation of people with physical disabilities, providing personalized support and access to a wide range of resources. From this, some key findings have been deduced from the research, showing a strong impact on user experience and efficiency in services, while at the same time bringing out important considerations related to privacy and ethical implementation. This study highlights the central role of AI in making library settings more inclusive, thereby allowing equal access to knowledge and participation in the community. Such insight thus serves professionals working in libraries, policymakers, and technology developers for innovations to occur uninterruptedly, with future research directions proposed that would refine such technologies, especially toward the special needs of diverse populations. By adopting AI, libraries could uphold their mission of providing equal access to knowledge through full and equal participation of all persons, regardless of any type of physical ability, in the learning and community activities carried out by the library. This study paves the way for future innovations in creating more accessible and inclusive library spaces.

The current system of scholarly publishing is often criticized for being slow, expensive, and not transparent. The rise of open access publishing as part of open science tenets, promoting transparency and collaboration, together with calls for research assesment reforms are the results of these criticisms. The emergence of new open access publishers presents a unique opportunity to empirically test how universities and countries respond to shifts in the academic publishing landscape. These new actors challenge traditional publishing models, offering faster review times and broader accessibility, which could influence strategic publishing decisions. Our findings reveal a clear division in European publishing practices, with countries clustering into two groups distinguished by the ratio of publications in new open access journals with accelerated review times versus legacy journals. This divide underscores a broader shift in academic culture, highlighting new open access publishing venues as a strategic factor influencing national and institutional publishing practices, with significant implications for research accessibility and collaboration across Europe.

Scholarly publishing involves multiple stakeholders having various types of interest. In Canada, the implication of universities, the presence of societies and the availability of governmental support for periodicals seem to have contributed to a rather diverse ecosystem of journals. This study presents in detail the current state of these journals, in addition to past trends and transformations during the 20th century and, in particular, the digital era. To this effect, we created a new dataset, including a total of 1256 journals, 944 of which appeared to be active today, specifically focusing on the supporting organizations behind the journals, the types of (open) access, disciplines, geographic origins, languages of publication and hosting platforms and tools. The main overarching traits across Canadian scholarly journals are an important presence of Diamond open access, which has been adopted by 62% of the journals, a predominance of the Social Sciences and Humanities disciplines and a scarce presence of the major commercial publishers. The digital era allowed for the development of open infrastructures, which contributed to the creation of a new generation of journals that massively adopted Diamond open access, often supported by university libraries. However, journal cessation also increased, especially among the recently founded journals. These results provide valuable insights for the design of tailored practices and policies that cater to the needs of different types of periodicals and that take into account the evolving practices across the Canadian scholarly journal landscape., Comment: 23 pages; 7 figures and 3 tables

For many public research organizations, funding creation of science and maximizing scientific output is of central interest. Typically, when evaluating scientific production for funding, citations are utilized as a proxy, although these are severely influenced by factors beyond scientific impact. This study aims to mitigate the consequences of the Matthew effect in citations, where prominent authors and prestigious journals receive more citations regardless of the scientific content of the publications. To this end, the study presents an approach to predicting citations of papers based solely on observable characteristics available at the submission stage of a double-blind peer-review process. Combining classical linear models, generalized linear models and utilizing large-scale data sets on biomedical papers based on the PubMed database, the results demonstrate that it is possible to make fairly accurate predictions of citations using only observable characteristics of papers excluding information on authors and journals, thereby mitigating the Matthew effect. Thus, the outcomes have important implications for the field of scientometrics, providing a more objective method for citation prediction by relying on pre-publication variables that are immune to manipulation by authors and journals, thereby enhancing the objectivity of the evaluation process. Our approach is thus important for government agencies responsible for funding the creation of high-quality scientific content rather than perpetuating prestige.

Current metadata creation for web archives is time consuming and costly due to reliance on human effort. This paper explores the use of gpt-4o for metadata generation within the Web Archive Singapore, focusing on scalability, efficiency, and cost effectiveness. We processed 112 Web ARChive (WARC) files using data reduction techniques, achieving a notable 99.9% reduction in metadata generation costs. By prompt engineering, we generated titles and abstracts, which were evaluated both intrinsically using Levenshtein Distance and BERTScore, and extrinsically with human cataloguers using McNemar's test. Results indicate that while our method offers significant cost savings and efficiency gains, human curated metadata maintains an edge in quality. The study identifies key challenges including content inaccuracies, hallucinations, and translation issues, suggesting that Large Language Models (LLMs) should serve as complements rather than replacements for human cataloguers. Future work will focus on refining prompts, improving content filtering, and addressing privacy concerns through experimentation with smaller models. This research advances the integration of LLMs in web archiving, offering valuable insights into their current capabilities and outlining directions for future enhancements. The code is available at https://github.com/masamune-prog/warc2summary for further development and use by institutions facing similar challenges.

The Generalist Repository Ecosystem Initiative (GREI), funded by the NIH, developed an AI taxonomy tailored to data repository roles to guide AI integration across repository management. It categorizes the roles into stages, including acquisition, validation, organization, enhancement, analysis, sharing, and user support, providing a structured framework for implementing AI in repository workflows.

In democratic societies, electoral systems play a crucial role in translating public preferences into political representation. Among these, the D'Hondt method is widely used to ensure proportional representation, balancing fair representation with governmental stability. Recently, there has been a growing interest in applying similar principles of proportional representation to enhance interpretability in machine learning, specifically in Explainable AI (XAI). This study investigates the integration of D'Hondt-based voting principles in the DhondtXAI method, which leverages resource allocation concepts to interpret feature importance within AI models. Through a comparison of SHAP (Shapley Additive Explanations) and DhondtXAI, we evaluate their effectiveness in feature attribution within CatBoost and XGBoost models for breast cancer and diabetes prediction, respectively. The DhondtXAI approach allows for alliance formation and thresholding to enhance interpretability, representing feature importance as seats in a parliamentary view. Statistical correlation analyses between SHAP values and DhondtXAI allocations support the consistency of interpretations, demonstrating DhondtXAI's potential as a complementary tool for understanding feature importance in AI models. The results highlight that integrating electoral principles, such as proportional representation and alliances, into AI explainability can improve user understanding, especially in high-stakes fields like healthcare.

Existing text simplification or paraphrase datasets mainly focus on sentence-level text generation in a general domain. These datasets are typically developed without using domain knowledge. In this paper, we release a novel dataset, VTechAGP, which is the first academic-to-general-audience text paraphrase dataset consisting of 4,938 document-level these and dissertation academic and general-audience abstract pairs from 8 colleges authored over 25 years. We also propose a novel dynamic soft prompt generative language model, DSPT5. For training, we leverage a contrastive-generative loss function to learn the keyword vectors in the dynamic prompt. For inference, we adopt a crowd-sampling decoding strategy at both semantic and structural levels to further select the best output candidate. We evaluate DSPT5 and various state-of-the-art large language models (LLMs) from multiple perspectives. Results demonstrate that the SOTA LLMs does not provide satisfactory outcomes, while the lightweight DSPT5 can achieve competitive results. To the best of our knowledge, we are the first to build a benchmark dataset and solutions for academic-to-general-audience text paraphrase dataset., Comment: 21 pages, 3 figures

Digital libraries that maintain extensive textual collections may want to further enrich their content for certain downstream applications, e.g., building knowledge graphs, semantic enrichment of documents, or implementing novel access paths. All of these applications require some text processing, either to identify relevant entities, extract semantic relationships between them, or to classify documents into some categories. However, implementing reliable, supervised workflows can become quite challenging for a digital library because suitable training data must be crafted, and reliable models must be trained. While many works focus on achieving the highest accuracy on some benchmarks, we tackle the problem from a digital library practitioner. In other words, we also consider trade-offs between accuracy and application costs, dive into training data generation through distant supervision and large language models such as ChatGPT, LLama, and Olmo, and discuss how to design final pipelines. Therefore, we focus on relation extraction and text classification, using the showcase of eight biomedical benchmarks., Comment: JCD2024 Full Paper, 12 pages, 6 figures

This paper contributes a new idea for exploring research funding effects on scholar performance. By collecting details of 9,501 research grants received by principal investigators from universities in the U.S. social sciences from 2000 to 2019 and data on their publications and citations in the Microsoft Academic Graph and Web of Science bibliographic collections, we build a novel dataset of grants and article counts, citations, and journal CiteScore. Based on this dataset, we first introduce three instrumental variables (IVs) suitable for isolating endogeneity issues in the study of competing grant effects, namely scholars' political hegemony in academia, imitation isomorphic behavior among scholars, and project familiarity. Then, this study explains the research funding effects by combining the three IVs with a two-stage least square (2SLS) model and by considering gaps from individual academics, universities, and research fields. Also, we provide validity and robustness tests of these three IVs and research funding effects. We find that our IVs serve the function of exogenizing and isolating endogeneity in capturing the research funding effect. Furthermore, empirical findings show that receiving research funding increases a scholar's research output and impact, but does not result in the scholar's research being published in more prestigious academic journals., Comment: This manuscript has successfully completed major revisions and is presently under review at Quantitative Science Studies (QSS)

Hunt for predatory journals is based on binary division of the publishing world to legitime and predatory journals. This leads to difficulties in labeling publishers like MDPI with questionable practices. However, the root cause is not the publisher itself but the problems in the peer review system. These problems have created markets for the for-profit publishers shaking the understanding of conservative researchers and their belief about what is good science. In this paper, we analyze the problems via examples based on our own experience as author, reviewer, guest editor, associate editor, and editor-in-chief. We discuss how MDPI has addressed these problems.

Large language models (LLMs) represent a promising, but controversial, tool in aiding scientific peer review. This study evaluates the usefulness of LLMs in a conference setting as a tool for vetting paper submissions against submission standards. We conduct an experiment at the 2024 Neural Information Processing Systems (NeurIPS) conference, where 234 papers were voluntarily submitted to an "LLM-based Checklist Assistant." This assistant validates whether papers adhere to the author checklist used by NeurIPS, which includes questions to ensure compliance with research and manuscript preparation standards. Evaluation of the assistant by NeurIPS paper authors suggests that the LLM-based assistant was generally helpful in verifying checklist completion. In post-usage surveys, over 70% of authors found the assistant useful, and 70% indicate that they would revise their papers or checklist responses based on its feedback. While causal attribution to the assistant is not definitive, qualitative evidence suggests that the LLM contributed to improving some submissions. Survey responses and analysis of re-submissions indicate that authors made substantive revisions to their submissions in response to specific feedback from the LLM. The experiment also highlights common issues with LLMs: inaccuracy (20/52) and excessive strictness (14/52) were the most frequent issues flagged by authors. We also conduct experiments to understand potential gaming of the system, which reveal that the assistant could be manipulated to enhance scores through fabricated justifications, highlighting potential vulnerabilities of automated review tools.

This study investigated the integration of cutting-edge technologies and methodologies for creating dynamic, user-centered library environments. In creative strategies for engagement and innovation, library users must be empowered to undertake the new role of modernizing library services and enhancing user experiences. It also enhances the information management and user engagement. This can be attained from personalized approaches, such as recommendation systems to interactive platforms that will have effective experiences tailored to users of different natures. It investigates the consumer engagement practices of enthusiasm, sharing, and learning about their roles in cognitive, affective, and behavioural engagements. Combined, these new approaches will help promote learning, interaction, and growth, add value, and have a more positive impact on users. The challenge for libraries in this rapidly changing, technologically advancing, and digitally networked world, with a base of expectant users, is to remain relevant and engaging. This study discusses innovative strategies for empowering library users and enhancing their engagement through creative and technological approaches. This investigation was conducted to integrate cutting-edge technologies and methodologies into creating dynamic library settings that are user-centered and foster learning, interaction, and personal growth.

The current situation of open research data in Spanish university repositories is analyzed by means of twelve indicators that allow us to compare them with each other. The twelve self-developed indicators deal with research datasets and institutional policies linked to open access, as well as some of the key characteristics of the repositories. The methodology used consists of comparing the repositories of the different universities linked to REBIUN. The result has been that datasets in institutional repositories are scarce, and the situation is heterogeneous across the territory. This raises questions about future open access policies for research data in the country's main scientific institutions., Comment: 17 pages, 9 Figures

The concept of epistemic breadth of the work of a researcher refers to the scope of their knowledge claims, as reflected in published research reports. Studies of epistemic breadth have been hampered by the lack of a validated measure of the concept. Here we introduce a knowledge space approach to the measurement of epistemic breadth and propose to use the semantic similarity network of an author's publication record to operationalize a measure. In this approach, each paper has its own location in a common abstract vector space based on its content. Proximity in knowledge space corresponds to thematic similarity of publications. Candidate measures of epistemic breadth derived from aggregate similarity values of researchers' bodies of work are tested against external validation data of researchers known to have made a major change in research topic and against self-citation data. We find that some candidate measures co-vary well with known epistemic breadth of researchers in the empirical data and can serve as valid indicators of the concept.

Evaluating the quality of published research is time-consuming but important for departmental evaluations, appointments, and promotions. Previous research has shown that ChatGPT can score articles for research quality, with the results correlating positively with an indicator of quality in all fields except Clinical Medicine. This article investigates this anomaly with the largest dataset yet and a more detailed analysis. The results showed that ChatGPT 4o-mini scores for articles submitted to the UK's Research Excellence Framework (REF) 2021 Unit of Assessment (UoA) 1 Clinical Medicine correlated positively (r=0.134, n=9872) with departmental mean REF scores, against a theoretical maximum correlation of r=0.226 (due to the departmental averaging involved). At the departmental level, mean ChatGPT scores correlated more strongly with departmental mean REF scores (r=0.395, n=31). For the 100 journals with the most articles in UoA 1, their mean ChatGPT score correlated strongly with their REF score (r=0.495) but negatively with their citation rate (r=-0.148). Journal and departmental anomalies in these results point to ChatGPT being ineffective at assessing the quality of research in prestigious medical journals or research directly affecting human health, or both. Nevertheless, the results give evidence of ChatGPT's ability to assess research quality overall for Clinical Medicine, so now there is evidence of its ability in all academic fields.

Large language models (LLMs) excel in tasks requiring processing and interpretation of input text. Abstract screening is a labour-intensive component of systematic review involving repetitive application of inclusion and exclusion criteria on a large volume of studies identified by a literature search. Here, LLMs (GPT-3.5 Turbo, GPT-4 Turbo, GPT-4o, Llama 3 70B, Gemini 1.5 Pro, and Claude Sonnet 3.5) were trialled on systematic reviews in a full issue of the Cochrane Library to evaluate their accuracy in zero-shot binary classification for abstract screening. Trials over a subset of 800 records identified optimal prompting strategies and demonstrated superior performance of LLMs to human researchers in terms of sensitivity (LLM-max = 1.000, human-max = 0.775), precision (LLM-max = 0.927, human-max = 0.911), and balanced accuracy (LLM-max = 0.904, human-max = 0.865). The best performing LLM-prompt combinations were trialled across every replicated search result (n = 119,691), and exhibited consistent sensitivity (range 0.756-1.000) but diminished precision (range 0.004-0.096). 66 LLM-human and LLM-LLM ensembles exhibited perfect sensitivity with a maximal precision of 0.458, with less observed performance drop in larger trials. Significant variation in performance was observed between reviews, highlighting the importance of domain-specific validation before deployment. LLMs may reduce the human labour cost of systematic review with maintained or improved accuracy and sensitivity. Systematic review is the foundation of evidence synthesis across academic disciplines, including evidence-based medicine, and LLMs may increase the efficiency and quality of this mode of research., Comment: RS and AJT are joint-first authors

This study demonstrates that Large Language Models (LLMs) can transcribe historical handwritten documents with significantly higher accuracy than specialized Handwritten Text Recognition (HTR) software, while being faster and more cost-effective. We introduce an open-source software tool called Transcription Pearl that leverages these capabilities to automatically transcribe and correct batches of handwritten documents using commercially available multimodal LLMs from OpenAI, Anthropic, and Google. In tests on a diverse corpus of 18th/19th century English language handwritten documents, LLMs achieved Character Error Rates (CER) of 5.7 to 7% and Word Error Rates (WER) of 8.9 to 15.9%, improvements of 14% and 32% respectively over specialized state-of-the-art HTR software like Transkribus. Most significantly, when LLMs were then used to correct those transcriptions as well as texts generated by conventional HTR software, they achieved near-human levels of accuracy, that is CERs as low as 1.8% and WERs of 3.5%. The LLMs also completed these tasks 50 times faster and at approximately 1/50th the cost of proprietary HTR programs. These results demonstrate that when LLMs are incorporated into software tools like Transcription Pearl, they provide an accessible, fast, and highly accurate method for mass transcription of historical handwritten documents, significantly streamlining the digitization process., Comment: 29 Pages, 11 Tables, 2 Figures

Large scientific institutions, such as the Space Telescope Science Institute, track the usage of their facilities to understand the needs of the research community. Astrophysicists incorporate facility usage data into their scientific publications, embedding this information in plain-text. Traditional automatic search queries prove unreliable for accurate tracking due to the misidentification of facility names in plain-text. As automatic search queries fail, researchers are required to manually classify publications for facility usage, which consumes valuable research time. In this work, we introduce a machine learning classification framework for the automatic identification of facility usage of observation sections in astrophysics publications. Our framework identifies sentences containing telescope mission keywords (e.g., Kepler and TESS) in each publication. Subsequently, the identified sentences are transformed using Term Frequency-Inverse Document Frequency and classified with a Support Vector Machine. The classification framework leverages the context surrounding the identified telescope mission keywords to provide relevant information to the classifier. The framework successfully classifies usage of MAST hosted missions with a 92.9% accuracy. Furthermore, our framework demonstrates robustness when compared to other approaches, considering common metrics and computational complexity. The framework's interpretability makes it adaptable for use across observatories and other scientific facilities worldwide., Comment: 11 pages, 6 figures, 1 table

In machine learning research, it is common to evaluate algorithms via their performance on standard benchmark datasets. While a growing body of work establishes guidelines for -- and levies criticisms at -- data and benchmarking practices in machine learning, comparatively less attention has been paid to the data repositories where these datasets are stored, documented, and shared. In this paper, we analyze the landscape of these $\textit{benchmark data repositories}$ and the role they can play in improving benchmarking. This role includes addressing issues with both datasets themselves (e.g., representational harms, construct validity) and the manner in which evaluation is carried out using such datasets (e.g., overemphasis on a few datasets and metrics, lack of reproducibility). To this end, we identify and discuss a set of considerations surrounding the design and use of benchmark data repositories, with a focus on improving benchmarking practices in machine learning., Comment: Accepted to NeurIPS Datasets and Benchmarks 2024

This study presents a comparative analysis between two scientific document classification systems. The first system employs the Scopus journal-based assignment method, adapted to a fractional model, while the second system uses an item-by-item system based on reclassified references according to the origin of the citers. The study's results are divided into three different sections: the first involves comparisons at the Scopus area level, the second examines comparisons at the category level, and the third tests various bibliometric indicators to identify the variations between the two systems. Highlighting the characteristics of the paper level system, it offers a reduction in the number of categories to which each document is assigned, achieving higher values of single-category assignment compared to the All Science Journal Classification (ASJC). When reclassifying areas and categories, the paper level system tends to accentuate differences at the extreme values, increasing the size of the largest categories and reducing that of the smallest ones. Moreover, the paper-by-paper system provides more homogeneous distributions in normalised impacts and adjusts values related to excellence more uniformly.

The rise of large language models (LLMs) has led many researchers to consider their usage for scientific work. Some have found benefits using LLMs to augment or automate aspects of their research pipeline, while others have urged caution due to risks and ethical concerns. Yet little work has sought to quantify and characterize how researchers use LLMs and why. We present the first large-scale survey of 816 verified research article authors to understand how the research community leverages and perceives LLMs as research tools. We examine participants' self-reported LLM usage, finding that 81% of researchers have already incorporated LLMs into different aspects of their research workflow. We also find that traditionally disadvantaged groups in academia (non-White, junior, and non-native English speaking researchers) report higher LLM usage and perceived benefits, suggesting potential for improved research equity. However, women, non-binary, and senior researchers have greater ethical concerns, potentially hindering adoption., Comment: 30 pages, 5 figures