Search

This paper presents Papilusion, an AI-generated scientific text detector developed within the DAGPap24 shared task on detecting automatically generated scientific papers. We propose an ensemble-based approach and conduct ablation studies to analyze the effect of the detector configurations on the performance. Papilusion is ranked 6th on the leaderboard, and we improve our performance after the competition ended, achieving 99.46 (+9.63) of the F1-score on the official test set., Comment: to appear in "The 4th Workshop on Scholarly Document Processing @ ACL 2024" proceedings

This paper presents an overview of a survey of jet substructure observables used to study modifications of jets induced by interaction with a Quark Gluon Plasma. We further outline ideas that were presented and discussed at the \textit{New jet quenching tools to explore equilibrium and non-equilibrium dynamics in heavy-ion collisions} workshop, which was held in February 2024 at the ECT$^{*}$ in Trento, Italy. The goal of this white paper is to provide a brief report on the study of jet quenching observables earlier conducted and to present new ideas that could be relevant for future explorations., Comment: 12 pages, 2 figures, NA3:Jet-QGP group white paper

The Atacama Large Millimetre/submillimetre Array (ALMA) receivers and technical papers are cited fewer than once in every six publications. This citation shortage is impeding the development of future (sub)millimetre instruments. In an effort to facilitate the correct citations of ALMA receivers and technical papers, this memo provides a comprehensive list of papers for the scientific community. This list was produced in discussion with the scientific and instrumentalist community, based on a June 2024 survey at the European Southern Observatory workshop on the ALMA Wideband Sensitivity Upgrade, as well as with the ALMA technical staff. The authors now encourage the community to enhance their already-excellent ALMA science with the appropriate references to ensure future (sub)millimetre instrumentation can keep addressing the key questions about our Universe., Comment: 8 pages; 1 figure; 3 tables; to be updated as more papers become available. Suggestions are warmly welcomed. ALMA memos solely contains the opinion of the authors in discussion with the astronomer and instrument builder community, and does not reflect the official policy of the ALMA telescope

Due to the rapid growth of scientific publications, identifying all related reference articles in the literature has become increasingly challenging yet highly demanding. Existing methods primarily assess candidate publications from a static perspective, focusing on the content of articles and their structural information, such as citation relationships. There is a lack of research regarding how to account for the evolving impact among papers on their embeddings. Toward this goal, this paper introduces a temporal dimension to paper recommendation strategies. The core idea is to continuously update a paper's embedding when new citation relationships appear, enhancing its relevance for future recommendations. Whenever a citation relationship is added to the literature upon the publication of a paper, the embeddings of the two related papers are updated through a Temporal Graph Neural Network (TGN). A learnable memory update module based on a Recurrent Neural Network (RNN) is utilized to study the evolution of the embedding of a paper in order to predict its reference impact in a future timestamp. Such a TGN-based model learns a pattern of how people's views of the paper may evolve, aiming to guide paper recommendations more precisely. Extensive experiments on an open citation network dataset, including 313,278 articles from https://paperswithcode.com/about PaperWithCode, have demonstrated the effectiveness of the proposed approach., Comment: 10 pages, 4 figures, accepted by SDU@AAAI-2024. The AAAI Workshop on Scientific Document Understanding (2024)

We conducted an experiment during the review process of the 2023 International Conference on Machine Learning (ICML) that requested authors with multiple submissions to rank their own papers based on perceived quality. We received 1,342 rankings, each from a distinct author, pertaining to 2,592 submissions. In this paper, we present an empirical analysis of how author-provided rankings could be leveraged to improve peer review processes at machine learning conferences. We focus on the Isotonic Mechanism, which calibrates raw review scores using author-provided rankings. Our analysis demonstrates that the ranking-calibrated scores outperform raw scores in estimating the ground truth ``expected review scores'' in both squared and absolute error metrics. Moreover, we propose several cautious, low-risk approaches to using the Isotonic Mechanism and author-provided rankings in peer review processes, including assisting senior area chairs' oversight of area chairs' recommendations, supporting the selection of paper awards, and guiding the recruitment of emergency reviewers. We conclude the paper by addressing the study's limitations and proposing future research directions., Comment: See more details about the experiment at https://openrank.cc/

This paper presents CodeRefine, a novel framework for automatically transforming research paper methodologies into functional code using Large Language Models (LLMs). Our multi-step approach first extracts and summarizes key text chunks from papers, analyzes their code relevance, and creates a knowledge graph using a predefined ontology. Code is then generated from this structured representation and enhanced through a proposed retrospective retrieval-augmented generation approach. CodeRefine addresses the challenge of bridging theoretical research and practical implementation, offering a more accurate alternative to LLM zero-shot prompting. Evaluations on diverse scientific papers demonstrate CodeRefine's ability to improve code implementation from the paper, potentially accelerating the adoption of cutting-edge algorithms in real-world applications.

Machine learning is widely utilized across various industries. Identifying the appropriate machine learning models and datasets for specific tasks is crucial for the effective industrial application of machine learning. However, this requires expertise in both machine learning and the relevant domain, leading to a high learning cost. Therefore, research focused on extracting combinations of tasks, machine learning models, and datasets from academic papers is critically important, as it can facilitate the automatic recommendation of suitable methods. Conventional information extraction methods from academic papers have been limited to identifying machine learning models and other entities as named entities. To address this issue, this study proposes a methodology extracting tasks, machine learning methods, and dataset names from scientific papers and analyzing the relationships between these information by using LLM, embedding model, and network clustering. The proposed method's expression extraction performance, when using Llama3, achieves an F-score exceeding 0.8 across various categories, confirming its practical utility. Benchmarking results on financial domain papers have demonstrated the effectiveness of this method, providing insights into the use of the latest datasets, including those related to ESG (Environmental, Social, and Governance) data., Comment: 10 pages, 8 figures

We describe the design, construction and characterisation of paper based devices to perform spirometry, a standard test for lung function assessment. In this device the instantaneous flowrate of the incoming breath from a person gets transformed to a specific acoustic frequency. In this manner, the time course of the persons breath profile is mapped to a time varying acoustic signal. The captured acoustic signal can be converted to standard spirometry curves and relevant parameters can be extracted as we describe in this paper. We compared our device with commercially available devices and showed that these paper based devices provide similar performance. These devices have advantage of low cost and simplicity of operation compared to currently available commercial products.

Teaching Computer Science (CS) by having students write programs by hand on paper has key pedagogical advantages: It allows focused learning and requires careful thinking compared to the use of Integrated Development Environments (IDEs) with intelligent support tools or "just trying things out". The familiar environment of pens and paper also lessens the cognitive load of students with no prior experience with computers, for whom the mere basic usage of computers can be intimidating. Finally, this teaching approach opens learning opportunities to students with limited access to computers. However, a key obstacle is the current lack of teaching methods and support software for working with and running handwritten programs. Optical character recognition (OCR) of handwritten code is challenging: Minor OCR errors, perhaps due to varied handwriting styles, easily make code not run, and recognizing indentation is crucial for languages like Python but is difficult to do due to inconsistent horizontal spacing in handwriting. Our approach integrates two innovative methods. The first combines OCR with an indentation recognition module and a language model designed for post-OCR error correction without introducing hallucinations. This method, to our knowledge, surpasses all existing systems in handwritten code recognition. It reduces error from 30\% in the state of the art to 5\% with minimal hallucination of logical fixes to student programs. The second method leverages a multimodal language model to recognize handwritten programs in an end-to-end fashion. We hope this contribution can stimulate further pedagogical research and contribute to the goal of making CS education universally accessible. We release a dataset of handwritten programs and code to support future research at https://github.com/mdoumbouya/codeocr

I analyze the evolution of papers certified by the Graphics Replicability Stamp Initiative (GRSI) to be reproducible, with a specific focus on the subset of publications that address visualization-related topics. With this analysis I show that, while the number of papers is increasing overall and within the visualization field, we still have to improve quite a bit to escape the replication crisis. I base my analysis on the data published by the GRSI as well as publication data for the different venues in visualization and lists of journal papers that have been presented at visualization-focused conferences. I also analyze the differences between the involved journals as well as the percentage of reproducible papers in the different presentation venues. Furthermore, I look at the authors of the publications and, in particular, their affiliation countries to see where most reproducible papers come from. Finally, I discuss potential reasons for the low reproducibility numbers and suggest possible ways to overcome these obstacles. This paper is reproducible itself, with source code and data available from github.com/tobiasisenberg/Visualization-Reproducibility as well as a free paper copy and all supplemental materials at osf.io/mvnbj., Comment: 9 pages plus appendix; 12 figures plus 14 figures in the appendix

Counterfeit products cause severe harm to public safety and health by penetrating untrusted supply chains. Numerous anti-counterfeiting techniques have been proposed, among which the use of inherent, unclonable irregularities of paper surfaces has shown considerable potential as a high-performance economical solution. Prior works do not consider supply chains cohesively, either focusing on creating or improving unclonable identifiers or on securing digital records of products. This work aims to systematically unify these two separate but connected research areas by comprehensively analyzing the needs of supply chains. We construct a generalized paper-based authentication framework and identify important shortcomings and promising ideas in the existing literature. Next, we do a stage-wise security analysis of our consolidated framework by drawing inspiration from works in signal processing, cryptography, and biometric systems. Finally, we examine key representative scenarios that illustrate the range of practical and technical challenges in real-world supply chains, and we outline the best practices to guide future research.

The impact of research papers, typically measured in terms of citation counts, depends on several factors, including the reputation of the authors, journals, and institutions, in addition to the quality of the scientific work. In this paper, we present an approach that combines natural language processing and machine learning to predict the impact of papers in a specific journal. Our focus is on the text, which should correlate with impact and the topics covered in the research. We employed a dataset of over 40,000 articles from ACS Applied Materials and Interfaces spanning from 2012 to 2022. The data was processed using various text embedding techniques and classified with supervised machine learning algorithms. Papers were categorized into the top 20% most cited within the journal, using both yearly and cumulative citation counts as metrics. Our analysis reveals that the method employing generative pre-trained transformers (GPT) was the most efficient for embedding, while the random forest algorithm exhibited the best predictive power among the machine learning algorithms. An optimized accuracy of 80\% in predicting whether a paper was among the top 20% most cited was achieved for the cumulative citation count when abstracts were processed. This accuracy is noteworthy, considering that author, institution, and early citation pattern information were not taken into account. The accuracy increased only slightly when the full texts of the papers were processed. Also significant is the finding that a simpler embedding technique, term frequency-inverse document frequency (TFIDF), yielded performance close to that of GPT. Since TFIDF captures the topics of the paper we infer that, apart from considering author and institution biases, citation counts for the considered journal may be predicted by identifying topics and "reading" the abstract of a paper.

Most published Information Systems research are of the behavioral science research (BSR) category rather than the design science research (DSR) category. This is due in part to the BSR orientation of many IS doctoral programs, which often do not involve much technical courses. This includes IS doctoral programs that train Information and Communication Technologies for Development (ICT4D) researchers. Without such technical knowledge many doctoral and postdoctoral researchers will not feel confident in engaging in DSR research. Given the importance of designing artifacts that are appropriate for a given context, an important question is how can ICT4D and other IS researchers increase their IS technical content knowledge and intimacy with the DSR process. In this paper we present, a process for reviewing DSR papers that has as its objectives: enhancing technical content knowledge, increasing knowledge and understanding of approaches to designing and evaluating IS/IT artifacts, and facilitating the identification of new DSR opportunities. This process has been applied for more than a decade at a USA research university., Comment: 44 pages, 9 Figures

Identifying significant references within the complex interrelations of a citation knowledge graph is challenging, which encompasses connections through citations, authorship, keywords, and other relational attributes. The Paper Source Tracing (PST) task seeks to automate the identification of pivotal references for given scholarly articles utilizing advanced data mining techniques. In the KDD CUP OAG-Challenge PST track, we design a recommendation-based framework tailored for the PST task. This framework employs the Neural Collaborative Filtering (NCF) model to generate final predictions. To process the textual attributes of the papers and extract input features for the model, we utilize SciBERT, a pre-trained language model. According to the experimental results, our method achieved a score of 0.37814 on the Mean Average Precision (MAP) metric, outperforming baseline models and ranking 11th among all participating teams. The source code is publicly available at https://github.com/MyLove-XAB/KDDCupFinal., Comment: KDD CUP 2024 OAG-Challenges, Paper Source Tracing, Technical Report of Team AoboSama @ KDD CUP 2024. August 25--29, 2024. Barcelona, Spain

Seeking answers to questions within long scientific research articles is a crucial area of study that aids readers in quickly addressing their inquiries. However, existing question-answering (QA) datasets based on scientific papers are limited in scale and focus solely on textual content. To address this limitation, we introduce SPIQA (Scientific Paper Image Question Answering), the first large-scale QA dataset specifically designed to interpret complex figures and tables within the context of scientific research articles across various domains of computer science. Leveraging the breadth of expertise and ability of multimodal large language models (MLLMs) to understand figures, we employ automatic and manual curation to create the dataset. We craft an information-seeking task involving multiple images that cover a wide variety of plots, charts, tables, schematic diagrams, and result visualizations. SPIQA comprises 270K questions divided into training, validation, and three different evaluation splits. Through extensive experiments with 12 prominent foundational models, we evaluate the ability of current multimodal systems to comprehend the nuanced aspects of research articles. Additionally, we propose a Chain-of-Thought (CoT) evaluation strategy with in-context retrieval that allows fine-grained, step-by-step assessment and improves model performance. We further explore the upper bounds of performance enhancement with additional textual information, highlighting its promising potential for future research and the dataset's impact on revolutionizing how we interact with scientific literature., Comment: preprint

Occlusal contacts are the locations at which the occluding surfaces of the maxilla and the mandible posterior teeth meet. Occlusal contact detection is a vital tool for restoring the loss of masticatory function and is a mandatory assessment in the field of dentistry, with particular importance in prosthodontics and restorative dentistry. The most common method for occlusal contact detection is articulating paper. However, this method can indicate significant medically false positive and medically false negative contact areas, leaving the identification of true occlusal indications to clinicians. To address this, we propose a multiclass Vision Transformer and Fully Convolutional Network ensemble semantic segmentation model with a combination hierarchical loss function, which we name as Hierarchical Fully Convolutional Branch Transformer (H-FCBFormer). We also propose a method of generating medically true positive semantic segmentation masks derived from expert annotated articulating paper masks and gold standard masks. The proposed model outperforms other machine learning methods evaluated at detecting medically true positive contacts and performs better than dentists in terms of accurately identifying object-wise occlusal contact areas while taking significantly less time to identify them. Code is available at https://github.com/Banksylel/H-FCBFormer., Comment: 15 pages, 5 figures, 2 tables, 5 equations, peer reviewed and accepted to Medical Imaging Understanding and Analysis (MIUA 2024)

In recent years, the rapid increase in scientific papers has overwhelmed traditional review mechanisms, resulting in varying quality of publications. Although existing methods have explored the capabilities of Large Language Models (LLMs) for automated scientific reviewing, their generated contents are often generic or partial. To address the issues above, we introduce an automated paper reviewing framework SEA. It comprises of three modules: Standardization, Evaluation, and Analysis, which are represented by models SEA-S, SEA-E, and SEA-A, respectively. Initially, SEA-S distills data standardization capabilities of GPT-4 for integrating multiple reviews for a paper. Then, SEA-E utilizes standardized data for fine-tuning, enabling it to generate constructive reviews. Finally, SEA-A introduces a new evaluation metric called mismatch score to assess the consistency between paper contents and reviews. Moreover, we design a self-correction strategy to enhance the consistency. Extensive experimental results on datasets collected from eight venues show that SEA can generate valuable insights for authors to improve their papers.

The paper referred to in the title is Allard/Almgren 1981 [AA81]. Several months ago Francesco Maggi emailed me saying that the inequalities 5.3(4),(5) of \cite{AA81} were wrong. In fact, as he pointed out, their incorrectness is immediately apparent if one takes $Z=0$ there. Maggi and his coauthor wanted to use these inequalities in their paper \cite{MN}. They were able to obtain a version of these inequalities which suffice for the carrying out the work in \cite{MN}. I started writing this paper in order to provide a version of these inequalities as needed in [AA81]. In thinking about this material I began to realize there were other problems with the paper. As a result I ended up {\em completely rewriting 5.1-5.4 on pages 243-248 of \cite{AA81}}; this rewrite is the contents of this paper. In addition to many annoying misprints many of the needed definitions and proofs in 5.1-5.4 are incomplete or absent. For example, it is not said where $z$ in 5.1(2) comes from; this omission completely surprised me since I remember doing a lot of work to come up with $z$ when \cite{AA81} was being written. Also, much of the necessary material about the reach of a submanifold as in \cite{FE2} is not provided in \cite{AA81}. The table of contents can serve as an index. In particular one sees there where the constants $\epsilon_1$ through $\epsilon_6$ are introduced. This material is extremely technical. One way to navigate this paper would be to start looking at Proposition 9.5 and work backwards.

Testing in production-like test environments is an essential part of quality assurance processes in many industries. Provisioning of such test environments, for information-intensive services, involves setting up databases that are rich-enough to enable simulating a wide variety of user scenarios. While production data is perhaps the gold-standard here, many organizations, particularly within the public sectors, are not allowed to use production data for testing purposes due to privacy concerns. The alternatives are to use anonymized data, or synthetically generated data. In this paper, we elaborate on these alternatives and compare them in an industrial context. Further we focus on synthetic data generation and investigate the use of recurrent neural networks for this purpose. In our preliminary experiments, we were able to generate representative and highly accurate data using a recurrent neural network. These results open new research questions that we discuss here, and plan to investigate in our future research., Comment: This paper was published in the proceedings of RAISE@ICSE in 2019

Human-computer interaction scholars are increasingly touching on topics related to politics or democracy. As these concepts are ambiguous, an examination of concepts' invoked meanings aids in the self-reflection of our research efforts. We conduct a thematic analysis of all papers with the word `politics' in abstract, title or keywords ($n$=378) and likewise 152 papers with the word `democracy.' We observe that these words are increasingly being used in human-computer interaction, both in absolute and relative terms. At the same time, we show that researchers invoke these words with diverse levels of analysis in mind: the early research focused on mezzo-level (i.e., small groups), but more recently the work has begun to include macro-level analysis (i.e., society and politics as played in the public sphere). After the increasing focus on the macro-level, we see a transition towards more normative and activist research, in some areas it replaces observational and empirical research. These differences indicate semantic differences, which -- in the worst case -- may limit scientific progress. We bring these differences visible to help in further exchanges of ideas and human-computer interaction community to explore how it orients itself to politics and democracy.