Search

Companies support their customers using live chats and chatbots to gain their loyalty. AFAS is a Dutch company aiming to leverage the opportunity large language models (LLMs) offer to answer customer queries with minimal to no input from its customer support team. Adding to its complexity, it is unclear what makes a response correct, and that too in Dutch. Further, with minimal data available for training, the challenge is to identify whether an answer generated by a large language model is correct and do it on the fly. This study is the first to define the correctness of a response based on how the support team at AFAS makes decisions. It leverages literature on natural language generation and automated answer grading systems to automate the decision-making of the customer support team. We investigated questions requiring a binary response (e.g., Would it be possible to adjust tax rates manually?) or instructions (e.g., How would I adjust tax rate manually?) to test how close our automated approach reaches support rating. Our approach can identify wrong messages in 55\% of the cases. This work shows the viability of automatically assessing when our chatbot tell lies., Comment: 10 pages + 2 pages references, 4 figures

Rapid detection of gas concentration is important in different domains like gas leakage monitoring, pollution control, and so on, for the prevention of health hazards. Out of different types of gas sensors, Metal oxide (MOx) sensors are extensively used in such applications because of their portability, low cost, and high sensitivity for specific gases. However, how to effectively sample the MOx data for the real-time detection of gas and its concentration level remains an open question. Here we introduce a simple analog front-end for one MOx sensor that encodes the gas concentration in the time difference between pulses of two separate pathways. This front-end design is inspired by the spiking output of a mammalian olfactory bulb. We show that for a gas pulse injected in a constant airflow, the time difference between pulses decreases with increasing gas concentration, similar to the spike time difference between the two principal output neurons in the olfactory bulb. The circuit design is further extended to a MOx sensor array and this sensor array front-end was tested in the same environment for gas identification and concentration estimation. Encoding of gas stimulus features in analog spikes at the sensor level itself may result in data and power-efficient real-time gas sensing systems in the future that can ultimately be used in uncontrolled and turbulent environments for longer periods without data explosion.

As machine learning models scale in size and complexity, their computational requirements become a significant barrier. Mixture-of-Experts (MoE) models alleviate this issue by selectively activating relevant experts. Despite this, MoE models are hindered by high communication overhead from all-to-all operations, low GPU utilization due to the synchronous communication constraint, and complications from heterogeneous GPU environments. This paper presents Aurora, which optimizes both model deployment and all-to-all communication scheduling to address these challenges in MoE inference. Aurora achieves minimal communication times by strategically ordering token transmissions in all-to-all communications. It improves GPU utilization by colocating experts from different models on the same device, avoiding the limitations of synchronous all-to-all communication. We analyze Aurora's optimization strategies theoretically across four common GPU cluster settings: exclusive vs. colocated models on GPUs, and homogeneous vs. heterogeneous GPUs. Aurora provides optimal solutions for three cases, and for the remaining NP-hard scenario, it offers a polynomial-time sub-optimal solution with only a 1.07x degradation from the optimal. Aurora is the first approach to minimize MoE inference time via optimal model deployment and communication scheduling across various scenarios. Evaluations demonstrate that Aurora significantly accelerates inference, achieving speedups of up to 2.38x in homogeneous clusters and 3.54x in heterogeneous environments. Moreover, Aurora enhances GPU utilization by up to 1.5x compared to existing methods.

AI systems crucially rely on human ratings, but these ratings are often aggregated, obscuring the inherent diversity of perspectives in real-world phenomenon. This is particularly concerning when evaluating the safety of generative AI, where perceptions and associated harms can vary significantly across socio-cultural contexts. While recent research has studied the impact of demographic differences on annotating text, there is limited understanding of how these subjective variations affect multimodal safety in generative AI. To address this, we conduct a large-scale study employing highly-parallel safety ratings of about 1000 text-to-image (T2I) generations from a demographically diverse rater pool of 630 raters balanced across 30 intersectional groups across age, gender, and ethnicity. Our study shows that (1) there are significant differences across demographic groups (including intersectional groups) on how severe they assess the harm to be, and that these differences vary across different types of safety violations, (2) the diverse rater pool captures annotation patterns that are substantially different from expert raters trained on specific set of safety policies, and (3) the differences we observe in T2I safety are distinct from previously documented group level differences in text-based safety tasks. To further understand these varying perspectives, we conduct a qualitative analysis of the open-ended explanations provided by raters. This analysis reveals core differences into the reasons why different groups perceive harms in T2I generations. Our findings underscore the critical need for incorporating diverse perspectives into safety evaluation of generative AI ensuring these systems are truly inclusive and reflect the values of all users., Comment: 20 pages, 7 figures

Desert locusts are notorious agriculture pests prompting billions in losses and global food scarcity concerns. With billions of these locusts invading agrarian lands, this is no longer a thing of the past. This study taps into the existing doppler weather radar (DWR) infrastructure which was originally deployed for meteorological applications. This study demonstrates a systematic approach to distinctly identify and track concentrations of desert locust swarms in near real time using single polarization radars. Findings reveal the potential to establish early warning systems with lead times of around 7 hours and spatial coverage of approximately 100 kilometers. Embracing these technological advancements are crucial to safeguard agricultural landscapes and upload global food security., Comment: 18 pages, 5 figures

The pull-based development model facilitates global collaboration within open-source software projects. Most research on the pull request decision-making process explored factors within projects, not the broader software ecosystem they comprise. We uncover ecosystem-wide factors that influence pull request acceptance decisions. We collected a dataset of approximately 1.8 million pull requests and 2.1 million issues from 20,052 GitHub projects within the NPM ecosystem. Of these, 98% depend on another project in the dataset, enabling studying collaboration across dependent projects. We employed social network analysis to create a collaboration network in the ecosystem, and mixed effects logistic regression and random forest techniques to measure the impact and predictive strength of the tested features. We find that gaining experience within the software ecosystem through active participation in issue-tracking systems, submitting pull requests, and collaborating with pull request integrators and experienced developers benefits all open-source contributors, especially project newcomers. The results show that combining ecosystem-wide factors with features studied in previous work to predict the outcome of pull requests reached an overall F1 score of 0.92., Comment: 34 pages, 2 figures, 4 tables

Software practitioners often face unfairness in their work, such as unequal recognition of contributions, gender bias, and unclear criteria for performance reviews. While the link between fairness and job satisfaction has been established in other fields, its relevance to software professionals remains underexplored. This study aims to examine how fairness perceptions relate to job satisfaction among software practitioners, focusing on both general trends and demographic-specific differences. We conducted an online survey of 108 software practitioners, followed by ordinal logistic regression to analyze the relationship between fairness perceptions and job satisfaction in software engineering contexts, with moderation analysis examining how this relationship varies across demographic groups. Our findings indicate that all four fairness dimensions, distributive, procedural, interpersonal, and informational, significantly affect both overall job satisfaction and satisfaction with job security. Among these, interpersonal fairness has the biggest impact, being more than twice as influential on overall job satisfaction. The relationship between fairness perceptions and job satisfaction is notably stronger for female, ethnically underrepresented, less experienced practitioners, and those with work limitations. Fairness in authorship emerged as an important factor for job satisfaction collectively, while fairness in policy implementation, high-demand situations, and working hours particularly impacted specific demographic groups. This study highlights the unique role of fairness in software engineering, offering strategies for organizations to promote fair practices and targeted approaches specific for certain demographic groups., Comment: This work has been submitted to the IEEE for possible publication

Mobile health has the potential to revolutionize health care delivery and patient engagement. In this work, we discuss how integrating Artificial Intelligence into digital health applications-focused on supply chain, patient management, and capacity building, among other use cases-can improve the health system and public health performance. We present an Artificial Intelligence and Reinforcement Learning platform that allows the delivery of adaptive interventions whose impact can be optimized through experimentation and real-time monitoring. The system can integrate multiple data sources and digital health applications. The flexibility of this platform to connect to various mobile health applications and digital devices and send personalized recommendations based on past data and predictions can significantly improve the impact of digital tools on health system outcomes. The potential for resource-poor settings, where the impact of this approach on health outcomes could be more decisive, is discussed specifically. This framework is, however, similarly applicable to improving efficiency in health systems where scarcity is not an issue., Comment: This article has been accepted for publication in Health Systems & Reform, published by Taylor & Francis

Surface-gated quantum dots (QDs) in semiconductor heterostructures represent a highly attractive platform for quantum computation and simulation. However, in this implementation, the barriers through which the QD is tunnel-coupled to source and drain reservoirs (or neighboring QDs) are usually non-rigid, and capacitively influenced by the plunger gate voltage (VP). In transport spectroscopy measurements, this leads to complete suppression of current and lifting of Coulomb blockade, for large negative and positive values of VP, respectively. Consequently, the charge-occupancy of the QD can be tuned over a rather small range of VP. By dynamically tuning the tunnel barriers to compensate for the capacitive effect of VP, here we demonstrate a protocol which allows the Coulomb blockade to be preserved over a remarkably large span of charge-occupancies, as demonstrated by clean Coulomb diamonds and well-resolved excited state features. The protocol will be highly beneficial for automated tuning and identification of the gatevoltage-space for optimal operation of QDs, in large arrays required for a scalable spin quantum computing architecture., Comment: 5 pages, 5 figures, 1 table

The increasing reliance on machine learning (ML) in computer security, particularly for malware classification, has driven significant advancements. However, the replicability and reproducibility of these results are often overlooked, leading to challenges in verifying research findings. This paper highlights critical pitfalls that undermine the validity of ML research in Android malware detection, focusing on dataset and methodological issues. We comprehensively analyze Android malware detection using two datasets and assess offline and continual learning settings with six widely used ML models. Our study reveals that when properly tuned, simpler baseline methods can often outperform more complex models. To address reproducibility challenges, we propose solutions for improving datasets and methodological practices, enabling fairer model comparisons. Additionally, we open-source our code to facilitate malware analysis, making it extensible for new models and datasets. Our paper aims to support future research in Android malware detection and other security domains, enhancing the reliability and reproducibility of published results.

Emulsion fuels have the potential to reduce both particulate matter and NOx emissions and can potentially improve the efficiency of combustion engines. However, their limited stability remains a critical barrier to practical use as an alternative fuel. In this study, we explore the evaporation behavior of thermodynamically stable water-in-oil microemulsions. The water-in-oil microemulsion droplets prepared from different types of oil were acoustically levitated and heated using a continuous laser at different irradiation intensities. We show that the evaporation characteristics of these microemulsions can be controlled by varying water-to-surfactant molar ratio ({\omega}) and volume fraction of the dispersed phase ({\phi}). The emulsion droplets undergo three distinct stages of evaporation, namely pre-heating, steady evaporation, and unsteady evaporation. During the steady evaporation phase, increasing {\phi} reduces the evaporation rate for a fixed {\omega}. It is observed that the evaporation of microemulsion is governed by the complex interplay between its constituents and their properties. We propose a parameter ({\eta}) denoting the volume fraction ratio between volatile and non-volatile components, which indicates the cumulative influence of various factors affecting the evaporation process. The evaporation of microemulsions eventually leads to the formation of solid spherical shells, which may undergo buckling. The distinction in the morphology of these shells is explored in detail using SEM imaging., Comment: 42 pages, 11 figures

Causal probing is an approach to interpreting foundation models, such as large language models, by training probes to recognize latent properties of interest from embeddings, intervening on probes to modify this representation, and analyzing the resulting changes in the model's behavior. While some recent works have cast doubt on the theoretical basis of several leading causal probing intervention methods, it has been unclear how to systematically and empirically evaluate their effectiveness in practice. To address this problem, we propose a general empirical analysis framework to evaluate the reliability of causal probing interventions, formally defining and quantifying two key causal probing desiderata: completeness (fully transforming the representation of the target property) and selectivity (minimally impacting other properties). Our formalism allows us to make the first direct comparisons between different families of causal probing methods (e.g., linear vs. nonlinear or counterfactual vs. nullifying interventions). We conduct extensive experiments across several leading methods, finding that (1) there is an inherent tradeoff between these criteria, and no method is able to consistently satisfy both at once; and (2) across the board, nullifying interventions are always far less complete than counterfactual interventions, indicating that nullifying methods may not be an effective approach to causal probing.

This paper introduces a reinforcement learning (RL) platform that enhances end-to-end user journeys in healthcare digital tools through personalization. We explore a case study with SwipeRx, the most popular all-in-one app for pharmacists in Southeast Asia, demonstrating how the platform can be used to personalize and adapt user experiences. Our RL framework is tested through a series of experiments with product recommendations tailored to each pharmacy based on real-time information on their purchasing history and in-app engagement, showing a significant increase in basket size. By integrating adaptive interventions into existing mobile health solutions and enriching user journeys, our platform offers a scalable solution to improve pharmaceutical supply chain management, health worker capacity building, and clinical decision and patient care, ultimately contributing to better healthcare outcomes., Comment: Presented at the Third Workshop on End-to-End Customer Journey Optimization at KDD 2024 (KDD CJ Workshop '24), August 26, Barcelona, Spain

Pharmacies are critical in healthcare systems, particularly in low- and middle-income countries. Procuring pharmacists with the right behavioral interventions or nudges can enhance their skills, public health awareness, and pharmacy inventory management, ensuring access to essential medicines that ultimately benefit their patients. We introduce a reinforcement learning operational system to deliver personalized behavioral interventions through mobile health applications. We illustrate its potential by discussing a series of initial experiments run with SwipeRx, an all-in-one app for pharmacists, including B2B e-commerce, in Indonesia. The proposed method has broader applications extending beyond pharmacy operations to optimize healthcare delivery., Comment: Presented at The First Workshop on AI Behavioral Science (AIBS'24) at KDD 2024, August 25, Barcelona, Spain

By providing evidence-based clinical decision support, digital tools and electronic health records can revolutionize patient management, especially in resource-poor settings where fewer health workers are available and often need more training. When these tools are integrated with AI, they can offer personalized support and adaptive interventions, effectively connecting community health workers (CHWs) and healthcare facilities. The CHARM (Community Health Access & Resource Management) app is an AI-native mobile app for CHWs. Developed through a joint partnership of Causal Foundry (CF) and mothers2mothers (m2m), CHARM empowers CHWs, mainly local women, by streamlining case management, enhancing learning, and improving communication. This paper details CHARM's development, integration, and upcoming reinforcement learning-based adaptive interventions, all aimed at enhancing health worker engagement, efficiency, and patient outcomes, thereby enhancing CHWs' capabilities and community health., Comment: Presented at the 7th epiDAMIK ACM SIGKDD International Workshop on Epidemiology meets Data Mining and Knowledge Discovery, August 26, 2024, Barcelona, Spain

We introduce Imagen 3, a latent diffusion model that generates high quality images from text prompts. We describe our quality and responsibility evaluations. Imagen 3 is preferred over other state-of-the-art (SOTA) models at the time of evaluation. In addition, we discuss issues around safety and representation, as well as methods we used to minimize the potential harm of our models.

In response to the unique challenges of Mixed Reality (MR) game development, we developed GAMR, an analytics tool specifically designed for MR games. GAMR aims to assist developers in identifying and resolving gameplay issues effectively. It features reconstructed gameplay sessions, heatmaps for data visualization, a comprehensive annotation system, and advanced tracking for hands, camera, input, and audio, providing in-depth insights for nuanced game analysis. To evaluate GAMR's effectiveness, we conducted an experimental study with game development students across two game genres: action-adventure and first-person shooter (FPS). The participants used GAMR and provided feedback on its utility. The results showed a significant positive impact of GAMR in both genres, particularly in action-adventure games. This study demonstrates GAMR's effectiveness in MR game development and suggests its potential to influence future MR game analytics, addressing the specific needs of developers in this evolving area., Comment: 32 pages

Parameter-efficient fine-tuning techniques like Low-Rank Adaptation (LoRA) have revolutionized the adaptation of large language models (LLMs) to diverse tasks. Recent efforts have explored mixtures of LoRA modules for multi-task settings. However, our analysis reveals redundancy in the down-projection matrices of these architectures. This observation motivates our proposed method, Mixture of Dyadic Experts (MoDE), which introduces a novel design for efficient multi-task adaptation. This is done by sharing the down-projection matrix across tasks and employing atomic rank-one adapters, coupled with routers that allow more sophisticated task-level specialization. Our design allows for more fine-grained mixing, thereby increasing the model's ability to jointly handle multiple tasks. We evaluate MoDE on the Supernatural Instructions (SNI) benchmark consisting of a diverse set of 700+ tasks and demonstrate that it outperforms state-of-the-art multi-task parameter-efficient fine-tuning (PEFT) methods, without introducing additional parameters. Our findings contribute to a deeper understanding of parameter efficiency in multi-task LLM adaptation and provide a practical solution for deploying high-performing, lightweight models.

We find the commutant of a pure contractive semigroup on a Hilbert space. We demonstrate that any tuple of doubly commuting pure contractive semigroups can be dilated to a tuple of doubly commuting pure isometric semigroups. En route, we obtain a complete model for the tuples of doubly commuting isometric semigroups., Comment: Preliminary version, comments are welcome

Cyber threats are constantly evolving. Extracting actionable insights from unstructured Cyber Threat Intelligence (CTI) data is essential to guide cybersecurity decisions. Increasingly, organizations like Microsoft, Trend Micro, and CrowdStrike are using generative AI to facilitate CTI extraction. This paper addresses the challenge of automating the extraction of actionable CTI using advancements in Large Language Models (LLMs) and Knowledge Graphs (KGs). We explore the application of state-of-the-art open-source LLMs, including the Llama 2 series, Mistral 7B Instruct, and Zephyr for extracting meaningful triples from CTI texts. Our methodology evaluates techniques such as prompt engineering, the guidance framework, and fine-tuning to optimize information extraction and structuring. The extracted data is then utilized to construct a KG, offering a structured and queryable representation of threat intelligence. Experimental results demonstrate the effectiveness of our approach in extracting relevant information, with guidance and fine-tuning showing superior performance over prompt engineering. However, while our methods prove effective in small-scale tests, applying LLMs to large-scale data for KG construction and Link Prediction presents ongoing challenges., Comment: 6th Workshop on Attackers and Cyber-Crime Operations, 12 pages, 1 figure, 9 tables

In this work, we demonstrate the Empirical Bayes approach to learning a Dynamic Bayesian Network. By starting with several point estimates of structure and weights, we can use a data-driven prior to subsequently obtain a model to quantify uncertainty. This approach uses a recent development of Generalized Variational Inference, and indicates the potential of sampling the uncertainty of a mixture of DAG structures as well as a parameter posterior.

Cyber threat intelligence (CTI) is crucial in today's cybersecurity landscape, providing essential insights to understand and mitigate the ever-evolving cyber threats. The recent rise of Large Language Models (LLMs) have shown potential in this domain, but concerns about their reliability, accuracy, and hallucinations persist. While existing benchmarks provide general evaluations of LLMs, there are no benchmarks that address the practical and applied aspects of CTI-specific tasks. To bridge this gap, we introduce CTIBench, a benchmark designed to assess LLMs' performance in CTI applications. CTIBench includes multiple datasets focused on evaluating knowledge acquired by LLMs in the cyber-threat landscape. Our evaluation of several state-of-the-art models on these tasks provides insights into their strengths and weaknesses in CTI contexts, contributing to a better understanding of LLM capabilities in CTI.

Complex multi-step reasoning tasks, such as solving mathematical problems or generating code, remain a significant hurdle for even the most advanced large language models (LLMs). Verifying LLM outputs with an Outcome Reward Model (ORM) is a standard inference-time technique aimed at enhancing the reasoning performance of LLMs. However, this still proves insufficient for reasoning tasks with a lengthy or multi-hop reasoning chain, where the intermediate outcomes are neither properly rewarded nor penalized. Process supervision addresses this limitation by assigning intermediate rewards during the reasoning process. To date, the methods used to collect process supervision data have relied on either human annotation or per-step Monte Carlo estimation, both prohibitively expensive to scale, thus hindering the broad application of this technique. In response to this challenge, we propose a novel divide-and-conquer style Monte Carlo Tree Search (MCTS) algorithm named \textit{OmegaPRM} for the efficient collection of high-quality process supervision data. This algorithm swiftly identifies the first error in the Chain of Thought (CoT) with binary search and balances the positive and negative examples, thereby ensuring both efficiency and quality. As a result, we are able to collect over 1.5 million process supervision annotations to train a Process Reward Model (PRM). Utilizing this fully automated process supervision alongside the weighted self-consistency algorithm, we have enhanced the instruction tuned Gemini Pro model's math reasoning performance, achieving a 69.4\% success rate on the MATH benchmark, a 36\% relative improvement from the 51\% base model performance. Additionally, the entire process operates without any human intervention, making our method both financially and computationally cost-effective compared to existing methods., Comment: 18 pages, 5 figures, 1 table

We consider holomorphic functions on the unit disc whose images are contained in a strip of the complex plane. Under an additional condition, such functions are constants. We also consider appropriate operator valued versions. Applications are found to the theory of semigroups., Comment: To appear in Infinite Dimensional Analysis, Quantum Probability and Related Topics

Animals have evolved to rapidly detect and recognise brief and intermittent encounters with odour packages, exhibiting recognition capabilities within milliseconds. Artificial olfaction has faced challenges in achieving comparable results -- existing solutions are either slow; or bulky, expensive, and power-intensive -- limiting applicability in real-world scenarios for mobile robotics. Here we introduce a miniaturised high-speed electronic nose; characterised by high-bandwidth sensor readouts, tightly controlled sensing parameters and powerful algorithms. The system is evaluated on a high-fidelity odour delivery benchmark. We showcase successful classification of tens-of-millisecond odour pulses, and demonstrate temporal pattern encoding of stimuli switching with up to 60 Hz. Those timescales are unprecedented in miniaturised low-power settings, and demonstrably exceed the performance observed in mice. For the first time, it is possible to match the temporal resolution of animal olfaction in robotic systems. This will allow for addressing challenges in environmental and industrial monitoring, security, neuroscience, and beyond.

In this paper, we present a multidimensional, highly effective method for aggregating data for wireless sensor networks while maintaining privacy. The suggested system is resistant to data loss and secure against both active and passive privacy compromising attacks, such as the coalition attack from a rogue base station and kidnapped sensor nodes. With regard to cluster size, it achieves consistent communication overhead, which is helpful in large-scale WSNs. Due to its constant size communication overhead, the suggested strategy outperforms the previous privacy-preserving data aggregation scheme not only in terms of privacy preservation but also in terms of communication complexity and energy costs., Comment: 4 pages

Large Language Models (LLMs) have demonstrated potential in cybersecurity applications but have also caused lower confidence due to problems like hallucinations and a lack of truthfulness. Existing benchmarks provide general evaluations but do not sufficiently address the practical and applied aspects of LLM performance in cybersecurity-specific tasks. To address this gap, we introduce the SECURE (Security Extraction, Understanding \& Reasoning Evaluation), a benchmark designed to assess LLMs performance in realistic cybersecurity scenarios. SECURE includes six datasets focussed on the Industrial Control System sector to evaluate knowledge extraction, understanding, and reasoning based on industry-standard sources. Our study evaluates seven state-of-the-art models on these tasks, providing insights into their strengths and weaknesses in cybersecurity contexts, and offer recommendations for improving LLMs reliability as cyber advisory tools.

Gliomas are the most common malignant primary brain tumors in adults and one of the deadliest types of cancer. There are many challenges in treatment and monitoring due to the genetic diversity and high intrinsic heterogeneity in appearance, shape, histology, and treatment response. Treatments include surgery, radiation, and systemic therapies, with magnetic resonance imaging (MRI) playing a key role in treatment planning and post-treatment longitudinal assessment. The 2024 Brain Tumor Segmentation (BraTS) challenge on post-treatment glioma MRI will provide a community standard and benchmark for state-of-the-art automated segmentation models based on the largest expert-annotated post-treatment glioma MRI dataset. Challenge competitors will develop automated segmentation models to predict four distinct tumor sub-regions consisting of enhancing tissue (ET), surrounding non-enhancing T2/fluid-attenuated inversion recovery (FLAIR) hyperintensity (SNFH), non-enhancing tumor core (NETC), and resection cavity (RC). Models will be evaluated on separate validation and test datasets using standardized performance metrics utilized across the BraTS 2024 cluster of challenges, including lesion-wise Dice Similarity Coefficient and Hausdorff Distance. Models developed during this challenge will advance the field of automated MRI segmentation and contribute to their integration into clinical practice, ultimately enhancing patient care., Comment: 10 pages, 4 figures, 1 table

Collaborative software development happens in teams, that cooperate on shared artefacts, and discuss development on online platforms. Due to the complexity of development and the variety of teams, software components often act as effective containers for parallel work and teams. Past research has shown how communication between team members, especially in an open-source environment, can become extremely toxic, and lead to members leaving the development team. This has a direct effect on the evolution and maintenance of the project in which the former members were active in. The purpose of our study is two-fold: first, we propose an approach to evaluate, at a finer granularity, the positive and negative emotions in the communication between developers; and second, we aim to characterise a project's development paths, or components, as more or less impacted by the emotions. Our analysis evaluates single sentences rather than whole messages as the finest granularity of communication. The previous study found that the high positivity or negativity at the sentence level may indirectly impact the writer him/herself, or the reader. In this way, we could highlight specific paths of Gentoo as the most affected by negative emotions, and show how negative emotions have evolved and changed along the same paths. By joining the analysis of the mailing lists, from which we derive the sentiment of the developers, with the information derived from the development logs, we obtained a longitudinal picture of how development paths have been historically affected by positive or negative emotions. Our study shows that, in recent years, negative emotions have generally decreased in the communication between Gentoo developers. We also show how file paths, as collaborative software development artefacts, were more or less impacted by the emotions of the developers.

The exoplanet detection is the most exciting and challenging field of astronomy. The discovery of many exoplanets has revolutionized our understanding of the formation and evolution of planetary systems and has showed new ways to search for extra terrestrial life. In recent years, some primary methods of exoplanet detection like transit, radial velocity, gravitational microlensing, direct imaging and astrometry have played a important role for the discovery of exoplanets. In this paper we explored detection methodologies with all the implications and analytics of comparison between them. Here we also discussed on different machine learning algorithms for exoplanet detection and visualization. Finally, concluded with the significant discoveries made by some missions and their implications on our understanding for the properties, environmental conditions and importance of exoplanets in the universe., Comment: 8 pages, 5 figures, 4 tables

Deep neural networks for classification are vulnerable to adversarial attacks, where small perturbations to input samples lead to incorrect predictions. This susceptibility, combined with the black-box nature of such networks, limits their adoption in critical applications like autonomous driving. Feature-attribution-based explanation methods provide relevance of input features for model predictions on input samples, thus explaining model decisions. However, we observe that both model predictions and feature attributions for input samples are sensitive to noise. We develop a practical method for this characteristic of model prediction and feature attribution to detect adversarial samples. Our method, PASA, requires the computation of two test statistics using model prediction and feature attribution and can reliably detect adversarial samples using thresholds learned from benign samples. We validate our lightweight approach by evaluating the performance of PASA on varying strengths of FGSM, PGD, BIM, and CW attacks on multiple image and non-image datasets. On average, we outperform state-of-the-art statistical unsupervised adversarial detectors on CIFAR-10 and ImageNet by 14\% and 35\% ROC-AUC scores, respectively. Moreover, our approach demonstrates competitive performance even when an adversary is aware of the defense mechanism., Comment: 9th IEEE European Symposium on Security and Privacy

Memory safety violations in low-level code, written in languages like C, continues to remain one of the major sources of software vulnerabilities. One method of removing such violations by construction is to port C code to a safe C dialect. Such dialects rely on programmer-supplied annotations to guarantee safety with minimal runtime overhead. This porting, however, is a manual process that imposes significant burden on the programmer and, hence, there has been limited adoption of this technique. The task of porting not only requires inferring annotations, but may also need refactoring/rewriting of the code to make it amenable to such annotations. In this paper, we use Large Language Models (LLMs) towards addressing both these concerns. We show how to harness LLM capabilities to do complex code reasoning as well as rewriting of large codebases. We also present a novel framework for whole-program transformations that leverages lightweight static analysis to break the transformation into smaller steps that can be carried out effectively by an LLM. We implement our ideas in a tool called MSA that targets the CheckedC dialect. We evaluate MSA on several micro-benchmarks, as well as real-world code ranging up to 20K lines of code. We showcase superior performance compared to a vanilla LLM baseline, as well as demonstrate improvement over a state-of-the-art symbolic (non-LLM) technique.

Douglas and Rudin proved that any unimodular function on the unit circle $\T$ can be uniformly approximated by quotients of inner functions. We extend this result to the operator-valued unimodular functions defined on the boundary of the open unit ball of $\mathbb{C}^d$. Our proof technique combines the spectral theorem for unitary operators with the Douglas-Rudin theorem in the scalar case to bootstrap the result to the operator-valued case. This yields a new proof and a significant generalization of Barclay's result [Proc. Lond. Math. Soc. 2009] on the approximation of matrix-valued unimodular functions on $\T$., Comment: 10 pages+References. Minor improvements in style. Some typos fixed. Current version to appear in JFA

While Reinforcement Learning from Human Feedback (RLHF) effectively aligns pretrained Large Language and Vision-Language Models (LLMs, and VLMs) with human preferences, its computational cost and complexity hamper its wider adoption. To alleviate some of the computational burden of fine-tuning, parameter efficient methods, like LoRA were introduced. In this work, we empirically evaluate the setup of Parameter Efficient Reinforcement Learning from Human Feedback (PE-RLHF) that leverages LoRA fine-tuning for Reward Modeling, and Reinforcement Learning. We benchmark the PE-RLHF setup on six diverse datasets spanning summarization, harmless/helpful response generation, UI automation, and visual question answering in terms of effectiveness of the trained models, and the training resources required. Our findings show, for the first time, that PE-RLHF achieves comparable performance to RLHF, while significantly reducing training time (up to 90% faster for reward models, and 30% faster for RL), and memory footprint (up to 50% reduction for reward models, and 27% for RL). We provide comprehensive ablations across LoRA ranks, and model sizes for both reward modeling and reinforcement learning. By mitigating the computational burden associated with RLHF, we push for a broader adoption of PE-RLHF as an alignment technique for LLMs and VLMs.

LLMs are revolutionizing NLP tasks. However, the use of the most advanced LLMs, such as GPT-4, is often prohibitively expensive for most specialized fields. We introduce HEAL, the first continuously trained 13B LLaMA2-based LLM that is purpose-built for medical conversations and measured on automated scribing. Our results demonstrate that HEAL outperforms GPT-4 and PMC-LLaMA in PubMedQA, with an accuracy of 78.4\%. It also achieves parity with GPT-4 in generating medical notes. Remarkably, HEAL surpasses GPT-4 and Med-PaLM 2 in identifying more correct medical concepts and exceeds the performance of human scribes and other comparable models in correctness and completeness., Comment: Accepted to NAACL 2024

Peer review often involves reviewers submitting their independent reviews, followed by a discussion among reviewers of each paper. A question among policymakers is whether the reviewers of a paper should be anonymous to each other during the discussion. We shed light on this by conducting a randomized controlled trial at the UAI 2022 conference. We randomly split the reviewers and papers into two conditions--one with anonymous discussions and the other with non-anonymous discussions, and conduct an anonymous survey of all reviewers, to address the following questions: 1. Do reviewers discuss more in one of the conditions? Marginally more in anonymous (n = 2281, p = 0.051). 2. Does seniority have more influence on final decisions when non-anonymous? Yes, the decisions are closer to senior reviewers' scores in the non-anonymous condition than in anonymous (n = 484, p = 0.04). 3. Are reviewers more polite in one of the conditions? No significant difference in politeness of reviewers' text-based responses (n = 1125, p = 0.72). 4. Do reviewers' self-reported experiences differ across the two conditions? No significant difference for each of the five questions asked (n = 132 and p > 0.3). 5. Do reviewers prefer one condition over the other? Yes, there is a weak preference for anonymous discussions (n = 159 and Cohen's d= 0.25). 6. What do reviewers consider important to make policy on anonymity among reviewers? Reviewers' feeling of safety in expressing their opinions was rated most important, while polite communication among reviewers was rated least important (n = 159). 7. Have reviewers experienced dishonest behavior due to non-anonymity in discussions? Yes, roughly 7% of respondents answered affirmatively (n = 167). Overall, this experiment reveals evidence supporting an anonymous discussion setup in the peer-review process, in terms of the evaluation criteria considered., Comment: 18 pages, 4 figures, 3 tables