Search

Large language models (LLMs) have shown great potential for the automatic generation of feedback in a wide range of computing contexts. However, concerns have been voiced around the privacy and ethical implications of sending student work to proprietary models. This has sparked considerable interest in the use of open source LLMs in education, but the quality of the feedback that such open models can produce remains understudied. This is a concern as providing flawed or misleading generated feedback could be detrimental to student learning. Inspired by recent work that has utilised very powerful LLMs, such as GPT-4, to evaluate the outputs produced by less powerful models, we conduct an automated analysis of the quality of the feedback produced by several open source models using a dataset from an introductory programming course. First, we investigate the viability of employing GPT-4 as an automated evaluator by comparing its evaluations with those of a human expert. We observe that GPT-4 demonstrates a bias toward positively rating feedback while exhibiting moderate agreement with human raters, showcasing its potential as a feedback evaluator. Second, we explore the quality of feedback generated by several leading open-source LLMs by using GPT-4 to evaluate the feedback. We find that some models offer competitive performance with popular proprietary LLMs, such as ChatGPT, indicating opportunities for their responsible use in educational settings., Comment: 7 pages, 4 figures, 2 tables. Accepted for publication at the 29th annual ACM conference on Innovation and Technology in Computer Science Education (ITiCSE 2024)

The last twenty years have seen the development and popularity of network measurement infrastructures. Internet measurement platforms have become common and have demonstrated their relevance in Internet understanding and security observation. However, despite their popularity, those platforms lack of flexibility and reactivity, as they are usually used for longitudinal measurements. As a consequence, they may miss detecting events that are security or Internet-related. During the same period, operating systems have evolved to virtual machines (VMs) as self-contained units for running applications, with the recent rise of unikernels, ultra-lightweight VMs tailored for specific applications, eliminating the need for a host OS. In this paper, we advocate that measurement infrastructures could take advantage of unikernels to become more flexible and efficient. We propose uTNT, a proof-of-concept unikernel-based implementation of TNT, a traceroute extension able to reveal MPLS tunnels. This paper documents the full toolchain for porting TNT into a unikernel and evaluates uTNT performance with respect to more traditional approaches. The paper also discusses a use case in which uTNT could find a suitable usage. uTNT source code is publicly available on Gitlab., Comment: 10 pages, 11 figures, IFIP extended-abstract

As robotics becomes increasingly vital for environmental protection, there is a growing need for effective deployment methods that match the pace of robotics innovation. Current strategies often fall short, leaving a gap between the potential of robotics and their practical application in the field. Addressing this challenge, we introduce a mobile hub concept designed to provide the necessary infrastructure and support for deploying a diverse, multi-domain robot team effectively. This paper presents the development and insights into `SVAN' (Synchronous Team-Robot Van), a prototype of our mobile hub concept. We delve into the mechanical construction and software setup of SVAN, offering a comprehensive overview of its capabilities and design considerations. Further, we discuss the hardware specifications and share valuable lessons learned during the prototype's development and deployment. In addition to this paper, an accepted video complements our exploration by depicting SVAN in its envisioned role as an environmental guardian, highlighting its potential in ecological monitoring and preservation. Furthermore, our discussion is enriched by referencing a previously accepted paper detailing a novel methodology for continuous UAV mission cycling enabled by a mobile hub like SVAN. These accompanying works underscore our contribution towards addressing the existing gaps in robot deployment strategies, presenting a scalable and efficient framework to overcome operational challenges in environmental robotics., Comment: 7 pages, 10 figures, accepted paper at IEEE ICRA Workshop on Field Robotics 2024

In this study, we present a model for the behavior of Dirac particles under the tensor effect in the spherical core/shell regime. We examine the change of energy levels corresponding to the particles localized in a space of approximately 1.0 fm in the core region of the quantum sphere, with the well width. It also occurs from the analytical solutions that the two different levels accompany particle states of the same mass. Additionally, the solutions exhibiting anomalous behavior, giving rise to antiparticle-type states, occur at heavier mass.

As the amount and complexity of data grows, retrieving it has become a more difficult task that requires greater knowledge and resources. This is especially true for the logistics industry, where new technologies for data collection provide tremendous amounts of interconnected real-time data. The Intelligent Query and Learning System (IQLS) simplifies the process by allowing natural language use to simplify data retrieval . It maps structured data into a framework based on the available metadata and available data models. This framework creates an environment for an agent powered by a Large Language Model. The agent utilizes the hierarchical nature of the data to filter iteratively by making multiple small context-aware decisions instead of one-shot data retrieval. After the Data filtering, the IQLS enables the agent to fulfill tasks given by the user query through interfaces. These interfaces range from multimodal transportation information retrieval to route planning under multiple constraints. The latter lets the agent define a dynamic object, which is determined based on the query parameters. This object represents a driver capable of navigating a road network. The road network is depicted as a graph with attributes based on the data. Using a modified version of the Dijkstra algorithm, the optimal route under the given constraints can be determined. Throughout the entire process, the user maintains the ability to interact and guide the system. The IQLS is showcased in a case study on the Canadian logistics sector, allowing geospatial, visual, tabular and text data to be easily queried semantically in natural language.

Next basket recommendation (NBR) is a special type of sequential recommendation that is increasingly receiving attention. So far, most NBR studies have focused on optimizing the accuracy of the recommendation, whereas optimizing for beyond-accuracy metrics, e.g., item fairness and diversity remains largely unexplored. Recent studies into NBR have found a substantial performance difference between recommending repeat items and explore items. Repeat items contribute most of the users' perceived accuracy compared with explore items. Informed by these findings, we identify a potential "short-cut" to optimize for beyond-accuracy metrics while maintaining high accuracy. To leverage and verify the existence of such short-cuts, we propose a plug-and-play two-step repetition-exploration (TREx) framework that treats repeat items and explores items separately, where we design a simple yet highly effective repetition module to ensure high accuracy, while two exploration modules target optimizing only beyond-accuracy metrics. Experiments are performed on two widely-used datasets w.r.t. a range of beyond-accuracy metrics, viz. five fairness metrics and three diversity metrics. Our experimental results verify the effectiveness of TREx. Prima facie, this appears to be good news: we can achieve high accuracy and improved beyond-accuracy metrics at the same time. However, we argue that the real-world value of our algorithmic solution, TREx, is likely to be limited and reflect on the reasonableness of the evaluation setup. We end up challenging existing evaluation paradigms, particularly in the context of beyond-accuracy metrics, and provide insights for researchers to navigate potential pitfalls and determine reasonable metrics to consider when optimizing for accuracy and beyond-accuracy metrics., Comment: To appear at SIGIR'24

Star formation estimates based on the counting of YSOs is commonly applied to nearby star-forming regions in the Galaxy. With this method, the SFRs are measured using the counts of YSOs in a particular protostellar Class, a typical protostellar mass, and the lifetime associated with this Class. However, the assumptions underlying the validity of the method such as that of a constant star formation history (SFH) and whether the method is valid for all protostellar Classes has never been fully tested. In this work, we use Monte Carlo models to test the validity of the method. We build synthetic clusters in which stars form at times that are randomly drawn from a specified SFH. The latter is either constant or time-dependent with a burst like behavior. The masses of the protostars are randomly drawn from an IMF which can be either similar to that of the Milky Way field or be variable . For each star in every cluster, the lifetimes associated with the different protostellar classes are also randomly drawn from Gaussian distribution functions centered around their most likely value as suggested by the observations. We find that only the SFR derived using the Class 0 population can reproduce the true SFR at all epochs, and this is true irrespective of the shape of the SFH. For a constant SFH, the SFR derived using the more evolved populations of protostars (Classes I, F, II, and III) reproduce the real SFR only at later epochs which correspond to epochs at which their numbers have reached a steady state. For a time-dependent burst-like SFH, all SFR estimates based on the number counts of the evolved populations fail to reproduce the true SFR. We also show how the offsets between Class I and Class II based SFRs and the true SFR plotted as a function of the number ratios of Class I and Class II versus Class III YSOs can be used in order to constrain the SFH of observed molecular clouds., Comment: Accepted for publication in Astronomy & Astrophysics. Missing references included. Matches the published version

In this paper, we present a novel approach to improving software quality and efficiency through a Large Language Model (LLM)-based model designed to review code and identify potential issues. Our proposed LLM-based AI agent model is trained on large code repositories. This training includes code reviews, bug reports, and documentation of best practices. It aims to detect code smells, identify potential bugs, provide suggestions for improvement, and optimize the code. Unlike traditional static code analysis tools, our LLM-based AI agent has the ability to predict future potential risks in the code. This supports a dual goal of improving code quality and enhancing developer education by encouraging a deeper understanding of best practices and efficient coding techniques. Furthermore, we explore the model's effectiveness in suggesting improvements that significantly reduce post-release bugs and enhance code review processes, as evidenced by an analysis of developer sentiment toward LLM feedback. For future work, we aim to assess the accuracy and efficiency of LLM-generated documentation updates in comparison to manual methods. This will involve an empirical study focusing on manually conducted code reviews to identify code smells and bugs, alongside an evaluation of best practice documentation, augmented by insights from developer discussions and code reviews. Our goal is to not only refine the accuracy of our LLM-based tool but also to underscore its potential in streamlining the software development lifecycle through proactive code improvement and education., Comment: 8 pages

Accurate and timely mapping of flood extent from high-resolution satellite imagery plays a crucial role in disaster management such as damage assessment and relief activities. However, current state-of-the-art solutions are based on U-Net, which can-not segment the flood pixels accurately due to the ambiguous pixels (e.g., tree canopies, clouds) that prevent a direct judgement from only the spectral features. Thanks to the digital elevation model (DEM) data readily available from sources such as United States Geological Survey (USGS), this work explores the use of an elevation map to improve flood extent mapping. We propose, EvaNet, an elevation-guided segmentation model based on the encoder-decoder architecture with two novel techniques: (1) a loss function encoding the physical law of gravity that if a location is flooded (resp. dry), then its adjacent locations with a lower (resp. higher) elevation must also be flooded (resp. dry); (2) a new (de)convolution operation that integrates the elevation map by a location sensitive gating mechanism to regulate how much spectral features flow through adjacent layers. Extensive experiments show that EvaNet significantly outperforms the U-Net baselines, and works as a perfect drop-in replacement for U-Net in existing solutions to flood extent mapping., Comment: Published at the International Joint Conference on Artificial Intelligence (IJCAI, 2024)

Whilst holding great promise for low noise, ease of operation and networking, useful photonic quantum computing has been precluded by the need for beyond-state-of-the-art components, manufactured by the millions. Here we introduce a manufacturable platform for quantum computing with photons. We benchmark a set of monolithically-integrated silicon photonics-based modules to generate, manipulate, network, and detect photonic qubits, demonstrating dual-rail photonic qubits with $99.98\% \pm 0.01\%$ state preparation and measurement fidelity, Hong-Ou-Mandel quantum interference between independent photon sources with $99.50\%\pm0.25\%$ visibility, two-qubit fusion with $99.22\%\pm0.12\%$ fidelity, and a chip-to-chip qubit interconnect with $99.72\%\pm0.04\%$ fidelity, not accounting for loss. In addition, we preview a selection of next generation technologies, demonstrating low-loss silicon nitride waveguides and components, fabrication-tolerant photon sources, high-efficiency photon-number-resolving detectors, low-loss chip-to-fiber coupling, and barium titanate electro-optic phase shifters., Comment: 8 pages, 5 figures

We show that for each $n \geq 2$, the systoles of closed hyperbolic $n$-manifolds form a dense subset of $(0, +\infty)$. We also show that for any $n\geq 2$ and any Salem number $\lambda$, there is a closed arithmetic hyperbolic $n$-manifold of systole $\log(\lambda)$. In particular, the Salem conjecture holds if and only if the systoles of closed arithmetic hyperbolic manifolds in some (any) dimension fail to be dense in $(0, +\infty)$., Comment: Minor edits, as per the referee's comments. Final version, to appear in Comptes Rendus Math\'ematique

Training Deep Neural Networks (DNNs) can be computationally demanding, particularly when dealing with large models. Recent work has aimed to mitigate this computational challenge by introducing 8-bit floating-point (FP8) formats for multiplication. However, accumulations are still done in either half (16-bit) or single (32-bit) precision arithmetic. In this paper, we investigate lowering accumulator word length while maintaining the same model accuracy. We present a multiply-accumulate (MAC) unit with FP8 multiplier inputs and FP12 accumulations, which leverages an optimized stochastic rounding (SR) implementation to mitigate swamping errors that commonly arise during low precision accumulations. We investigate the hardware implications and accuracy impact associated with varying the number of random bits used for rounding operations. We additionally attempt to reduce MAC area and power by proposing a new scheme to support SR in floating-point MAC and by removing support for subnormal values. Our optimized eager SR unit significantly reduces delay and area when compared to a classic lazy SR design. Moreover, when compared to MACs utilizing single-or half-precision adders, our design showcases notable savings in all metrics. Furthermore, our approach consistently maintains near baseline accuracy across a diverse range of computer vision tasks, making it a promising alternative for low-precision DNN training.

We characterize the quantum states dual to entanglement wedges in arbitrary spacetimes, in settings where the matter entropy can be neglected compared to the geometric entropy. In AdS/CFT, such states obey special entropy inequalities known as the holographic entropy cone. In particular, the mutual information of CFT subregions is monogamous (MMI). We extend this result to arbitrary spacetimes, using a recent proposal for the generalized entanglement wedge e(a) of a gravitating region a. Given independent input regions a, b, and c, we prove MMI: Area[e(a)]+Area[e(b)]+Area[e(c)]-Area[e(ab)]-Area[e(bc)]-Area[e(ca)]+Area[e(abc)] $\leq$ 0. We expect that the full holographic entropy cone can be extended to arbitrary spacetimes using similar methods., Comment: 13 pages, 4 figures

Large Language Models (LLMs) are revolutionizing Software Engineering (SE) by introducing innovative methods for tasks such as collecting requirements, designing software, generating code, and creating test cases, among others. This article focuses on requirements engineering, typically seen as the initial phase of software development that involves multiple system stakeholders. Despite its key role, the challenge of identifying requirements and satisfying all stakeholders within time and budget constraints remains significant. To address the challenges in requirements engineering, this study introduces a web-based software tool utilizing AI agents and prompt engineering to automate task prioritization and apply diverse prioritization techniques, aimed at enhancing project management within the agile framework. This approach seeks to transform the prioritization of agile requirements, tackling the substantial challenge of meeting stakeholder needs within set time and budget limits. Furthermore, the source code of our developed prototype is available on GitHub, allowing for further experimentation and prioritization of requirements, facilitating research and practical application.

The use of pointed or edged tools or objects is one of the most challenging aspects of today's application of physical human-robot interaction (pHRI). One reason for this is that the severity of harm caused by such edged or pointed impactors is less well studied than for blunt impactors. Consequently, the standards specify well-reasoned force and pressure thresholds for blunt impactors and advise avoiding any edges and corners in contacts. Nevertheless, pointed or edged impactor geometries cannot be completely ruled out in real pHRI applications. For example, to allow edged or pointed tools such as screwdrivers near human operators, the knowledge of injury severity needs to be extended so that robot integrators can perform well-reasoned, time-efficient risk assessments. In this paper, we provide the initial datasets on injury prevention for the human hand based on drop tests with surrogates for the human hand, namely pig claws and chicken drumsticks. We then demonstrate the ease and efficiency of robot use using the dataset for contact on two examples. Finally, our experiments provide a set of injuries that may also be expected for human subjects under certain robot mass-velocity constellations in collisions. To extend this work, testing on human samples and a collaborative effort from research institutes worldwide is needed to create a comprehensive human injury avoidance database for any pHRI scenario and thus for safe pHRI applications including edged and pointed geometries., Comment: \c{opyright} 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Technical Debt is a common issue that arises when short-term gains are prioritized over long-term costs, leading to a degradation in the quality of the code. Self-Admitted Technical Debt (SATD) is a specific type of Technical Debt that involves documenting code to remind developers of its debt. Previous research has explored various aspects of SATD, including detection methods, distribution, and its impact on software quality. To better understand SATD, one comprehension technique is to examine its co-occurrence with other activities, such as refactoring and bug fixing. This study investigates the relationship between removing and adding SATD and activities such as refactoring, bug fixing, adding new features, and testing. To do so, we analyzed 77 open-source Java projects using TODO/FIXME/XXX removal or addition in inline comments as indicators of SATD. We examined the co-occurrence of SATD with each activity in each project through chi-square and odds ratio evaluations. Our results show that SATD removal occurs simultaneously with refactoring in 95% of projects, while its addition occurs in 89% of projects. Furthermore, we found that three types of refactoring - "move class", "remove method", and "move attribute" - occur more frequently in the presence of SATD. However, their distribution is similar in projects with and without SATD., Comment: 6 pages, DOI: 10.1109/ICCKE60553.2023.10326279

Relying on a teacher-level randomized experiment with a sample of 87 middle and high school teachers, this study was designed to examine the implementation and impact of MyTeachingPartner-Secondary delivered by local coaches who were trained and supported by the program provider. Due to disruptions caused by the COVID-19 pandemic, implementation of the 2-year program during the first year (2021-22) was weak, and only four of six study districts continued to participate in the study in the second year. Implementation during the second year was stronger but still did not achieve fidelity based on pre-specified fidelity thresholds. Analyses of teacher survey data revealed that the program had a statistically significant positive impact on teachers' enthusiasm about teaching and a marginally significant positive impact on teachers' self-efficacy at the end of the 2-year program. Nevertheless, the study did not find any significant impact on the quality of classroom interactions or student engagement based on observations at the end of Year 2 or on students' math or English language arts achievement at the end of Year 1. (Year 2 student achievement data are not yet available.) Results from this study need to be interpreted with caution given study limitations resulting largely from the influence of the pandemic. [American Institutes for Research (AIR) collaborated with Teachstone to implement this study.]