Search

In many low- and medium-income countries (LMICs), student-centered pedagogies are often implicitly or explicitly at the heart of innovative pedagogical reforms. In recent years, there has been a growing emphasis on student-centered pedagogies, which aim to shift power dynamics, increase interaction, and prioritize the needs of learners. Many international agencies, governments, and education experts view these pedagogies as "best practices" or a pedagogical "silver bullet" to improve classroom practice. This paper is the second in a series of three working papers meant to serve as references and conversation starters for policymakers and researchers as they navigate pedagogical reform for education system transformation in their local contexts. Together, the three working papers emphasize the need for more locally driven collaborative research on how the interaction of culture, local education ecosystems, and learning theories--collectively called Invisible Pedagogical Mindsets--influences teachers' pedagogical choices in the classroom. This paper details why the authors recommend policymakers examine Invisible Pedagogical Mindsets in their local context to inform pedagogical reforms. The authors discuss the reasons why generalized "best practices"--namely "student-centered pedagogies" as currently implemented--do not often successfully transfer to new cultures, countries, and contexts and argue that many pedagogical reforms do not adequately consider the Invisible Pedagogical Mindsets embedded in each local context.

Although global access to schooling has increased over the last several decades, Sustainable Development Goal 4 (SDG 4), which champions inclusive, equitable, quality education, is far from being achieved. Experts predict that if the global community continues to operate education systems in the same way, by 2030, only one in six countries will reach the universal secondary school completion targets, and approximately 300 million students in school will continue to lack basic numeracy and literacy skills. The 2022 United Nations Transforming Education Summit emphasized the urgent need for a complete overhaul of education systems to meet SDG 4 targets. One significant outcome of the summit was a call to improve student learning by transforming teacher classroom practice. This paper is the first in a series of three working papers meant to serve as references and conversation starters for policymakers and researchers as they navigate pedagogical reform for education system transformation in their local contexts. This paper explores various definitions of pedagogies, the lack of consensus on what pedagogy means in practice, and the effects of Invisible Pedagogical Mindsets on pedagogical approaches.

Since the 1990s, there has been a growing demand for evidence-based education policy and practice. This demand stems from concerns that education systems are not meeting the needs of a changing world and that education research lacks rigor. While this demand aims to improve the quality of education, silos between different actors often hinder how evidence informs policymaking. We encourage researchers to use a collaborative research approach by involving multiple education actors in the research process to close the gaps between research, policy, and practice. This paper is the third in a series of three working papers meant to serve as references and conversation starters for policymakers and researchers as they navigate pedagogical reform for education system transformation in their local contexts. Together, the three working papers emphasize the need for more locally driven collaborative research on how the interaction of culture, local education ecosystems, and learning theories--collectively called Invisible Pedagogical Mindsets--influences teachers' pedagogical choices in the classroom. Primarily intended for education researchers, Working Paper III advocates the use of collaborative research approaches to actively include multiple education actors in the research process, foster complementary relationships between actors with different expertise, and make research findings more relevant and responsive to the local education ecosystem. The paper has three parts that discuss the need for flexible research approaches to inform policy given the complexities of education decision-making, the importance of communication and dissemination, and how collaborative research can bridge the gaps between research, policy, and practice. The paper concludes by looking at the ongoing work of the SPARKS project at the Center for Universal Education and how collaborative research can contribute to education systems transformation.

Recent studies show that large language models (LLMs) struggle with technical standards in telecommunications. We propose a fine-tuned retrieval-augmented generation (RAG) system based on the Phi-2 small language model (SLM) to serve as an oracle for communication networks. Our developed system leverages forward-looking semantic chunking to adaptively determine parsing breakpoints based on embedding similarity, enabling effective processing of diverse document formats. To handle the challenge of multiple similar contexts in technical standards, we employ a re-ranking algorithm to prioritize the most relevant retrieved chunks. Recognizing the limitations of Phi-2's small context window, we implement a recent technique, namely SelfExtend, to expand the context window during inference, which not only boosts the performance but also can accommodate a wider range of user queries and design requirements from customers to specialized technicians. For fine-tuning, we utilize the low-rank adaptation (LoRA) technique to enhance computational efficiency during training and enable effective fine-tuning on small datasets. Our comprehensive experiments demonstrate substantial improvements over existing question-answering approaches in the telecom domain, achieving performance that exceeds larger language models such as GPT-4 (which is about 880 times larger in size). This work presents a novel approach to leveraging SLMs for communication networks, offering a balance of efficiency and performance. This work can serve as a foundation towards agentic language models for networks., Comment: submitted to Proc. IEEE Globecom

Prior works formulate the extraction of event-specific arguments as a span extraction problem, where event arguments are explicit -- i.e. assumed to be contiguous spans of text in a document. In this study, we revisit this definition of Event Extraction (EE) by introducing two key argument types that cannot be modeled by existing EE frameworks. First, implicit arguments are event arguments which are not explicitly mentioned in the text, but can be inferred through context. Second, scattered arguments are event arguments that are composed of information scattered throughout the text. These two argument types are crucial to elicit the full breadth of information required for proper event modeling. To support the extraction of explicit, implicit, and scattered arguments, we develop a novel dataset, DiscourseEE, which includes 7,464 argument annotations from online health discourse. Notably, 51.2% of the arguments are implicit, and 17.4% are scattered, making DiscourseEE a unique corpus for complex event extraction. Additionally, we formulate argument extraction as a text generation problem to facilitate the extraction of complex argument types. We provide a comprehensive evaluation of state-of-the-art models and highlight critical open challenges in generative event extraction. Our data and codebase are available at https://omar-sharif03.github.io/DiscourseEE., Comment: Accepted in EMNLP-2024 (Main). 21 pages, 8 figures, and 11 tables

Software applications are designed to assist users in conducting a wide range of tasks or interactions. They have become prevalent and play an integral part in people's lives in this digital era. To use those software applications, users are sometimes requested to provide their personal information. As privacy has become a significant concern and many data protection regulations exist worldwide, software applications must provide users with a privacy policy detailing how their personal information is collected and processed. We propose an approach that generates a comprehensive and compliant privacy policy with respect to the General Data Protection Regulation (GDPR) for diverse software applications. To support this, we first built a library of privacy clauses based on existing privacy policy analysis. We then developed an interactive rule-based system that prompts software developers with a series of questions and uses their answers to generate a customised privacy policy for a given software application. We evaluated privacy policies generated by our approach in terms of readability, completeness and coverage and compared them to privacy policies generated by three existing privacy policy generators and a Generative AI-based tool. Our evaluation results show that the privacy policy generated by our approach is the most complete and comprehensive.

Image-to-image translation is a topic in computer vision that has a vast range of use cases ranging from medical image translation, such as converting MRI scans to CT scans or to other MRI contrasts, to image colorization, super-resolution, domain adaptation, and generating photorealistic images from sketches or semantic maps. Image style transfer is also a widely researched application of image-to-image translation, where the goal is to synthesize an image that combines the content of one image with the style of another. Existing state-of-the-art methods often rely on complex neural networks, including diffusion models and language models, to achieve high-quality style transfer, but these methods can be computationally expensive and intricate to implement. In this paper, we propose a novel pixel shuffle method that addresses the image-to-image translation problem generally with a specific demonstrative application in style transfer. The proposed method approaches style transfer by shuffling the pixels of the style image such that the mutual information between the shuffled image and the content image is maximized. This approach inherently preserves the colors of the style image while ensuring that the structural details of the content image are retained in the stylized output. We demonstrate that this simple and straightforward method produces results that are comparable to state-of-the-art techniques, as measured by the Learned Perceptual Image Patch Similarity (LPIPS) loss for content preservation and the Fr\'echet Inception Distance (FID) score for style similarity. Our experiments validate that the proposed pixel shuffle method achieves competitive performance with significantly reduced complexity, offering a promising alternative for efficient image style transfer, as well as a promise in usability of the method in general image-to-image translation tasks.

Feature selection can greatly improve performance and interpretability in machine learning problems. However, existing nonparametric feature selection methods either lack theoretical error control or fail to accurately control errors in practice. Many methods are also slow, especially in high dimensions. In this paper, we introduce a general feature selection method that applies integrated path stability selection to thresholding to control false positives and the false discovery rate. The method also estimates q-values, which are better suited to high-dimensional data than p-values. We focus on two special cases of the general method based on gradient boosting (IPSSGB) and random forests (IPSSRF). Extensive simulations with RNA sequencing data show that IPSSGB and IPSSRF have better error control, detect more true positives, and are faster than existing methods. We also use both methods to detect microRNAs and genes related to ovarian cancer, finding that they make better predictions with fewer features than other methods.

Background: One of the key FDA-approved medications for Opioid Use Disorder (OUD) is buprenorphine. Despite its popularity, individuals often report various information needs regarding buprenorphine treatment on social media platforms like Reddit. However, the key challenge is to characterize these needs. In this study, we propose a theme-based framework to curate and analyze large-scale data from social media to characterize self-reported treatment information needs (TINs). Methods: We collected 15,253 posts from r/Suboxone, one of the largest Reddit sub-community for buprenorphine products. Following the standard protocol, we first identified and defined five main themes from the data and then coded 6,000 posts based on these themes, where one post can be labeled with applicable one to three themes. Finally, we determined the most frequently appearing sub-themes (topics) for each theme by analyzing samples from each group. Results: Among the 6,000 posts, 40.3% contained a single theme, 36% two themes, and 13.9% three themes. The most frequent topics for each theme or theme combination came with several key findings - prevalent reporting of psychological and physical effects during recovery, complexities in accessing buprenorphine, and significant information gaps regarding medication administration, tapering, and usage of substances during different stages of recovery. Moreover, self-treatment strategies and peer-driven advice reveal valuable insights and potential misconceptions. Conclusions: The findings obtained using our proposed framework can inform better patient education and patient-provider communication, design systematic interventions to address treatment-related misconceptions and rumors, and streamline the generation of hypotheses for future research.

Ultrarelativistic heavy-ion collisions produce a state of hot, dense, strongly interacting QCD matter known as quark-gluon plasma (QGP). On an event-by-event basis, the volume of the QGP in ultra-central collisions is mostly constant, while its total entropy can vary significantly, leading to variations in the temperature of the system. Exploiting this unique feature of ultra-central collisions allows us to interpret the correlation between the mean transverse momentum $(\langle p_{\mathrm{T}} \rangle)$ of produced charged hadrons and the number of charged hadrons as a measure of the the speed of sound, $c_{s}$. The speed of sound, $c_{s}$, which relates to the speed at which compression waves travel in a medium (in this case the QGP), is determined by fitting the relative increase of $\langle p_{\mathrm{T}} \rangle$ with respect to the relative change of the average charged-particle density $(\langle \mathrm{d}N_{\mathrm{ch}}/ \mathrm{d}\eta \rangle)$ measured at midrapidity. This study reports the $\langle p_{\mathrm{T}} \rangle$ of charged particles in ultra-central Pb-Pb collisions at a center-of-mass energy of 5.02 TeV per nucleon pair, using the ALICE detector. Different centrality estimators based on charged-particle multiplicity or the transverse energy of the event are employed to select ultra-central collisions. By ensuring a pseudorapidity gap between the region used to define the centrality and the region used for measurement, the influence of biases and their potential effects on the rise of the mean transverse momentum are tested. The measured $c_{s}^{2}$ is found to strongly depend on the type of the centrality estimator, ranging from $0.113 \pm 0.003 \, \mathrm{(stat)} \pm 0.007 \, \mathrm{(syst)}$ to $0.438 \pm 0.001 \, \mathrm{(stat)} \pm 0.019 \, \mathrm{(syst)}$ in natural units.

In this paper, a mathematical model is presented to infer the wave free surface elevation from the horizontal velocity components using Physics Informed Neural Network (PINN). PINN is a deep learning framework to solve forward and inverse Ordinary/Partial Differential Equations (ODEs/PDEs). The model is verified by measuring a numerically generated Kelvin waves downstream of a KRISO Container Ship (KCS). The KCS Kelvin waves are generated using two phase Volume of Fluid (VoF) Computational Fluid Dynamics (CFD) simulation with OpenFOAM. In addition, the paper presented the use of the Fourier Features decomposition of the Neural Network inputs to avoid the spectral bias phenomena; Spectral bias is the tendency of Neural Network to converge towards the low frequency solution faster than the high frequency one. Fourier Features decomposition layer showed an improvement for the model learning, as the model was able to learn the high and low frequency components simultaneously.

According to the WHO, in 2021, about 32% of pregnant women in sub-Saharan Africa were infected with malaria during pregnancy. Malaria infection during pregnancy can cause various adverse birth outcomes such as low birthweight. Over the past two decades, while some sub-Saharan African areas have experienced a large reduction in malaria prevalence due to improved malaria control and treatments, others have observed little change. Individual-level interventional studies have shown that preventing malaria infection during pregnancy can improve birth outcomes such as birthweight; however, it is still unclear whether natural reductions in malaria prevalence may help improve community-level birth outcomes. We conduct an observational study using 203,141 children's records in 18 sub-Saharan African countries from 2000 to 2018. Using heterogeneity of changes in malaria prevalence, we propose and apply a novel pair-of-pairs design via two-stage bipartite and non-bipartite matching to conduct a difference-in-differences study with a continuous measure of malaria prevalence, namely the Plasmodium falciparum parasite rate among children aged 2 to 10 ($\text{PfPR}_{2-10}$). The proposed novel statistical methodology allows us to apply difference-in-differences without dichotomizing $\text{PfPR}_{2-10}$, which can substantially increase the effective sample size, improve covariate balance, and facilitate the dose-response relationship during analysis. Our outcome analysis finds that among the pairs of clusters we study, the largest reduction in $\text{PfPR}_{2-10}$ over early and late years is estimated to increase the average birthweight by 98.899 grams (95% CI: $[39.002, 158.796]$), which is associated with reduced risks of several adverse birth or life-course outcomes. The proposed novel statistical methodology can be replicated in many other disease areas.

Multi-distribution or collaborative learning involves learning a single predictor that works well across multiple data distributions, using samples from each during training. Recent research on multi-distribution learning, focusing on binary loss and finite VC dimension classes, has shown near-optimal sample complexity that is achieved with oracle efficient algorithms. That is, these algorithms are computationally efficient given an efficient ERM for the class. Unlike in classical PAC learning, where the optimal sample complexity is achieved with deterministic predictors, current multi-distribution learning algorithms output randomized predictors. This raises the question: can these algorithms be derandomized to produce a deterministic predictor for multiple distributions? Through a reduction to discrepancy minimization, we show that derandomizing multi-distribution learning is computationally hard, even when ERM is computationally efficient. On the positive side, we identify a structural condition enabling an efficient black-box reduction, converting existing randomized multi-distribution predictors into deterministic ones.

We present MathDSL, a Domain-Specific Language (DSL) for mathematical equation solving, which, when deployed in program synthesis models, outperforms state-of-the-art reinforcement-learning-based methods. We also introduce a quantitative metric for measuring the conciseness of a mathematical solution and demonstrate the improvement in the quality of generated solutions compared to other methods. Our system demonstrates that a program synthesis system (DreamCoder) using MathDSL can generate programs that solve linear equations with greater accuracy and conciseness than using reinforcement learning systems. Additionally, we demonstrate that if we use the action spaces of previous reinforcement learning systems as DSLs, MathDSL outperforms the action-space-DSLs. We use DreamCoder to store equation-solving strategies as learned abstractions in its program library and demonstrate that by using MathDSL, these can be converted into human-interpretable solution strategies that could have applications in mathematical education.

Understanding the origins of massive black hole seeds and their co-evolution with their host galaxy requires studying intermediate-mass black holes (IMBHs) and estimating their mass. However, measuring the mass of these IMBHs is challenging due to the high spatial resolution requirement. A spectrophotometric reverberation monitoring is performed for a low-luminosity Seyfert 1 galaxy NGC 4395 to measure the size of the broad line region (BLR) and black hole mass. The data were collected using the 1.3-m Devasthal fast optical telescope (DFOT) and 3.6-m Devasthal optical telescope (DOT) at ARIES, Nainital, over two consecutive days in March 2022. The analysis revealed strong emission lines in the spectra and light curves of merged 5100{\AA} spectroscopic continuum flux ($f_{\mathrm{5100}}$) with photometric continuum V-band and H$\alpha$, with fractional variabilities of 6.38\% and 6.31\% respectively. In comparison to several previous studies with lag estimation $<$ 90 minutes, our calculated H$\alpha$ lag supersedes by $125.0^{+6.2}_{-6.1}$ minutes using ICCF and {\small JAVELIN} methods. The velocity dispersion ($\sigma_{\mathrm{line}}$) of the broad line clouds is measured to be $544.7^{+22.4}_{-25.1}$ km s$^{-1}$, yielding a black hole mass of $\sim$ $2.2^{+0.2}_{-0.2}\times 10^{4}M_{\mathrm{\odot}}$ and an Eddington ratio of 0.06., Comment: 17 pages, 8 figures, accepted for publication in ApJ

Despite the recent broad adoption of Large Language Models (LLMs) across various domains, their potential for enriching information systems in extracting and exploring Linked Data (LD) and Resource Description Framework (RDF) triplestores has not been extensively explored. This paper examines the integration of LLMs within existing systems, emphasising the enhancement of conversational user interfaces (UIs) and their capabilities for data extraction by producing more accurate SPARQL queries without the requirement for model retraining. Typically, conversational UI models necessitate retraining with the introduction of new datasets or updates, limiting their functionality as general-purpose extraction tools. Our approach addresses this limitation by incorporating LLMs into the conversational UI workflow, significantly enhancing their ability to comprehend and process user queries effectively. By leveraging the advanced natural language understanding capabilities of LLMs, our method improves RDF entity extraction within web systems employing conventional chatbots. This integration facilitates a more nuanced and context-aware interaction model, critical for handling the complex query patterns often encountered in RDF datasets and Linked Open Data (LOD) endpoints. The evaluation of this methodology shows a marked enhancement in system expressivity and the accuracy of responses to user queries, indicating a promising direction for future research in this area. This investigation not only underscores the versatility of LLMs in enhancing existing information systems but also sets the stage for further explorations into their potential applications within more specialised domains of web information systems., Comment: This paper has been accepted at the 25th International Web Information Systems Engineering Conference (WISE 2024)

Data privacy and authentication are two main security requirements for remote access and cloud services. While QKD has been explored to address data privacy concerns, oftentimes its use is separate from the client authentication protocol despite implicitly providing authentication. Here, we present a quantum authentication key expansion (QAKE) protocol that (1) integrates both authentication and key expansion within a single protocol, and (2) provides key recycling property -- allowing all authentication keys to be reused. We analyse the security of the protocol in a QAKE framework adapted from a classical authentication key exchange (AKE) framework, providing separate security conditions for authentication and data privacy. An experimental implementation of the protocol, with appropriate post-selection, was performed to demonstrate its feasibility., Comment: 71 pages, comments are welcome

This paper presents surface acoustic wave (SAW) acoustic delay lines (ADL) for studying propagation loss mechanisms in Lithium Niobate (LN). Devices were fabricated by depositing 50 nm aluminum patterns on 600 nm X-Cut LN on amorphous silicon on silicon carbide, where longitudinally dominant SAW was targeted. Upon fabrication, higher-order thickness-based cross-sectional Lam\'e modes and Rayleigh modes were studied for their Q factors using acoustic delay lines. Utilizing bi-directional electrodes, ADL with lateral lambda values ranging from 0.4 um to 0.6 um were measured. Higher order Lame modes were found to have consistently higher Q factors than their Rayleigh mode counterpart, on the order of 1000-3000, showing high-frequency SAW devices as still viable candidates for frequency scaling without a substantial increase in loss., Comment: 4 pages, 7 figures, accepted by IEEE UFFC-JS

Cosserat theory of elasticity is a generalization of classical elasticity that allows for asymmetry in the stress tensor by taking into account micropolar rotations in the medium. The equations involve a rotation field and associated "couple stress" as variables, in addition to the conventional displacement and Cauchy stress fields. In recent work, we derived a mixed finite element method (MFEM) for the linear Cosserat equations that converges optimally in these four variables. The drawback of this method is that it retains the stresses as unknowns, and therefore leads to relatively large saddle point system that are computationally demanding to solve. As an alternative, we developed a finite volume method in which the stress variables are approximated using a minimal, two-point stencil (TPSA). The system consists of the displacement and rotation variables, with an additional "solid pressure" unknown. Both the MFEM and TPSA methods are robust in the incompressible limit and in the Cauchy limit, for which the Cosserat equations degenerate to classical linearized elasticity. We report on the construction of the methods, their a priori properties, and compare their numerical performance against an MPSA finite volume method., Comment: European conference on the mathematics of geological reservoirs (ECMOR), 2024, Oslo

We present an exploratory study of the possibilities of the Deep Ritz Method (DRM) for the modeling of strain localization in solids as a sharp discontinuity in the displacement field. For this, we use a regularized strong discontinuity kinematics within a variational setting for elastoplastic solids. The corresponding mathematical model is discretized using Artificial Neural Networks (ANNs). The architecture takes care of the kinematics, while the variational statement of the boundary value problem is taken care of by the loss function. The main idea behind this approach is to solve both the equilibrium problem and the location of the localization band by means of trainable parameters in the ANN. As a proof of concept, we show through both 1D and 2D numerical examples that the computational modeling of strain localization for elastoplastic solids within the framework of DRM is feasible., Comment: The article has 22 pages including 14 figures and 26 references. The manuscript was prepared for submission to Archives of Computational Methods in Engineering

The translation of imaging Mueller polarimetry to clinical practice is often hindered by large footprint and relatively slow acquisition speed of the existing instruments. Using polarization-sensitive camera as a detector may reduce instrument dimensions and allow data streaming at video rate. However, only the first three rows of a complete 4x4 Mueller matrix can be measured. To overcome this hurdle we developed a machine learning approach using sequential neural network algorithm for the reconstruction of missing elements of a Mueller matrix from the measured elements of the first three rows. The algorithm was trained and tested on the dataset of polarimetric images of various excised human tissues (uterine cervix, colon, skin, brain) acquired with two different imaging Mueller polarimeters operating in either reflection (wide-field imaging system) or transmission (microscope) configurations at different wavelengths of 550 nm and 385 nm, respectively. The reconstruction performance was evaluated using various error metrics, all of which confirmed low error values. The execution time of the trained neural network algorithm was about 300 microseconds for a single image pixel. It suggests that a machine learning approach with parallel processing of all image pixels combined with the partial Mueller polarimeter operating at video rate can effectively substitute for the complete Mueller polarimeter and produce accurate maps of depolarization, linear retardance and orientation of the optical axis of biological tissues, which can be used for medical diagnosis in clinical settings.

In order to solve problems of practical importance, quantum computers will likely need to incorporate quantum error correction, where a logical qubit is redundantly encoded in many noisy physical qubits. The large physical-qubit overhead typically associated with error correction motivates the search for more hardware-efficient approaches. Here, using a microfabricated superconducting quantum circuit, we realize a logical qubit memory formed from the concatenation of encoded bosonic cat qubits with an outer repetition code of distance $d=5$. The bosonic cat qubits are passively protected against bit flips using a stabilizing circuit. Cat-qubit phase-flip errors are corrected by the repetition code which uses ancilla transmons for syndrome measurement. We realize a noise-biased CX gate which ensures bit-flip error suppression is maintained during error correction. We study the performance and scaling of the logical qubit memory, finding that the phase-flip correcting repetition code operates below threshold, with logical phase-flip error decreasing with code distance from $d=3$ to $d=5$. Concurrently, the logical bit-flip error is suppressed with increasing cat-qubit mean photon number. The minimum measured logical error per cycle is on average $1.75(2)\%$ for the distance-3 code sections, and $1.65(3)\%$ for the longer distance-5 code, demonstrating the effectiveness of bit-flip error suppression throughout the error correction cycle. These results, where the intrinsic error suppression of the bosonic encodings allows us to use a hardware-efficient outer error correcting code, indicate that concatenated bosonic codes are a compelling paradigm for reaching fault-tolerant quantum computation., Comment: Comments on the manuscript welcome!

Classifying subsets based on spatial and temporal features is crucial to the analysis of spatiotemporal data given the inherent spatial and temporal variability. Since no single clustering algorithm ensures optimal results, researchers have increasingly explored the effectiveness of ensemble approaches. Ensemble clustering has attracted much attention due to increased diversity, better generalization, and overall improved clustering performance. While ensemble clustering may yield promising results on simple datasets, it has not been fully explored on complex multivariate spatiotemporal data. For our contribution to this field, we propose a novel hybrid ensemble deep graph temporal clustering (HEDGTC) method for multivariate spatiotemporal data. HEDGTC integrates homogeneous and heterogeneous ensemble methods and adopts a dual consensus approach to address noise and misclassification from traditional clustering. It further applies a graph attention autoencoder network to improve clustering performance and stability. When evaluated on three real-world multivariate spatiotemporal data, HEDGTC outperforms state-of-the-art ensemble clustering models by showing improved performance and stability with consistent results. This indicates that HEDGTC can effectively capture implicit temporal patterns in complex spatiotemporal data., Comment: 10 pages

With the ever-growing complexity of models in the field of remote sensing (RS), there is an increasing demand for solutions that balance model accuracy with computational efficiency. Knowledge distillation (KD) has emerged as a powerful tool to meet this need, enabling the transfer of knowledge from large, complex models to smaller, more efficient ones without significant loss in performance. This review article provides an extensive examination of KD and its innovative applications in RS. KD, a technique developed to transfer knowledge from a complex, often cumbersome model (teacher) to a more compact and efficient model (student), has seen significant evolution and application across various domains. Initially, we introduce the fundamental concepts and historical progression of KD methods. The advantages of employing KD are highlighted, particularly in terms of model compression, enhanced computational efficiency, and improved performance, which are pivotal for practical deployments in RS scenarios. The article provides a comprehensive taxonomy of KD techniques, where each category is critically analyzed to demonstrate the breadth and depth of the alternative options, and illustrates specific case studies that showcase the practical implementation of KD methods in RS tasks, such as instance segmentation and object detection. Further, the review discusses the challenges and limitations of KD in RS, including practical constraints and prospective future directions, providing a comprehensive overview for researchers and practitioners in the field of RS. Through this organization, the paper not only elucidates the current state of research in KD but also sets the stage for future research opportunities, thereby contributing significantly to both academic research and real-world applications., Comment: 50 pages, 11 figures and 9 tables

Non-stationarity poses a fundamental challenge in Multi-Agent Reinforcement Learning (MARL), arising from agents simultaneously learning and altering their policies. This creates a non-stationary environment from the perspective of each individual agent, often leading to suboptimal or even unconverged learning outcomes. We propose an open-source framework named XP-MARL, which augments MARL with auxiliary prioritization to address this challenge in cooperative settings. XP-MARL is 1) founded upon our hypothesis that prioritizing agents and letting higher-priority agents establish their actions first would stabilize the learning process and thus mitigate non-stationarity and 2) enabled by our proposed mechanism called action propagation, where higher-priority agents act first and communicate their actions, providing a more stationary environment for others. Moreover, instead of using a predefined or heuristic priority assignment, XP-MARL learns priority-assignment policies with an auxiliary MARL problem, leading to a joint learning scheme. Experiments in a motion-planning scenario involving Connected and Automated Vehicles (CAVs) demonstrate that XP-MARL improves the safety of a baseline model by 84.4% and outperforms a state-of-the-art approach, which improves the baseline by only 12.8%. Code: github.com/cas-lab-munich/sigmarl, Comment: 7 pages, 5 figures. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Motivated by structural properties of differential field extensions, we introduce the notion of a theory $T$ being derivation-like with respect to another model-complete theory $T_0$. We prove that when $T$ admits a model-companion $T_+$, then several model-theoretic properties transfer from $T_0$ to $T_+$. These properties include completeness, quantifier-elimination, stability, simplicity, and NSOP$_1$. We also observe that, aside from the theory of differential fields, examples of derivation-like theories are plentiful.

Comorbid anxiety disorders are common among patients with major depressive disorder (MDD), and numerous studies have identified an association between comorbid anxiety and resistance to pharmacological depression treatment. However, less is known regarding the effect of anxiety on non-pharmacological therapies for MDD. We apply machine learning techniques to analyze MDD treatment responses in a large-scale clinical trial (n=754), in which participants with MDD were recruited online and randomized to different smartphone-based depression treatments. We find that a baseline GAD-7 questionnaire score in the "moderate" to "severe" range (>10) predicts greatly reduced probability of responding to treatment across treatment groups. Our findings suggest that depressed individuals with comorbid anxiety face lower odds of substantial improvement in the context of smartphone-based therapeutic interventions for MDD. Our work highlights a simple methodology for identifying clinically useful "rules of thumb" in treatment response prediction using interpretable machine learning models and a forward variable selection process.

We study a linear contextual optimization problem where a decision maker has access to historical data and contextual features to learn a cost prediction model aimed at minimizing decision error. We adopt the predict-then-optimize framework for this analysis. Given that perfect model alignment with reality is often unrealistic in practice, we focus on scenarios where the chosen hypothesis set is misspecified. In this context, it remains unclear whether current contextual optimization approaches can effectively address such model misspecification. In this paper, we present a novel integrated learning and optimization approach designed to tackle model misspecification in contextual optimization. This approach offers theoretical generalizability, tractability, and optimality guarantees, along with strong practical performance. Our method involves minimizing a tractable surrogate loss that aligns with the performance value from cost vector predictions, regardless of whether the model is misspecified, and can be optimized in reasonable time. To our knowledge, no previous work has provided an approach with such guarantees in the context of model misspecification.

We present an overview of Sherpa, an open source Python project, and discuss its development history, broad design concepts and capabilities. Sherpa contains powerful tools for combining parametric models into complex expressions that can be fit to data using a variety of statistics and optimization methods. It is easily extensible to include user-defined models, statistics, and optimization methods. It provides a high-level User Interface for interactive data-analysis, such as within a Jupyter notebook, and it can also be used as a library component, providing fitting and modeling capabilities to an application. We include a few examples of Sherpa applications to multiwavelength astronomical data. The code is available GitHub: https://github.com/sherpa/sherpa, Comment: accepted by the Astrophysical Journal Supplement Series, 12 pages, 11 figures

We present a novel method for symbolic regression (SR), the task of searching for compact programmatic hypotheses that best explain a dataset. The problem is commonly solved using genetic algorithms; we show that we can enhance such methods by inducing a library of abstract textual concepts. Our algorithm, called LaSR, uses zero-shot queries to a large language model (LLM) to discover and evolve concepts occurring in known high-performing hypotheses. We discover new hypotheses using a mix of standard evolutionary steps and LLM-guided steps (obtained through zero-shot LLM queries) conditioned on discovered concepts. Once discovered, hypotheses are used in a new round of concept abstraction and evolution. We validate LaSR on the Feynman equations, a popular SR benchmark, as well as a set of synthetic tasks. On these benchmarks, LaSR substantially outperforms a variety of state-of-the-art SR approaches based on deep learning and evolutionary algorithms. Moreover, we show that LaSR can be used to discover a novel and powerful scaling law for LLMs., Comment: preprint version; 10 pages

Multicomponent phase separation is a routine occurrence in both living and synthetic systems. Thermodynamics provides a straightforward path to determine the phase boundaries that characterize these transitions for systems at equilibrium. The prevalence of phase separation in complex systems outside the confines of equilibrium motivates the need for a genuinely nonequilibrium theory of multicomponent phase coexistence. Here, we develop a mechanical theory for coexistence that casts coexistence criteria into the familiar form of equality of state functions. Our theory generalizes traditional equilibrium notions such as the species chemical potential and thermodynamic pressure to systems out of equilibrium. Crucially, while these notions may not be identifiable for all nonequilibrium systems, we numerically verify their existence for a variety of systems by introducing the phenomenological Multicomponent Active Model B+. Our work establishes an initial framework for understanding multicomponent coexistence that we hope can serve as the basis for a comprehensive theory for high-dimensional nonequilibrium phase transitions.

This paper introduces generative Residual Networks (ResNet) as a surrogate Machine Learning (ML) tool for Large Eddy Simulation (LES) Sub Grid Scale (SGS) resolving. The study investigates the impact of incorporating Dual Scale Residual Blocks (DS-RB) within the ResNet architecture. Two LES SGS resolving models are proposed and tested for prior analysis test cases: a super-resolution model (SR-ResNet) and a SGS stress tensor inference model (SGS-ResNet). The SR-ResNet model task is to upscale LES solutions from coarse to finer grids by inferring unresolved SGS velocity fluctuations, exhibiting success in preserving high-frequency velocity fluctuation information, and aligning with higher-resolution LES solutions' energy spectrum. Furthermore, employing DS-RB enhances prediction accuracy and precision of high-frequency velocity fields compared to Single Scale Residual Blocks (SS-RB), evident in both spatial and spectral domains. The SR-ResNet model is tested and trained on filtered/downsampled 2-D LES planar jet injection problems at two Reynolds numbers, two jet configurations, and two upscale ratios. In the case of SGS stress tensor inference, both SS-RB and DS-RB exhibit higher prediction accuracy over the Smagorinsky model with reference to the true DNS SGS stress tensor, with DS-RB-based SGS-ResNet showing stronger statistical alignment with DNS data. The SGS-ResNet model is tested on a filtered/downsampled 2-D DNS isotropic homogenous decay turbulence problem. The adoption of DS-RB incurs notable increases in network size, training time, and forward inference time, with the network size expanding by over tenfold, and training and forward inference times increasing by approximately 0.5 and 3 times, respectively.

The rising use of Large Language Models (LLMs) to create and disseminate malware poses a significant cybersecurity challenge due to their ability to generate and distribute attacks with ease. A single prompt can initiate a wide array of malicious activities. This paper addresses this critical issue through a multifaceted approach. First, we provide a comprehensive overview of LLMs and their role in malware detection from diverse sources. We examine five specific applications of LLMs: Malware honeypots, identification of text-based threats, code analysis for detecting malicious intent, trend analysis of malware, and detection of non-standard disguised malware. Our review includes a detailed analysis of the existing literature and establishes guiding principles for the secure use of LLMs. We also introduce a classification scheme to categorize the relevant literature. Second, we propose performance metrics to assess the effectiveness of LLMs in these contexts. Third, we present a risk mitigation framework designed to prevent malware by leveraging LLMs. Finally, we evaluate the performance of our proposed risk mitigation strategies against various factors and demonstrate their effectiveness in countering LLM-enabled malware. The paper concludes by suggesting future advancements and areas requiring deeper exploration in this fascinating field of artificial intelligence., Comment: 26 pages, 7 figures, 4 tables

This paper presents a new data assimilation (DA) scheme based on a sequential Markov Chain Monte Carlo (SMCMC) DA technique [Ruzayqat et al. 2024] which is provably convergent and has been recently used for filtering, particularly for high-dimensional non-linear, and potentially, non-Gaussian state-space models. Unlike particle filters, which can be considered exact methods and can be used for filtering non-linear, non-Gaussian models, SMCMC does not assign weights to the samples/particles, and therefore, the method does not suffer from the issue of weight-degeneracy when a relatively small number of samples is used. We design a localization approach within the SMCMC framework that focuses on regions where observations are located and restricts the transition densities included in the filtering distribution of the state to these regions. This results in immensely reducing the effective degrees of freedom and thus improving the efficiency. We test the new technique on high-dimensional ($d \sim 10^4 - 10^5$) linear Gaussian model and non-linear shallow water models with Gaussian noise with real and synthetic observations. For two of the numerical examples, the observations mimic the data generated by the Surface Water and Ocean Topography (SWOT) mission led by NASA, which is a swath of ocean height observations that changes location at every assimilation time step. We also use a set of ocean drifters' real observations in which the drifters are moving according the ocean kinematics and assumed to have uncertain locations at the time of assimilation. We show that when higher accuracy is required, the proposed algorithm is superior in terms of efficiency and accuracy over competing ensemble methods and the original SMCMC filter., Comment: 24 pages, 8 figures

Metasurfaces are key to the development of flat optics and nanophotonic devices, offering significant advantages in creating structural colors and high-quality factor cavities. Multi-layer metasurfaces (MLMs) further amplify these benefits by enhancing light-matter interactions within individual nanopillars. However, the numerous design parameters involved make traditional simulation tools impractical and time-consuming for optimizing MLMs. This highlights the need for more efficient approaches to accelerate their design. In this work, we introduce NanoPhotoNet, an AI-driven design tool based on a hybrid deep neural network (DNN) model that combines convolutional neural networks (CNN) and Long Short-Term Memory (LSTM) networks. NanoPhotoNet enhances the design and optimization of MLMs, achieving a prediction accuracy of over 98.3% and a speed improvement of 50,000x compared to conventional methods. The tool enables MLMs to produce structural colors beyond the standard RGB region, expanding the RGB gamut area by 163%. Furthermore, we demonstrate the generation of tunable structural colors, extending the metasurface functionality to tunable color filters. These findings present a powerful method for applying NanoPhotoNet to MLMs, enabling strong light-matter interactions in applications such as tunable nanolasers and reconfigurable beam steering.

This paper proposes an approach for controlling surgical robotic systems, while complying with the Remote Center of Motion (RCM) constraint in Robot-Assisted Minimally Invasive Surgery (RA-MIS). In this approach, the RCM-constraint is upheld algorithmically, providing flexibility in the positioning of the insertion point and enabling compatibility with a wide range of general-purpose robots. The paper further investigates the impact of the tool's insertion ratio on the RCM-error, and introduces a manipulability index of the robot which considers the RCM-error that it is used to find a starting configuration. To accurately evaluate the proposed method's trajectory tracking within an RCM-constrained environment, an electromagnetic tracking system is employed. The results demonstrate the effectiveness of the proposed method in addressing the RCM constraint problem in RA-MIS., Comment: 6 pages, 7 figures. Published in the 21st International Conference on Advanced Robotics (ICAR 2023)

Estimating the fidelity between a desired target quantum state and an actual prepared state is essential for assessing the success of experiments. For pure target states, we use functional representations that can be measured directly and determine the number of copies of the prepared state needed for fidelity estimation. In continuous variable (CV) systems, we utilise the Wigner function, which can be measured via displaced parity measurements. We provide upper and lower bounds on the sample complexity required for fidelity estimation, considering the worst-case scenario across all possible prepared states. For target states of particular interest, such as Fock and Gaussian states, we find that this sample complexity is characterised by the $L^1$-norm of the Wigner function, a measure of Wigner negativity widely studied in the literature, in particular in resource theories of quantum computation. For discrete variable systems consisting of $n$ qubits, we explore fidelity estimation protocols using Pauli string measurements. Similarly to the CV approach, the sample complexity is shown to be characterised by the $L^1$-norm of the characteristic function of the target state for both Haar random states and stabiliser states. Furthermore, in a general black box model, we prove that, for any target state, the optimal sample complexity for fidelity estimation is characterised by the smoothed $L^1$-norm of the target state. To the best of our knowledge, this is the first time the $L^1$-norm of the Wigner function provides a lower bound on the cost of some information processing task., Comment: 40 pages main text, 4 pages of appendices

We initiate the study of relativistic zero-knowledge quantum proof of knowledge systems with classical communication, formally defining a number of useful concepts and constructing appropriate knowledge extractors for all the existing protocols in the relativistic setting which satisfy a weaker variant of the special soundness property due to Unruh (EUROCRYPT 2012). We show that there exists quantum proofs of knowledge with knowledge error 1/2 + negl({\eta}) for all relations in NP via a construction of such a system for the Hamiltonian cycle relation using a general relativistic commitment scheme exhibiting the fairly-binding property due to Fehr and Fillinger (EUROCRYPT 2016). We further show that one can construct quantum proof of knowledge extractors for proof systems which do not exhibit special soundness, and therefore require an extractor to rewind multiple times. We develop a new multi-prover quantum rewinding technique by combining ideas from monogamy of entanglement and gentle measurement lemmas that can break the quantum rewinding barrier. Finally, we prove a new bound on the impact of consecutive measurements and use it to significantly improve the soundness bound of some existing relativistic zero knowledge proof systems, such as the one due to Chailloux and Leverrier (EUROCRYPT 2017)., Comment: 37 pages

This paper proposes Proteus, a protocol state machine, property-guided, and budget-aware automated testing approach for discovering logical vulnerabilities in wireless protocol implementations. Proteus maintains its budget awareness by generating test cases (i.e., each being a sequence of protocol messages) that are not only meaningful (i.e., the test case mostly follows the desirable protocol flow except for some controlled deviations) but also have a high probability of violating the desirable properties. To demonstrate its effectiveness, we evaluated Proteus in two different protocol implementations, namely 4G LTE and BLE, across 23 consumer devices (11 for 4G LTE and 12 for BLE). Proteus discovered 25 unique issues, including 112 instances. Affected vendors have positively acknowledged 14 vulnerabilities through 5 CVEs., Comment: Accepted to ACM CCS 2024

We propose hydrostatic pressure -- a well-established tool for tuning properties of condensed matter -- as a novel route for optimizing targets for light dark matter direct detection, specifically via phonons. Pressure dramatically affects compressible solids by boosting the speed of sound and phonon frequencies. Focusing on helium -- the most compressible solid -- our ab initio calculations illustrate how high pressure elevates helium from lacking single-phonon reach to rivaling leading candidates. Our work establishes pressure as an unexplored tuning knob for accessing lower dark matter mass regimes., Comment: 8 pages, 3 figures, plus supplemental. Supplemental materials PDF available with source files