Search

Despite well-designed curriculum materials, teachers often face challenges in their implementation due to diverse classroom needs. This paper investigates whether Large Language Models (LLMs) can support middle-school math teachers by helping create high-quality curriculum scaffolds, which we define as the adaptations and supplements teachers employ to ensure all students can access and engage with the curriculum. Through Cognitive Task Analysis with expert teachers, we identify a three-stage process for curriculum scaffolding: observation, strategy formulation, and implementation. We incorporate these insights into three LLM approaches to create warmup tasks that activate background knowledge. The best-performing approach, which provides the model with the original curriculum materials and an expert-informed prompt, generates warmups that are rated significantly higher than warmups created by expert teachers in terms of alignment to learning objectives, accessibility to students working below grade level, and teacher preference. This research demonstrates the potential of LLMs to support teachers in creating effective scaffolds and provides a methodology for developing AI-driven educational tools. [This report was funded by Stanford's Center for Human-Centered AI and the Stanford Accelerator for Learning.]

The ground-state spin of optically active defects in hexagonal boron nitride (hBN) offers a promising platform for quantum information applications, such as qubits for quantum computing and nanoscale sensing. A key characteristic of a qubit is its decoherence time, as its duration and controllability are critical for practical applications in quantum technologies. In this work, we investigate the electron spin decoherence time of the negatively charged boron vacancies, $V_{B}$ centers, in the hBN lattice by considering the dipolar hyperfine as well as spin-phonon interactions. We employ an approximate method based on the Holstein-Primakoff transformation to take into account a large number of nuclear spins and Debye model to consider the effect of lattice phonons. We show that, in the presence of the dipolar hyperfine interactions, Hahn-echo coherence time of the $V_{B}$ electron spin is approximately $30\: \mathrm{\mu s}$ at room temperature, close to the previously reported results. Our findings suggest that the major source of decoherence is the noise caused by nuclear spins in the lattice. Our results provide a step forward in understanding the $V_{B}$ defect decoherence in the hBN, which might be used for quantum information applications.

Hate speech and abusive language are global phenomena that need socio-cultural background knowledge to be understood, identified, and moderated. However, in many regions of the Global South, there have been several documented occurrences of (1) absence of moderation and (2) censorship due to the reliance on keyword spotting out of context. Further, high-profile individuals have frequently been at the center of the moderation process, while large and targeted hate speech campaigns against minorities have been overlooked. These limitations are mainly due to the lack of high-quality data in the local languages and the failure to include local communities in the collection, annotation, and moderation processes. To address this issue, we present AfriHate: a multilingual collection of hate speech and abusive language datasets in 15 African languages. Each instance in AfriHate is annotated by native speakers familiar with the local culture. We report the challenges related to the construction of the datasets and present various classification baseline results with and without using LLMs. The datasets, individual annotations, and hate speech and offensive language lexicons are available on https://github.com/AfriHate/AfriHate

Long-context LLMs have enabled numerous downstream applications but also introduced significant challenges related to computational and memory efficiency. To address these challenges, optimizations for long-context inference have been developed, centered around the KV cache. However, existing benchmarks often evaluate in single-request, neglecting the full lifecycle of the KV cache in real-world use. This oversight is particularly critical, as KV cache reuse has become widely adopted in LLMs inference frameworks, such as vLLM and SGLang, as well as by LLM providers, including OpenAI, Microsoft, Google, and Anthropic. To address this gap, we introduce SCBench(SharedContextBench), a comprehensive benchmark for evaluating long-context methods from a KV cachecentric perspective: 1) KV cache generation, 2) KV cache compression, 3) KV cache retrieval, 4) KV cache loading. Specifically, SCBench uses test examples with shared context, ranging 12 tasks with two shared context modes, covering four categories of long-context capabilities: string retrieval, semantic retrieval, global information, and multi-task. With it, we provide an extensive KV cache-centric analysis of eight categories long-context solutions, including Gated Linear RNNs, Mamba-Attention hybrids, and efficient methods such as sparse attention, KV cache dropping, quantization, retrieval, loading, and prompt compression. The evaluation is conducted on 8 long-context LLMs. Our findings show that sub-O(n) memory methods suffer in multi-turn scenarios, while sparse encoding with O(n) memory and sub-O(n^2) pre-filling computation perform robustly. Dynamic sparsity yields more expressive KV caches than static patterns, and layer-level sparsity in hybrid architectures reduces memory usage with strong performance. Additionally, we identify attention distribution shift issues in long-generation scenarios. https://aka.ms/SCBench.

Semantic Edge Computing (SEC) and Semantic Communications (SemComs) have been proposed as viable approaches to achieve real-time edge-enabled intelligence in sixth-generation (6G) wireless networks. On one hand, SemCom leverages the strength of Deep Neural Networks (DNNs) to encode and communicate the semantic information only, while making it robust to channel distortions by compensating for wireless effects. Ultimately, this leads to an improvement in the communication efficiency. On the other hand, SEC has leveraged distributed DNNs to divide the computation of a DNN across different devices based on their computational and networking constraints. Although significant progress has been made in both fields, the literature lacks a systematic view to connect both fields. In this work, we fulfill the current gap by unifying the SEC and SemCom fields. We summarize the research problems in these two fields and provide a comprehensive review of the state of the art with a focus on their technical strengths and challenges., Comment: Submitted to ACM Computing Surveys (CSUR)

Given a connected graph $G=(V,E)$ and a crossing family $\mathcal{C}$ over ground set $V$ such that $|\delta_G(U)|\geq 2$ for every $U\in \mathcal{C}$, we prove there exists a strong orientation of $G$ for $\mathcal{C}$, i.e., an orientation of $G$ such that each set in $\mathcal{C}$ has at least one outgoing and at least one incoming arc. This implies the main conjecture in Chudnovsky et al. (Disjoint dijoins. Journal of Combinatorial Theory, Series B, 120:18--35, 2016). In particular, in every minimal counterexample to the Edmonds-Giles conjecture where the minimum weight of a dicut is $2$, the arcs of nonzero weight must be disconnected., Comment: 10 pages, 2 figures

This paper presents a multi-level approximate computing approach for real-time traffic sign recognition in autonomous vehicles called AppSign. Since autonomous vehicles are real-time systems, they must gather environmental information and process them instantaneously to respond properly. However, due to the limited resources of these systems, executing computation-intensive algorithms such as deep-learning schemes that lead to precise output is impossible and takes a long time. To tackle this, imprecise computation schemes compromise the complexity and real-time operations. In this context, AppSign presents a multi-level approximate computing scheme to balance the accuracy and computation cost of the computation-intensive schemes and make them appropriate for real-time applications. AppSign is applied to the CNN-based traffic sign recognition unit by approximating the convolution operation of CNN which is the primal solution for image processing applications. In AppSign a novel approximate multiplication method called "TIRuD" is proposed that truncates the operations while keeping the accuracy acceptable. Moreover, it provides the adaptive approximation of the underlying CNN by involving various levels of computation and considering different approximation methods. The efficiency of the proposed AppSign, in real-time traffic sign recognition, is evaluated through several experiments. Based on these experiments, our proposed TIRuD reduces the accuracy by about $10\%$ while saving execution time about $64\%$ over the exact multiplication, averagely. Moreover, employing our proposed hierarchical approximation in various model layers outperforms the exact computation $27.78\%$ considering "AoC" that joins accuracy and computation cost in a parameter.

We develop a method for entangling operations on nuclear spins surrounding a negatively charged boron vacancy (VB-center) point defect in hexagonal boron nitride (hBN). To this end, we propose to employ the electron spin of a VB-center as a control qubit. We show that in the presence of a background magnetic field and by applying control pulses one can collectively manipulate the state of the nuclei with $\hat{U}_z$ and $\hat{U}_x$ rotations. These rotations can serve for implementing the synchronous three-qubit $X$, $Z$, and the Hadamard gates. Through our numerical analyses considering realistic system parameters and the decoherence effects, we demonstrate that these gates can be executed with high fidelities. Furthermore, as an example for the application of our toolbox, we utilize these collective gates to prepare the highly entangled GHZ states among the three nuclear spins with a fidelity of $0.99$. By including the electron decoherence effects we find that the relative deviations of the gate fidelities from the noisy terms are negligibly small, proving the noise-resilience of our protocols. Our work can serve as the groundstone for exploiting the nuclear spins in hBN in future quantum technological applications., Comment: 14 pages, 5 figures

The need for higher power density in electrical machines require better cooling strategies. Spray cooling is a very promising and relatively simple technology to apply, but involves extremely complicated physics. In this paper, a quasi-3D thermal finite-element model of a stator winding is created, by extrusion of a 2D cross-sectional finite-element model along the winding direction. The possible effects of spray cooling are simulated as a heat flux using an impedance boundary condition at the surface of the winding overhang. The results confirm the beneficial performance of spray cooling. The model indicates that spray cooling may allow a ten times larger power density than for standard air- or water-cooled machines.

Let $D=(V,A)$ be a digraph whose underlying graph is $2$-edge-connected, and let $P$ be the polytope whose vertices are the incidence vectors of arc sets whose reversal makes $D$ strongly connected. We study the lattice theoretic properties of the integer points contained in a proper face $F$ of $P$ not contained in $\{x:x_a=i\}$ for any $a\in A,i\in \{0,1\}$. We prove under a mild necessary condition that $F\cap \{0,1\}^A$ contains an integral basis $B$, i.e., $B$ is linearly independent, and any integral vector in the linear hull of $F$ is an integral linear combination of $B$. This result is surprising as the integer points in $F$ do not necessarily form a Hilbert basis. In proving the result, we develop a theory similar to Matching Theory for degree-constrained dijoins in bipartite digraphs. Our result has consequences for head-disjoint strong orientations in hypergraphs, and also to a famous conjecture by Woodall that the minimum size of a dicut of $D$, say $\tau$, is equal to the maximum number of disjoint dijoins. We prove a relaxation of this conjecture, by finding for any prime number $p\geq 2$, a $p$-adic packing of dijoins of value $\tau$ and of support size at most $2|A|$. We also prove that the all-ones vector belongs to the lattice generated by $F\cap \{0,1\}^A$, where $F$ is the face of $P$ satisfying $x(\delta^+(U))=1$ for every minimum dicut $\delta^+(U)$., Comment: 29 pages, 1 figure

Anatomical landmarks are vital in medical imaging for navigation and anomaly detection. Modern large language models (LLMs), like Llama-2, offer promise for automating the mapping of these landmarks in free-text radiology reports to corresponding positions in image data. Recent studies propose LLMs may develop coherent representations of generative processes. Motivated by these insights, we investigated whether LLMs accurately represent the spatial positions of anatomical landmarks. Through experiments with Llama-2 models, we found that they can linearly represent anatomical landmarks in space with considerable robustness to different prompts. These results underscore the potential of LLMs to enhance the efficiency and accuracy of medical imaging workflows., Comment: 6 pages, 2 figures, 1 table

Training large models with millions or even billions of parameters from scratch incurs substantial computational costs. Parameter Efficient Fine-Tuning (PEFT) methods, particularly Low-Rank Adaptation (LoRA), address this challenge by adapting only a reduced number of parameters to specific tasks with gradient-based optimizers. In this paper, we cast PEFT as an optimal filtering/state estimation problem and present Low-Rank Kalman Optimizer (LoKO) to estimate the optimal trainable parameters in an online manner. We leverage the low-rank decomposition in LoRA to significantly reduce matrix sizes in Kalman iterations and further capitalize on a diagonal approximation of the covariance matrix to effectively decrease computational complexity from quadratic to linear in the number of trainable parameters. Moreover, we discovered that the initialization of the covariance matrix within the Kalman algorithm and the accurate estimation of the observation noise covariance are the keys in this formulation, and we propose robust approaches that work well across a vast range of well-established computer vision and language models. Our results show that LoKO converges with fewer iterations and yields better performance models compared to commonly used optimizers with LoRA in both image classifications and language tasks. Our study opens up the possibility of leveraging the Kalman filter as an effective optimizer for the online fine-tuning of large models.

Interfering-or-not-interfering quantum key distribution (INI-QKD) is an innovative protocol whose performance surpasses existing twin-field protocol variants. In this study, we introduce an additional step of advantage distillation (AD) after the quantum communication phase to further enhance its performance. Through the AD the raw key is partitioned into small blocks of bits to identify highly correlated bit pairs. We numerically compute the optimal partitioning for different realistic conditions. Our results show that by employing the advantage distillation the transmission distance is significantly increased and thus can potentially improve the secret key rate of INI-QKD. This in particular is most prominent in the presence of high polarization misalignment error rates and considerable phase mismatch, all without altering the experimental setup of the protocol.

Correspondence analysis, multiple correspondence analysis and their discriminant counterparts (i.e., discriminant simple correspondence analysis and discriminant multiple correspondence analysis) are methods of choice for analyzing multivariate categorical data. In these methods, variables are integrated into optimal components computed as linear combinations whose weights are obtained from a generalized singular value decomposition (GSVD) that integrates specific metric constraints on the rows and columns of the original data matrix. The weights of the linear combinations are, in turn, used to interpret the components, and this interpretation is facilitated when components are 1) pairwise orthogonal and 2) when the values of the weights are either large or small but not intermediate-a pattern called a simple or a sparse structure. To obtain such simple configurations, the optimization problem solved by the GSVD is extended to include new constraints that implement component orthogonality and sparse weights. Because multiple correspondence analysis represents qualitative variables by a set of binary variables, an additional group constraint is added to the optimization problem in order to sparsify the whole set representing one qualitative variable. This new algorithm-called group-sparse GSVD (gsGSVD)-integrates these constraints via an iterative projection scheme onto the intersection of subspaces where each subspace implements a specific constraint. In this paper, we expose this new algorithm and show how it can be adapted to the sparsification of simple and multiple correspondence analysis, and illustrate its applications with the analysis of four different data sets-each illustrating the sparsification of a particular CA-based analysis.

Detecting Out-of-Distribution (OOD) sensory data and covariate distribution shift aims to identify new test examples with different high-level image statistics to the captured, normal and In-Distribution (ID) set. Existing OOD detection literature largely focuses on semantic shift with little-to-no consensus over covariate shift. Generative models capture the ID data in an unsupervised manner, enabling them to effectively identify samples that deviate significantly from this learned distribution, irrespective of the downstream task. In this work, we elucidate the ability of generative models to detect and quantify domain-specific covariate shift through extensive analyses that involves a variety of models. To this end, we conjecture that it is sufficient to detect most occurring sensory faults (anomalies and deviations in global signals statistics) by solely modeling high-frequency signal-dependent and independent details. We propose a novel method, CovariateFlow, for OOD detection, specifically tailored to covariate heteroscedastic high-frequency image-components using conditional Normalizing Flows (cNFs). Our results on CIFAR10 vs. CIFAR10-C and ImageNet200 vs. ImageNet200-C demonstrate the effectiveness of the method by accurately detecting OOD covariate shift. This work contributes to enhancing the fidelity of imaging systems and aiding machine learning models in OOD detection in the presence of covariate shift., Comment: ECCV 2024, typos corrected

We analyze the coherence dynamics of a central spin coupled to a spin chain with a time-dependent noisy magnetic field, focusing on how noise influences the system's decoherence. Our results show that decoherency due to the nonequilibrium critical dynamics of the environment is amplified in the presence of uncorrelated and correlated Gaussian noise. We demonstrate that decoherence factor consistently signals the critical points, and exhibits exponential scaling with the system size, the square of noise intensity, and the noise correlation time at the critical points. We find that strong coupling between the qubit and the environment allows partial revivals of coherence, which diminish with increasing noise intensity or decreasing noise correlation time. In contrast, weak coupling leads to monotonic enhanced decoherence. The numerical results illustrate that, the revivals decay and scale exponentially with noise intensity. Moreover, the revivals increase and indicate linear or power law scaling with noise correlation time depends on how the correlated noise is fast or slow. Additionally, we explore the non-Markovianity of the dynamics, finding that it decays in the presence of noise but increases as the noise correlation time grows. Our findings have potential applications in the noise spectroscopy of external signals.

The Closest String Problem is an NP-hard problem that aims to find a string that has the minimum distance from all sequences that belong to the given set of strings. Its applications can be found in coding theory, computational biology, and designing degenerated primers, among others. There are efficient exact algorithms that have reached high-quality solutions for binary sequences. However, there is still room for improvement concerning the quality of solutions over DNA and protein sequences. In this paper, we introduce a three-stage algorithm that comprises the following process: first, we apply a novel alphabet pruning method to reduce the search space for effectively finding promising search regions. Second, a variant of beam search to find a heuristic solution is employed. This method utilizes a newly developed guiding function based on an expected distance heuristic score of partial solutions. Last, we introduce a local search to improve the quality of the solution obtained from the beam search. Furthermore, due to the lack of real-world benchmarks, two real-world datasets are introduced to verify the robustness of the method. The extensive experimental results show that the proposed method outperforms the previous approaches from the literature.

A promising way to mitigate the expensive process of obtaining a high-dimensional signal is to acquire a limited number of low-dimensional measurements and solve an under-determined inverse problem by utilizing the structural prior about the signal. In this paper, we focus on adaptive acquisition schemes to save further the number of measurements. To this end, we propose a reinforcement learning-based approach that sequentially collects measurements to better recover the underlying signal by acquiring fewer measurements. Our approach applies to general inverse problems with continuous action spaces and jointly learns the recovery algorithm. Using insights obtained from theoretical analysis, we also provide a probabilistic design for our methods using variational formulation. We evaluate our approach on multiple datasets and with two measurement spaces (Gaussian, Radon). Our results confirm the benefits of adaptive strategies in low-acquisition horizon settings.

Honeypot technologies provide an effective defense strategy for the Industrial Internet of Things (IIoT), particularly in enhancing the Advanced Metering Infrastructure's (AMI) security by bolstering the network intrusion detection system. For this security paradigm to be fully realized, it necessitates the active participation of small-scale power suppliers (SPSs) in implementing honeypots and engaging in collaborative data sharing with traditional power retailers (TPRs). To motivate this interaction, TPRs incentivize data sharing with tangible rewards. However, without access to an SPS's confidential data, it is daunting for TPRs to validate shared data, thereby risking SPSs' privacy and increasing sharing costs due to voluminous honeypot logs. These challenges can be resolved by utilizing Federated Learning (FL), a distributed machine learning (ML) technique that allows for model training without data relocation. However, the conventional FL algorithm lacks the requisite functionality for both the security defense model and the rewards system of the AMI network. This work presents two solutions: first, an enhanced and cost-efficient FedAvg algorithm incorporating a novel data quality measure, and second, FedPot, the development of an effective security model with a fair incentives mechanism under an FL architecture. Accordingly, SPSs are limited to sharing the ML model they learn after efficiently measuring their local data quality, whereas TPRs can verify the participants' uploaded models and fairly compensate each participant for their contributions through rewards. Simulation results, drawn from realistic mircorgrid network log datasets, demonstrate that the proposed solutions outperform state-of-the-art techniques by enhancing the security model and guaranteeing fair reward distributions., Comment: Accepted for IEEE TNSM

The computational challenges of Large Language Model (LLM) inference remain a significant barrier to their widespread deployment, especially as prompt lengths continue to increase. Due to the quadratic complexity of the attention computation, it takes 30 minutes for an 8B LLM to process a prompt of 1M tokens (i.e., the pre-filling stage) on a single A100 GPU. Existing methods for speeding up prefilling often fail to maintain acceptable accuracy or efficiency when applied to long-context LLMs. To address this gap, we introduce MInference (Milliontokens Inference), a sparse calculation method designed to accelerate pre-filling of long-sequence processing. Specifically, we identify three unique patterns in long-context attention matrices-the A-shape, Vertical-Slash, and Block-Sparsethat can be leveraged for efficient sparse computation on GPUs. We determine the optimal pattern for each attention head offline and dynamically build sparse indices based on the assigned pattern during inference. With the pattern and sparse indices, we perform efficient sparse attention calculations via our optimized GPU kernels to significantly reduce the latency in the pre-filling stage of long-context LLMs. Our proposed technique can be directly applied to existing LLMs without any modifications to the pre-training setup or additional fine-tuning. By evaluating on a wide range of downstream tasks, including InfiniteBench, RULER, PG-19, and Needle In A Haystack, and models including LLaMA-3-1M, GLM4-1M, Yi-200K, Phi-3-128K, and Qwen2-128K, we demonstrate that MInference effectively reduces inference latency by up to 10x for pre-filling on an A100, while maintaining accuracy. Our code is available at https://aka.ms/MInference., Comment: Accepted at NeurIPS 2024 (Spotlight)

The deployment of a growing number of devices in Internet of Things (IoT) networks implies that uninterrupted and seamless adaptation of wireless communication parameters (e.g., carrier frequency, bandwidth and modulation) will become essential. To utilize wireless devices capable of switching several communication parameters requires real-time self-optimizations at the radio frequency integrated circuit (RFIC) level based on system level performance metrics during the processing of complex modulated signals. This article introduces a novel design verification approach for reconfigurable RFICs based on end-to-end wireless system-level performance metrics while operating in a dynamically changing communication environment. In contrast to prior work, this framework includes two modules that simulate a wireless channel and decode waveforms. These are connected to circuit-level modules that capture device- and circuit-level non-idealities of RFICs for design validation and optimization, such as transistor noises, intermodulation/harmonic distortions, and memory effects from parasitic capacitances. We demonstrate this framework with a receiver (RX) consisting of a reconfigurable complementary metal-oxide semiconductor (CMOS) low-noise amplifier (LNA) designed at the transistor level, a behavioral model of a mixer, and an ideal filter model. The seamless integration between system-level wireless models with circuit-level and behavioral models (such as VerilogA-based models) for RFIC blocks enables to preemptively evaluate circuit and system designs, and to optimize for different communication scenarios with adaptive circuits having extensive tuning ranges. An exemplary case study is presented, in which simulation results reveal that the LNA power consumption can be reduced up to 16x depending on system-level requirements., Comment: Accepted, 5 figures, 1 table. To be included in Proc. IEEE DySPAN 2024

We investigate the sensing capacity of non-equilibrium dynamics in quantum systems exhibiting Bloch oscillations. By focusing on the resource efficiency of the probe, quantified by quantum Fisher information, we find different scaling behaviors in two different phases, namely localized and extended. Our results provide a quantitative ansatz for quantum Fisher information in terms of time, probe size, and the number of excitations. In the long-time regime, the quantum Fisher information is a quadratic function of time, touching the Heisenberg limit. The system size scaling drastically depends on the phase changing from super-Heisenberg scaling in the extended phase to size-independent behavior in the localized phase. Furthermore, increasing the number of excitations always enhances the precision of the probe, although, in the interacting systems the enhancement becomes less eminent than the non-interacting probes. This is due to the induced localization by increasing the interaction between the excitations.

Large language models (LLMs) can provide users with false, inaccurate, or misleading information, and we consider the output of this type of information as what Natale (2021) calls `banal' deceptive behaviour. Here, we investigate peoples' perceptions of ChatGPT-generated deceptive behaviour and how this affects peoples' own behaviour and trust. To do this, we use a mixed-methods approach comprising of (i) an online survey with 220 participants and (ii) semi-structured interviews with 12 participants. Our results show that (i) the most common types of deceptive information encountered were over-simplifications and outdated information; (ii) humans' perceptions of trust and `worthiness' of talking to ChatGPT are impacted by `banal' deceptive behaviour; (iii) the perceived responsibility for deception is influenced by education level and the frequency of deceptive information; and (iv) users become more cautious after encountering deceptive information, but they come to trust the technology more when they identify advantages of using it. Our findings contribute to the understanding of human-AI interaction dynamics in the context of \textit{Deceptive AI Ecosystems}, and highlight the importance of user-centric approaches to mitigating the potential harms of deceptive AI technologies.