Search

Large neural networks achieve remarkable performance, but their size hinders deployment on resource-constrained devices. While various compression techniques exist, parameter sharing remains relatively unexplored. This paper introduces Fine-grained Parameter Sharing (FiPS), a novel algorithm that leverages the relationship between parameter sharing, tensor decomposition, and sparsity to efficiently compress large vision transformer models. FiPS employs a shared base and sparse factors to represent shared neurons across multi-layer perception (MLP) modules. Shared parameterization is initialized via Singular Value Decomposition (SVD) and optimized by minimizing block-wise reconstruction error. Experiments demonstrate that FiPS compresses DeiT-B and Swin-L MLPs to 25-40% of their original parameter count while maintaining accuracy within 1 percentage point of the original models.

In recent years, Dynamic Sparse Training (DST) has emerged as an alternative to post-training pruning for generating efficient models. In principle, DST allows for a more memory efficient training process, as it maintains sparsity throughout the entire training run. However, current DST implementations fail to capitalize on this in practice. Because sparse matrix multiplication is much less efficient than dense matrix multiplication on GPUs, most implementations simulate sparsity by masking weights. In this paper, we leverage recent advances in semi-structured sparse training to apply DST in the domain of classification with large output spaces, where memory-efficiency is paramount. With a label space of possibly millions of candidates, the classification layer alone will consume several gigabytes of memory. Switching from a dense to a fixed fan-in sparse layer updated with sparse evolutionary training (SET); however, severely hampers training convergence, especially at the largest label spaces. We find that poor gradient flow from the sparse classifier to the dense text encoder make it difficult to learn good input representations. By employing an intermediate layer or adding an auxiliary training objective, we recover most of the generalisation performance of the dense model. Overall, we demonstrate the applicability and practical benefits of DST in a challenging domain -- characterized by a highly skewed label distribution that differs substantially from typical DST benchmark datasets -- which enables end-to-end training with millions of labels on commodity hardware., Comment: 20 pages, 7 figures, NeurIPS 2024

We consider the problem of testing and learning from data in the presence of resource constraints, such as limited memory or weak data access, which place limitations on the efficiency and feasibility of testing or learning. In particular, we ask the following question: Could a resource-constrained learner/tester use interaction with a resource-unconstrained but untrusted party to solve a learning or testing problem more efficiently than they could without such an interaction? In this work, we answer this question both abstractly and for concrete problems, in two complementary ways: For a wide variety of scenarios, we prove that a resource-constrained learner cannot gain any advantage through classical interaction with an untrusted prover. As a special case, we show that for the vast majority of testing and learning problems in which quantum memory is a meaningful resource, a memory-constrained quantum algorithm cannot overcome its limitations via classical communication with a memory-unconstrained quantum prover. In contrast, when quantum communication is allowed, we construct a variety of interactive proof protocols, for specific learning and testing problems, which allow memory-constrained quantum verifiers to gain significant advantages through delegation to untrusted provers. These results highlight both the limitations and potential of delegating learning and testing problems to resource-rich but untrusted third parties., Comment: 12 + 33 + 13 pages; 1 table; 2 figures

Achieving stable bipedal walking on surfaces with unknown motion remains a challenging control problem due to the hybrid, time-varying, partially unknown dynamics of the robot and the difficulty of accurate state and surface motion estimation. Surface motion imposes uncertainty on both system parameters and non-homogeneous disturbance in the walking robot dynamics. In this paper, we design an adaptive ankle torque controller to simultaneously address these two uncertainties and propose a step-length planner to minimize the required control torque. Typically, an adaptive controller is used for a continuous system. To apply adaptive control on a hybrid system such as a walking robot, an intermediate command profile is introduced to ensure a continuous error system. Simulations on a planar bipedal robot, along with comparisons against a baseline controller, demonstrate that the proposed approach effectively ensures stable walking and accurate tracking under unknown, time-varying disturbances.

Risk perception plays a key role in shaping the collective behavior of moving animal groups, yet the effects of variation in perceived risk within groups is unknown. Here, we merge two subgroups of fish with different levels of perceived risk, manipulated through habituation to the experimental arena, and quantitatively analyze their movement. Fish with heightened risk perception tend to occupy more central positions within the group, consistent with the selfish herd hypothesis. This behavior appears to be driven by adjusted local interactions with nearby individuals. Compared to more habituated fish, these individuals also exhibit greater coordination, shorter burst-and-coast dynamics, more efficient information transmission, and a stronger tendency to adopt follower roles. To validate the findings, we develop a machine learning tool that successfully classifies the subgroup identity of individual fish. Our study demonstrates how differences in local-scale inter-individual interactions can drive spatial assortment within groups, based on heterogeneity in the perception of risk., Comment: Main paper (12 pages) + Supplementary Information (14 pages)

Knowledge Distillation is a commonly used Deep Neural Network compression method, which often maintains overall generalization performance. However, we show that even for balanced image classification datasets, such as CIFAR-100, Tiny ImageNet and ImageNet, as many as 41% of the classes are statistically significantly affected by distillation when comparing class-wise accuracy (i.e. class bias) between a teacher/distilled student or distilled student/non-distilled student model. Changes in class bias are not necessarily an undesirable outcome when considered outside of the context of a model's usage. Using two common fairness metrics, Demographic Parity Difference (DPD) and Equalized Odds Difference (EOD) on models trained with the CelebA, Trifeature, and HateXplain datasets, our results suggest that increasing the distillation temperature improves the distilled student model's fairness -- for DPD, the distilled student even surpasses the fairness of the teacher model at high temperatures. This study highlights the uneven effects of Knowledge Distillation on certain classes and its potentially significant role in fairness, emphasizing that caution is warranted when using distilled models for sensitive application domains.

Artificial Intelligence (AI) has paved the way for revolutionary decision-making processes, which if harnessed appropriately, can contribute to advancements in various sectors, from healthcare to economics. However, its black box nature presents significant ethical challenges related to bias and transparency. AI applications are hugely impacted by biases, presenting inconsistent and unreliable findings, leading to significant costs and consequences, highlighting and perpetuating inequalities and unequal access to resources. Hence, developing safe, reliable, ethical, and Trustworthy AI systems is essential. Our team of researchers working with Trustworthy and Responsible AI, part of the Transdisciplinary Scholarship Initiative within the University of Calgary, conducts research on Trustworthy and Responsible AI, including fairness, bias mitigation, reproducibility, generalization, interpretability, and authenticity. In this paper, we review and discuss the intricacies of AI biases, definitions, methods of detection and mitigation, and metrics for evaluating bias. We also discuss open challenges with regard to the trustworthiness and widespread application of AI across diverse domains of human-centric decision making, as well as guidelines to foster Responsible and Trustworthy AI models., Comment: 44 pages, 2 figures

Analog quantum simulation allows for assessing static and dynamical properties of strongly correlated quantum systems to high precision. To perform simulations outside the reach of classical computers, accurate and reliable implementations of the anticipated Hamiltonians are required. To achieve those, characterization and benchmarking tools are a necessity. For digital quantum devices, randomized benchmarking can provide a benchmark on the average quality of the implementation of a gate set. In this work, we introduce a versatile framework for randomized analog benchmarking of bosonic and fermionic quantum devices implementing particle number preserving dynamics. The scheme makes use of the restricted operations which are native to analog simulators and other continuous variable systems. Importantly, like randomized benchmarking, it is robust against state preparation and measurement errors. We discuss the scheme's efficiency, derive theoretical performance guarantees and showcase the protocol with numerical examples., Comment: Main text 5 pages and 3 figures, references and supplementary material 16 pages and 5 figures

The latest advancements in Distributed Ledger Technology (DLT), and payment architectures such as the UK's New Payments Architecture, present opportunities for leveraging the hidden informational value and intelligence within payments. In this paper, we present Smart Money, an infrastructure capability for a Central Bank Digital Currency (CBDC) which enables real-time Value Added Tax split payments, oversight, controlled access and smart policy implementation. This capability is implemented as a prototype, called Making Tax Smart (MTS), which is based on the open source R3 Corda framework. The results presented herein confirm that it is feasible to build a MTS capability which is scalable and co-exists with the current payment systems. Smart Money CBDC has the potential to mobilise payments data in order to transform the role of money from a blunt instrument to a government policy sensor and actuator without disrupting the existing money system. DLT, smart contracts and programmable money have a crucial role to play with benefits for government departments, the economy and society as a whole.

Although many real-world applications, such as disease prediction, and fault detection suffer from class imbalance, most existing graph-based classification methods ignore the skewness of the distribution of classes; therefore, tend to be biased towards the majority class(es). Conventional methods typically tackle this problem through the assignment of weights to each one of the class samples based on a function of their loss, which can lead to over-fitting on outliers. In this paper, we propose a meta-learning algorithm, named Meta-GCN, for adaptively learning the example weights by simultaneously minimizing the unbiased meta-data set loss and optimizing the model weights through the use of a small unbiased meta-data set. Through experiments, we have shown that Meta-GCN outperforms state-of-the-art frameworks and other baselines in terms of accuracy, the area under the receiver operating characteristic (AUC-ROC) curve, and macro F1-Score for classification tasks on two different datasets.

Cross-platform verification is the task of comparing the output states produced by different physical platforms using solely local quantum operations and classical communication. While protocols have previously been suggested for this task, their exponential sample complexity renders them unpractical even for intermediate-scale quantum systems. In this work, we propose a novel protocol for this task based on Pauli sampling, a subroutine which generates Paulis distributed according to their weight in the expansion of a quantum state in the Pauli basis. We show that our protocols for both Pauli sampling and cross-platform verification are efficient for quantum states with low magic and entanglement (i.e., of the order $O(\log n)$). Conversely, we show super-polynomial lower bounds on the complexity of both tasks for states with $\omega(\log n)$ magic and entanglement. Interestingly, when considering states with real amplitudes the requirements of our protocol for cross-platform verification can be significantly weakened., Comment: Improved Lemma 1, added references and corrected typos

We study the propagation of both low- and high-amplitude ring-shaped sound waves in a 2D square lattice of acoustic waveguides with Helmholtz resonators. We show that the inclusion of the Helmholtz resonators suppresses the inherent anisotropy of the system in the low frequency regime allowing for radially symmetric solutions. By employing the electroacoustic analogue approach and asymptotic methods we derive an effective cylindrical Korteweg de Vries (cKdV) equation. Low-amplitude waveforms are self-similar structures of the Airy function profile, while high-amplitude ones are of the form of cylindrical solitons. Our analytical predictions are corroborated by results of direct numerical simulations, with a very good agreement between the two.

Arterial traffic interacts with freeway traffic, yet the two are controlled independently. Arterial traffic signals do not take into account freeway traffic and how ramps control ingress traffic and have no control over egress traffic from the freeway. This often results in long queues in either direction that block ramps and spill over to arterial streets or freeway lanes. In this paper, we propose an adaptive arterial traffic control strategy that combines traffic signal control (TSC) and dynamic speed offset (DSO) coordination using a Q-learning algorithm for a traffic network that involves a freeway segment and adjacent arterial streets. The TSC agent computes the signal cycle length and split based on observed intersection demands and adjacent freeway off-ramp queues. The DSO agent computes the relative offset and the recommended speeds of both ways between consecutive intersections based on their physical distance, intersection queues, and signal cycles. We evaluate the performance of the proposed arterial traffic control strategy using microscopic traffic simulations of an arterial corridor with seven intersections near the I-710 freeway. The proposed QL-based control significantly outperforms a fixed-time control and MAXBAND in terms of the travel time and the number of stops under low or moderate demands. In high-demand scenarios, the travel-time benefit provided by the QL-based control is reduced as it mitigates off-ramp and intersection queues, which is a necessary trade-off in our perspective. In addition, mutual benefit is obtained by implementing freeway and arterial traffic control simultaneously., Comment: submitted to TR-C

Photon loss represents a major challenge for the implementation of quantum communication protocols with untrusted devices, e.g. in the device-independent (DI) or semi-DI approaches. Determining critical loss thresholds is usually done in case-by-case studies. In the present work, we develop a general framework for characterizing the admissible levels of loss and noise in a wide range of scenarios and protocols with untrusted measurement devices. In particular, we present general bounds that apply to prepare-and-measure protocols for the semi-DI approach, as well as to Bell tests for DI protocols. A key step in our work is to establish a general connection between quantum protocols with untrusted measurement devices and the fundamental notions of channel extendibility and joint-measurability, which capture essential aspects of the communication and measurement of quantum information. In particular, this leads us to introduce the notion of partial joint-measurability, which naturally arises within quantum cryptography.

In the upcoming 6G era, mobile networks must deal with more challenging applications (e.g., holographic telepresence and immersive communication) and meet far more stringent application requirements stemming along the edge-cloud continuum. These new applications will create an elevated level of expectations on performance, reliability, ubiquity, trustworthiness, security, openness, and sustainability, pushing the boundaries of innovation and driving transformational change across the architecture of future mobile networks. Towards this end, ADROIT6G proposes a set of disruptive innovations with a clear vision on setting a 6G network architecture that can be tailored to the requirements of innovative applications and match the ambitious KPIs set for 6G networks. More specifically, the key transformations that ADROIT6G considers essential to 6G network evolution are: i) AI/ML-powered optimisations across the network, exploring solutions in the "Distributed Artificial Intelligence (DAI)" domain for high performance and automation; ii) Transforming to fully cloud-native network software, which can be implemented across various edge-cloud platforms, with security built integrally into the network user plan; and iii) Software driven, zero-touch operations and ultimately automation of every aspect of the network and the services it delivers.

Randomized measurements are increasingly appreciated as powerful tools to estimate properties of quantum systems, e.g., in the characterization of hybrid classical-quantum computation. On many platforms they constitute natively accessible measurements, serving as the building block of prominent schemes like shadow estimation. In the real world, however, the implementation of the random gates at the core of these schemes is susceptible to various sources of noise and imperfections, strongly limiting the applicability of protocols. To attenuate the impact of this shortcoming, in this work we introduce an error-mitigated method of randomized measurements, giving rise to a robust shadow estimation procedure. On the practical side, we show that error mitigation and shadow estimation can be carried out using the same session of quantum experiments, hence ensuring that we can address and mitigate the noise affecting the randomization measurements. Mathematically, we develop a picture derived from Fourier-transforms to connect randomized benchmarking and shadow estimation. We prove rigorous performance guarantees and show the functioning using comprehensive numerics. More conceptually, we demonstrate that, if properly used, easily accessible data from randomized benchmarking schemes already provide such valuable diagnostic information to inform about the noise dynamics and to assist in quantum learning procedures., Comment: 6+20 pages, 6 figures

The field of Neural Style Transfer (NST) has witnessed remarkable progress in the past few years, with approaches being able to synthesize artistic and photorealistic images and videos of exceptional quality. To evaluate such results, a diverse landscape of evaluation methods and metrics is used, including authors' opinions based on side-by-side comparisons, human evaluation studies that quantify the subjective judgements of participants, and a multitude of quantitative computational metrics which objectively assess the different aspects of an algorithm's performance. However, there is no consensus regarding the most suitable and effective evaluation procedure that can guarantee the reliability of the results. In this review, we provide an in-depth analysis of existing evaluation techniques, identify the inconsistencies and limitations of current evaluation methods, and give recommendations for standardized evaluation practices. We believe that the development of a robust evaluation framework will not only enable more meaningful and fairer comparisons among NST methods but will also enhance the comprehension and interpretation of research findings in the field., Comment: 21 pages, 8 figures

We report on a direct mapping of acoustic slender waveguides to the one dimensional trimer Su- Schrieffer-Heeger model, with neither chiral nor mirror symmetry. Importantly, we can choose to perform this mapping for either the acoustic velocity or pressure. We demonstrate that, for finite systems, this choice is necessarily linked to the boundary conditions. It allows for the unveiling of the edge states of the acoustic system through an edge state phase diagram. An experimental realization of our setup in the audible regime corroborates our theoretical predictions.

Neural Style Transfer (NST) research has been applied to images, videos, 3D meshes and radiance fields, but its application to 3D computer games remains relatively unexplored. Whilst image and video NST systems can be used as a post-processing effect for a computer game, this results in undesired artefacts and diminished post-processing effects. Here, we present an approach for injecting depth-aware NST as part of the 3D rendering pipeline. Qualitative and quantitative experiments are used to validate our in-game stylisation framework. We demonstrate temporally consistent results of artistically stylised game scenes, outperforming state-of-the-art image and video NST methods., Comment: 11 pages, 6 figures

Numerous studies have shown the effectiveness of intelligent transportation system techniques such as variable speed limit (VSL), lane change (LC) control, and ramp metering (RM) in freeway traffic flow control. The integration of these techniques has the potential to further enhance the traffic operation efficiency of both freeway and adjacent arterial networks. In this regard, we propose a freeway traffic control (FTC) strategy that coordinates VSL, LC, RM actions using a Q-learning (QL) framework which takes into account arterial traffic characteristics. The signal timing and demands of adjacent arterial intersections are incorporated as state variables of the FTC agent. The FTC agent is initially trained offline using a single-section road network, and subsequently deployed online in a connected freeway and arterial simulation network for continuous learning. The arterial network is assumed to be regulated by a traffic-responsive signal control strategy based on a cycle length model. Microscopic simulations demonstrate that the fully-trained FTC agent provides significant reductions in freeway travel time and the number of stops in scenarios with traffic congestion. It clearly outperforms an uncoordinated FTC and a decentralized feedback control strategy. Even though the FTC agent does not control the arterial traffic signals, it leads to shorter average queue lengths at arterial intersections by taking into account the arterial traffic conditions in controlling freeway traffic. These results motivate a future research where the QL framework will also include the control of arterial traffic signals., Comment: 12 pages, 9 figures, 5 tables

Efficient hyperpolarisation of nuclear spins via optically active defect centers, such as the nitrogen vacancy (NV) center in diamond, has great potential for enhancing NMR based quantum information processing and nanoscale magnetic resonance imaging. Recently, pulse-based protocols have been shown to efficiently transfer optically induced polarisation of the electron defect spin to surrounding nuclear spins -- at particular resonant pulse intervals. In this work, we investigate the performance of these protocols, both analytically and experimentally, with the electronic spin of a single NV defect. We find that whenever polarisation resonances of nuclear spins are near-degenerate with a `blocking' spin, which is single spin with stronger off-diagonal coupling to the electronic central spin, they are displaced out of the central resonant region -- without, in general, significant weakening of the resonance. We analyse the underlying physical mechanism and obtain a closed form expression for the displacement. We propose that spin blocking represents a common but overlooked effect in hyperpolarisation of nuclear spins and suggest solutions for improved protocol performance in the presence of (naturally occurring) blocking nuclear spins.

Although the roll-streak (R-S) is fundamentally involved in the dynamics of wall-turbulence, the physical mechanism responsible for its formation and maintenance remains controversial. In this work we investigate the dynamics maintaining the R-S in turbulent Poiseuille flow at R=1650. Spanwise collocation is used to remove spanwise displacement of the streaks and associated flow components, which isolates the streamwise-mean flow R-S component and the second-order statistics of the streamwise-varying fluctuations that are collocated with the R-S. This streamwise-mean/fluctuation partition of the dynamics facilitates exploiting insights gained from the analytic characterization of turbulence in the second-order statistical state dynamics (SSD), referred to as S3T, and its closely associated restricted nonlinear dynamics (RNL) approximation. Symmetry of the statistics about the streak centerline permits separation of the fluctuations into sinuous and varicose components. The Reynolds stress forcing induced by the sinuous and varicose fluctuations acting on the R-S is shown to reinforce low- and high-speed streaks respectively. This targeted reinforcement of streaks by the Reynolds stresses occurs continuously as the fluctuation field is strained by the streamwise-mean streak and not intermittently as would be associated with streak-breakdown events. The Reynolds stresses maintaining the streamwise-mean roll arise primarily from the dominant POD modes of the fluctuations, which can be identified with the time average structure of optimal perturbations growing on the streak. These results are consistent with a universal process of R-S growth and maintenance in turbulent shear flow arising from roll forcing generated by straining turbulent fluctuations, which was identified using the S3T SSD., Comment: 33 pages, 26 figures, submitted for publication to the Journal of Fluid Mechanics

Urban Air Mobility (UAM) offers a solution to current traffic congestion by providing on-demand air mobility in urban areas. Effective traffic management is crucial for efficient operation of UAM systems, especially for high-demand scenarios. In this paper, we present a centralized traffic management framework for on-demand UAM systems. Specifically, we provide a scheduling policy, called VertiSync, which schedules the aircraft for either servicing trip requests or rebalancing in the system subject to aircraft safety margins and energy requirements. We characterize the system-level throughput of VertiSync, which determines the demand threshold at which passenger waiting times transition from being stabilized to being increasing over time. We show that the proposed policy is able to maximize throughput for sufficiently large fleet sizes. We demonstrate the performance of VertiSync through a case study for the city of Los Angeles, and show that it significantly reduces passenger waiting times compared to a first-come first-serve scheduling policy., Comment: 9 pages, 6 figures

Despite the success of deep neural networks in medical image classification, the problem remains challenging as data annotation is time-consuming, and the class distribution is imbalanced due to the relative scarcity of diseases. To address this problem, we propose Class-Specific Distribution Alignment (CSDA), a semi-supervised learning framework based on self-training that is suitable to learn from highly imbalanced datasets. Specifically, we first provide a new perspective to distribution alignment by considering the process as a change of basis in the vector space spanned by marginal predictions, and then derive CSDA to capture class-dependent marginal predictions on both labeled and unlabeled data, in order to avoid the bias towards majority classes. Furthermore, we propose a Variable Condition Queue (VCQ) module to maintain a proportionately balanced number of unlabeled samples for each class. Experiments on three public datasets HAM10000, CheXpert and Kvasir show that our method provides competitive performance on semi-supervised skin disease, thoracic disease, and endoscopic image classification tasks., Comment: Paper appears in Computers in Biology and Medicine 2023, 164, 107280

We present and evaluate the ExaNeSt Prototype, a liquid-cooled rack prototype consisting of 256 Xilinx ZU9EG MPSoCs, 4 TBytes of DRAM, 16 TBytes of SSD, and configurable interconnection 10-Gbps hardware. We developed this testbed in 2016-2019 to validate the flexibility of FPGAs for experimenting with efficient hardware support for HPC communication among tens of thousands of processors and accelerators in the quest towards Exascale systems and beyond. We present our key design choices reagrding overall system architecture, PCBs and runtime software, and summarize insights resulting from measurement and analysis. Of particular note, our custom interconnect includes a low-cost low-latency network interface, offering user-level zero-copy RDMA, which we have tightly coupled with the ARMv8 processors in the MPSoCs. We have developed a system software runtime on top of these features, and have been able to run MPI. We have evaluated our testbed through MPI microbenchmarks, mini, and full MPI applications. Single hop, one way latency is $1.3$~$\mu$s; approximately $0.47$~$\mu$s out of these are attributed to network interface and the user-space library that exposes its functionality to the runtime. Latency over longer paths increases as expected, reaching $2.55$~$\mu$s for a five-hop path. Bandwidth tests show that, for a single hop, link utilization reaches $82\%$ of the theoretical capacity. Microbenchmarks based on MPI collectives reveal that broadcast latency scales as expected when the number of participating ranks increases. We also implemented a custom Allreduce accelerator in the network interface, which reduces the latency of such collectives by up to $88\%$. We assess performance scaling through weak and strong scaling tests for HPCG, LAMMPS, and the miniFE mini application; for all these tests, parallelization efficiency is at least $69\%$, or better., Comment: 45 pages, 23 figures

Quantum data access and quantum processing can make certain classically intractable learning tasks feasible. However, quantum capabilities will only be available to a select few in the near future. Thus, reliable schemes that allow classical clients to delegate learning to untrusted quantum servers are required to facilitate widespread access to quantum learning advantages. Building on a recently introduced framework of interactive proof systems for classical machine learning, we develop a framework for classical verification of quantum learning. We exhibit learning problems that a classical learner cannot efficiently solve on their own, but that they can efficiently and reliably solve when interacting with an untrusted quantum prover. Concretely, we consider the problems of agnostic learning parities and Fourier-sparse functions with respect to distributions with uniform input marginal. We propose a new quantum data access model that we call "mixture-of-superpositions" quantum examples, based on which we give efficient quantum learning algorithms for these tasks. Moreover, we prove that agnostic quantum parity and Fourier-sparse learning can be efficiently verified by a classical verifier with only random example or statistical query access. Finally, we showcase two general scenarios in learning and verification in which quantum mixture-of-superpositions examples do not lead to sample complexity improvements over classical data. Our results demonstrate that the potential power of quantum data for learning tasks, while not unlimited, can be utilized by classical agents through interaction with untrusted quantum entities., Comment: 12 + 46 + 23 pages, 1 table, 1 figure; V2 fixes some typos and includes new results in Section 6.3