Search

The development of robust generative models for highly varied non-stationary time series data is a complex yet important problem. Traditional models for time series data prediction, such as Long Short-Term Memory (LSTM), are inefficient and generalize poorly as they cannot capture complex temporal relationships. In this paper, we present a probabilistic generative model that can be trained to capture temporal information, and that is robust to data errors. We call it Time Deep Latent Gaussian Model (tDLGM). Its novel architecture is inspired by Deep Latent Gaussian Model (DLGM). Our model is trained to minimize a loss function based on the negative log loss. One contributing factor to Time Deep Latent Gaussian Model (tDLGM) robustness is our regularizer, which accounts for data trends. Experiments conducted show that tDLGM is able to reconstruct and generate complex time series data, and that it is robust against to noise and faulty data., Comment: New revision added a space between "for" and "Time-Series" in the title

Neural networks trained with stochastic gradient descent exhibit an inductive bias towards simpler decision boundaries, typically converging to a narrow family of functions, and often fail to capture more complex features. This phenomenon raises concerns about the capacity of deep models to adequately learn and represent real-world datasets. Traditional approaches such as explicit regularization, data augmentation, architectural modifications, etc., have largely proven ineffective in encouraging the models to learn diverse features. In this work, we investigate the impact of pre-training models with noisy labels on the dynamics of SGD across various architectures and datasets. We show that pretraining promotes learning complex functions and diverse features in the presence of noise. Our experiments demonstrate that pre-training with noisy labels encourages gradient descent to find alternate minima that do not solely depend upon simple features, rather learns more complex and broader set of features, without hurting performance., Comment: Accepted at Workshop on Scientific Methods for Understanding Deep Learning, NeurIPS 2024

In the supervised classification setting, during inference, deep networks typically make multiple predictions. For a pair of such predictions (that are in the top-k predictions), two distinct possibilities might occur. On the one hand, each of the two predictions might be primarily driven by two distinct sets of entities in the input. On the other hand, it is possible that there is a single entity or set of entities that is driving the prediction for both the classes in question. This latter case, in effect, corresponds to the network making two separate guesses about the identity of a single entity type. Clearly, both the guesses cannot be true, i.e. both the labels cannot be present in the input. Current techniques in interpretability research do not readily disambiguate these two cases, since they typically consider input attributions for one class label at a time. Here, we present a framework and method to do so, leveraging modern segmentation and input attribution techniques. Notably, our framework also provides a simple counterfactual "proof" of each case, which can be verified for the input on the model (i.e. without running the method again). We demonstrate that the method performs well for a number of samples from the ImageNet validation set and on multiple models., Comment: Accepted to Neural Information Processing Systems (NeurIPS 2024). ATTRIB Workshop

We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user's image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models. All videos from this paper are available at https://go.fb.me/MovieGenResearchVideos.

A new discrete cohesive zone model (DCZM) is presented for modeling the interface behavior of adhesive-bonded thin laminates and sandwich panels. The proposed model treats the interface as a spring element and the adherent as a beam element. The use of the preceding assumptions facilitates the simplification of the computational framework reducing the problem from a 2D to 1D, thereby relaxing the requirements of maintaining the aspect ratio of elements in the finite element mesh. For thin laminates, the constitutive relation of the adhesive is represented by a bi-linear traction-separation law, whereas for sandwich panels, an exponential law is employed to model the adhesive behavior. In order to validate the proposed model for thin laminates, simulations of three established fracture tests: DCB, ENF, and MMB have been undertaken. Additionally, for the sandwich panel, two experiments documented in the literature have been simulated for assessing the efficacy of the modelling. One experiment has a mode-I interface failure and the other one a mode-II interface failure between the core and skin. It has been observed that the model is not sensitive to either the element size or the load step size. The results have been compared with reported (benchmark) numerical, analytical, and experimental findings. The proposed methodology for thin laminates offers a significant reduction in the computational effort (reduced number of unknown degrees of freedom compared to existing methods) with no compromise on the accuracy of the predictions. Specifically, it reduces the unknown degrees of freedom by more than 25\% compared to the corresponding mesh used in existing continuum cohesive zone model (CCZM) approaches. The Newton-Raphson solver can achieve quick convergence and no line search feature is required. The proposed algorithm is easy to implement on any computational platform.

A robust quantum protocol has been developed that achieves highly efficient entanglement transfer from a three-atom Rydberg system, initially in a W state (|rrg>+|rgr>+|grr>)/3^0.5, to an equivalent photonic W state (|101>+|110>+|011>)/3^0.5. The entanglement transfer is achieved by dynamically adjusting the cavity mode frequencies and modulating the coupling rates, simplifying the complex transfer process into a sequence of processes involving two-level avoided crossings. We demonstrate that entanglement transfer can be achieved using either a fully adiabatic protocol or one with controlled non-adiabatic transitions at avoided crossings, generated by continuously chirping the cavity modes. Our adiabatic protocol uses the fractional STIRAP method to facilitate the partial population transfer required for generation of the photonic W state. In comparison, the non-adiabatic protocol uses non-adiabatic transitions to achieve the required partial population transfer. Furthermore, we propose two strategies for experimental implementation of our protocols., Comment: 22 pages, 14 figures

In recent years, deep neural networks (DNNs) have demonstrated remarkable performance in pathology applications, potentially even outperforming expert pathologists due to their ability to learn subtle features from large datasets. One complication in preparing digital pathology datasets for DNN tasks is variation in tinctorial qualities. A common way to address this is to perform stain normalization on the images. In this study, we show that a well-trained DNN model trained on one batch of histological slides failed to generalize to another batch prepared at a different time from the same tissue blocks, even when stain normalization methods were applied. This study used sample data from a previously reported DNN that was able to identify patients with early stage non-small cell lung cancer (NSCLC) whose tumors did and did not metastasize, with high accuracy, based on training and then testing of digital images from H&E stained primary tumor tissue sections processed at the same time. In this study we obtained a new series of histologic slides from the adjacent recuts of same tissue blocks processed in the same lab but at a different time. We found that the DNN trained on the either batch of slides/images was unable to generalize and failed to predict progression in the other batch of slides/images (AUC_cross-batch = 0.52 - 0.53 compared to AUC_same-batch = 0.74 - 0.81). The failure to generalize did not improve even when the tinctorial difference correction were made through either traditional color-tuning or stain normalization with the help of a Cycle Generative Adversarial Network (CycleGAN) process. This highlights the need to develop an entirely new way to process and collect consistent microscopy images from histologic slides that can be used to both train and allow for the general application of predictive DNN algorithms.

When estimating models of of a multivariable dynamic system, a typical condition for consistency is to require the input signals to be persistently exciting, which is guaranteed if the input spectrum is positive definite for a sufficient number of frequencies. In this paper it is investigated how such a condition can be relaxed by exploiting prior structural information on the multivariable system, such as structural zero elements in the transfer matrix or entries that are a priori known and therefore not parametrized. It is shown that in particular situations the data-informativity condition can be decomposed into different MISO (multiple input single output) situations, leading to relaxed conditions for the MIMO (multiple input multiple output) model. When estimating a single module in a linear dynamic network, the data-informativity conditions can generically be formulated as path-based conditions on the graph of the network. The new relaxed conditions for data-informativity will then also lead to relaxed path-based conditions on the network graph. Additionally the new expressions are shown to be closely related to earlier derived conditions for (generic) single module identifiability., Comment: 15 pages, 3 figures

Theoretical studies on the representation power of GNNs have been centered around understanding the equivalence of GNNs, using WL-Tests for detecting graph isomorphism. In this paper, we argue that such equivalence ignores the accompanying optimization issues and does not provide a holistic view of the GNN learning process. We illustrate these gaps between representation and optimization with examples and experiments. We also explore the existence of an implicit inductive bias (e.g. fully connected networks prefer to learn low frequency functions in their input space) in GNNs, in the context of graph classification tasks. We further prove theoretically that the message-passing layers in the graph, have a tendency to search for either discriminative subgraphs, or a collection of discriminative nodes dispersed across the graph, depending on the different global pooling layers used. We empirically verify this bias through experiments over real-world and synthetic datasets. Finally, we show how our work can help in incorporating domain knowledge via attention based architectures, and can evince their capability to discriminate coherent subgraphs.

This paper presents the novel method StreamLLM for generating streamliners in constraint programming using Large Language Models (LLMs). Streamliners are constraints that narrow the search space, enhancing the speed and feasibility of solving complex problems. Traditionally, streamliners were crafted manually or generated through systematically combined atomic constraints with high-effort offline testing. Our approach uses LLMs to propose effective streamliners. Our system StreamLLM generates streamlines for problems specified in the MiniZinc constraint programming language and integrates feedback to the LLM with quick empirical tests. Our rigorous empirical evaluation involving ten problems with several hundreds of test instances shows robust results that are highly encouraging, showcasing the transforming power of LLMs in the domain of constraint programming.

Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.

While contemporary reinforcement learning research and applications have embraced policy gradient methods as the panacea of solving learning problems, value-based methods can still be useful in many domains as long as we can wrangle with how to exploit them in a sample efficient way. In this paper, we explore the chaotic nature of DQNs in reinforcement learning, while understanding how the information that they retain when trained can be repurposed for adapting a model to different tasks. We start by designing a simple experiment in which we are able to observe the Q-values for each state and action in an environment. Then we train in eight different ways to explore how these training algorithms affect the way that accurate Q-values are learned (or not learned). We tested the adaptability of each trained model when retrained to accomplish a slightly modified task. We then scaled our setup to test the larger problem of an autonomous vehicle at an unprotected intersection. We observed that the model is able to adapt to new tasks quicker when the base model's Q-value estimates are closer to the true Q-values. The results provide some insights and guidelines into what algorithms are useful for sample efficient task adaptation.

Recent progress in artificial intelligence (AI) has been driven by insights from neuroscience, particularly with the development of artificial neural networks (ANNs). This has significantly enhanced the replication of complex cognitive tasks such as vision and natural language processing. Despite these advances, ANNs struggle with continual learning, adaptable knowledge transfer, robustness, and resource efficiency - capabilities that biological systems handle seamlessly. Specifically, ANNs often overlook the functional and morphological diversity of the brain, hindering their computational capabilities. Furthermore, incorporating cell-type specific neuromodulatory effects into ANNs with neuronal heterogeneity could enable learning at two spatial scales: spiking behavior at the neuronal level, and synaptic plasticity at the circuit level, thereby potentially enhancing their learning abilities. In this article, we summarize recent bio-inspired models, learning rules and architectures and propose a biologically-informed framework for enhancing ANNs. Our proposed dual-framework approach highlights the potential of spiking neural networks (SNNs) for emulating diverse spiking behaviors and dendritic compartments to simulate morphological and functional diversity of neuronal computations. Finally, we outline how the proposed approach integrates brain-inspired compartmental models and task-driven SNNs, balances bioinspiration and complexity, and provides scalable solutions for pressing AI challenges, such as continual learning, adaptability, robustness, and resource-efficiency., Comment: 30 pages, 4 figures, 3 boxes

We propose a framework to perform Bayesian inference using conditional score-based diffusion models to solve a class of inverse problems in mechanics involving the inference of a specimen's spatially varying material properties from noisy measurements of its mechanical response to loading. Conditional score-based diffusion models are generative models that learn to approximate the score function of a conditional distribution using samples from the joint distribution. More specifically, the score functions corresponding to multiple realizations of the measurement are approximated using a single neural network, the so-called score network, which is subsequently used to sample the posterior distribution using an appropriate Markov chain Monte Carlo scheme based on Langevin dynamics. Training the score network only requires simulating the forward model. Hence, the proposed approach can accommodate black-box forward models and complex measurement noise. Moreover, once the score network has been trained, it can be re-used to solve the inverse problem for different realizations of the measurements. We demonstrate the efficacy of the proposed approach on a suite of high-dimensional inverse problems in mechanics that involve inferring heterogeneous material properties from noisy measurements. Some examples we consider involve synthetic data, while others include data collected from actual elastography experiments. Further, our applications demonstrate that the proposed approach can handle different measurement modalities, complex patterns in the inferred quantities, non-Gaussian and non-additive noise models, and nonlinear black-box forward models. The results show that the proposed framework can solve large-scale physics-based inverse problems efficiently.

With the rise of freely available image generators, AI-generated art has become the centre of a series of heated debates, one of which concerns the concept of human creativity. Can an image generation AI exhibit ``creativity'' of the same type that artists do, and if so, how does that manifest? Our paper attempts to define and empirically measure one facet of creative behavior in AI, by conducting an experiment to quantify the "fluidity of prompt interpretation", or just "fluidity", in a series of selected popular image generators. To study fluidity, we (1) introduce a clear definition for it, (2) create chains of auto-generated prompts and images seeded with an initial "ground-truth: image, (3) measure these chains' breakage points using preexisting visual and semantic metrics, and (4) use both statistical tests and visual explanations to study these chains and determine whether the image generators used to produce them exhibit significant fluidity.

Deep learning assisted digital pathology has the potential to impact clinical practice in significant ways. In recent studies, deep neural network (DNN) enabled analysis outperforms human pathologists. Increasing sizes and complexity of the DNN architecture generally improves performance at the cost of DNN's explainability. For pathology, this lack of DNN explainability is particularly problematic as it hinders the broader clinical interpretation of the pathology features that may provide physiological disease insights. To better assess the features that DNN uses in developing predictive algorithms to interpret digital microscopic images, we sought to understand the role of resolution and tissue scale and here describe a novel method for studying the predictive feature length-scale that underpins a DNN's predictive power. We applied the method to study a DNN's predictive capability in the case example of brain metastasis prediction from early-stage non-small-cell lung cancer biopsy slides. The study highlights the DNN attention in the brain metastasis prediction targeting both cellular scale (resolution) and tissue scale features on H&E-stained histological whole slide images. At the cellular scale, we see that DNN's predictive power is progressively increased at higher resolution (i.e., lower resolvable feature length) and is largely lost when the resolvable feature length is longer than 5 microns. In addition, DNN uses more macro-scale features (maximal feature length) associated with tissue organization/architecture and is optimized when assessing visual fields larger than 41 microns. This study for the first time demonstrates the length-scale requirements necessary for optimal DNN learning on digital whole slide images.

Chiral active materials display odd dynamical effects in both their elastic and viscous responses. We show that the most symmetric mesophase with two-dimensional odd elasticity in three dimensions is chiral, polar and columnar, with two-dimensional translational order in the plane perpendicular to the columns and no elastic restoring force for their relative sliding. We derive its hydrodynamic equations from those of a chiral active variant of model H. The most striking prediction of the odd dynamics is two distinct types of column oscillation whose frequencies do not vanish at zero wavenumber. In addition, activity leads to a buckling instability coming from the generic force-dipole active stress analogous to the mechanical Helfrich-Hurault instability in passive materials, while the chiral torque-dipole active stress fundamentally modifies the instability by the selection of helical column undulations., Comment: replaces arXiv:2306.03695; new results

Systems driven far from equilibrium often retain structural memories of their processing history. This memory has, in some cases, been shown to dramatically alter the material response. For example, work hardening in crystalline metals can alter the hardness, yield strength, and tensile strength to prevent catastrophic failure. Whether memory of processing history can be similarly exploited in flowing systems, where significantly larger changes in structure should be possible, remains poorly understood. Here, we demonstrate a promising route to embedding such useful memories. We build on work showing that exposing a sheared dense suspension to acoustic perturbations of different power allows for dramatically tuning the sheared suspension viscosity and underlying structure. We find that, for sufficiently dense suspensions, upon removing the acoustic perturbations, the suspension shear jams with shear stress contributions from the maximum compressive and maximum extensive axes that reflect the acoustic training. Because the contributions from these two orthogonal axes to the total shear stress are antagonistic, it is possible to tune the resulting suspension response in surprising ways. For example, we show that differently trained sheared suspensions exhibit: 1) different susceptibility to the same acoustic perturbation; 2) orders of magnitude changes in their instantaneous viscosities upon shear reversal; and 3) even a shear stress that increases in magnitude upon shear cessation. To further illustrate the power of this approach for controlling suspension properties, we demonstrate that flowing states well below the shear jamming threshold can be shear jammed via acoustic training. Collectively, our work paves the way for using acoustically induced memory in dense suspensions to generate rapidly and widely tunable materials., Comment: To be published in Physical Review X

There currently exist two extreme viewpoints for neural network feature learning -- (i) Neural networks simply implement a kernel method (a la NTK) and hence no features are learned (ii) Neural networks can represent (and hence learn) intricate hierarchical features suitable for the data. We argue in this paper neither interpretation is likely to be correct based on a novel viewpoint. Neural networks can be viewed as a mixture of experts, where each expert corresponds to a (number of layers length) path through a sequence of hidden units. We use this alternate interpretation to motivate a model, called the Deep Linearly Gated Network (DLGN), which sits midway between deep linear networks and ReLU networks. Unlike deep linear networks, the DLGN is capable of learning non-linear features (which are then linearly combined), and unlike ReLU networks these features are ultimately simple -- each feature is effectively an indicator function for a region compactly described as an intersection of (number of layers) half-spaces in the input space. This viewpoint allows for a comprehensive global visualization of features, unlike the local visualizations for neurons based on saliency/activation/gradient maps. Feature learning in DLGNs is shown to happen and the mechanism with which this happens is through learning half-spaces in the input space that contain smooth regions of the target function. Due to the structure of DLGNs, the neurons in later layers are fundamentally the same as those in earlier layers -- they all represent a half-space -- however, the dynamics of gradient descent impart a distinct clustering to the later layer neurons. We hypothesize that ReLU networks also have similar feature learning behaviour.

We present experiments, supported by mechanically detailed simulations, establishing bulk vapor-liquid condensation of a hard-bead fluid by a tiny population of orientable motile grains that self-assembles into a moving polarized monolayer. In a quasi-1D geometry two such layers, oppositely aligned, immobilize the condensed non-motile component. We account for our observations through a continuum theory with a naturally non-reciprocal Cahn-Hilliard structure, whose predicted trends as a function of packing fraction are consistent with our observations., Comment: PDF repaired. References added; minor text changes. This article consists of thirteen pages and seven figures. For supplementary videos, please click on the link below: https://drive.google.com/drive/folders/19KhQcbNDJJTQsuF9Tr1apoEXon4UFyZv

We present a supervised learning approach for automatic extraction of keyphrases from single documents. Our solution uses simple to compute statistical and positional features of candidate phrases and does not rely on any external knowledge base or on pre-trained language models or word embeddings. The ranking component of our proposed solution is a fairly lightweight ensemble model. Evaluation on benchmark datasets shows that our approach achieves significantly higher accuracy than several state-of-the-art baseline models, including all deep learning-based unsupervised models compared with, and is competitive with some supervised deep learning-based models too. Despite the supervised nature of our solution, the fact that does not rely on any corpus of "golden" keywords or any external knowledge corpus means that our solution bears the advantages of unsupervised solutions to a fair extent.

Machine Learning (ML) is becoming increasingly important for IoT-based applications. However, the dynamic and ad-hoc nature of many IoT ecosystems poses unique challenges to the efficacy of ML algorithms. One such challenge is data incompleteness, which is manifested as missing sensor readings. Many factors, including sensor failures and/or network disruption, can cause data incompleteness. Furthermore, most IoT systems are severely power-constrained. It is important that we build IoT-based ML systems that are robust against data incompleteness while simultaneously being energy efficient. This paper presents an empirical study of SECOE - a recent technique for alleviating data incompleteness in IoT - with respect to its energy bottlenecks. Towards addressing the energy bottlenecks of SECOE, we propose ENAMLE - a proactive, energy-aware technique for mitigating the impact of concurrent missing data. ENAMLE is unique in the sense that it builds an energy-aware ensemble of sub-models, each trained with a subset of sensors chosen carefully based on their correlations. Furthermore, at inference time, ENAMLE adaptively alters the number of the ensemble of models based on the amount of missing data rate and the energy-accuracy trade-off. ENAMLE's design includes several novel mechanisms for minimizing energy consumption while maintaining accuracy. We present extensive experimental studies on two distinct datasets that demonstrate the energy efficiency of ENAMLE and its ability to alleviate sensor failures., Comment: 8 pages, 8 figures, 1 table, Accepted as a conference paper at IEEE INTERNATIONAL CONFERENCE ON DISTRIBUTED COMPUTING IN SMART SYSTEMS AND THE INTERNET OF THINGS (DCOSS-IoT 2024)

We show that an inclusion placed inside a dilute Stokesian suspension of microswimmers induces power-law number-density modulations and flows. These take a different form depending on whether the inclusion is held fixed by an external force, for example an optical tweezer, or if it is free. When the inclusion is held in place, the far-field fluid flow is a Stokeslet, while the microswimmer density decays as $1/r^{2+\epsilon}$, with $r$ the distance from the inclusion, and $\epsilon$ an anomalous exponent which depends on the symmetry of the inclusion and varies continuously as a function of a dimensionless number characterizing the relative amplitudes of the convective and diffusive effects. The angular dependence takes a non-trivial form which depends on the same dimensionless number. When the inclusion is free to move, the far-field fluid flow is a stresslet and the microswimmer density decays as $1/r^2$ with a simple angular dependence. These long-range modulations mediate long-range interactions between inclusions that we characterize., Comment: To be published in PRX

We discover an instability mechanism in suspensions of self-propelled particles that does not involve active stress. Instead, it is driven by a subtle interplay of inertia, swimmer motility, and concentration fluctuations, through a crucial time lag between the velocity and the concentration field. The resulting time-persistent state seen in our high-resolution numerical simulations consists of self-sustained waves of concentration and orientation, transiting from regular oscillations to wave turbulence. We analyze the statistical features of this active turbulence, including an intriguing connection to the Batchelor spectrum of passive scalars., Comment: 15 pages and 12 figures including supplementary material

The Hamiltonian nature of the precessional dynamics of the classical Heisenberg model leads to reciprocal interactions amongst the spins. Heisenberg spins are reciprocal in nature. In this work, we study the dynamics of a nonequilibrium classical spin chain in which the neighbours interact through a purely non-reciprocal exchange coupling [EPL 60, 418 (2002)] which preserves rotational symmetry. The resultant dynamics conserves neither magnetization nor energy. We uncover other local conservation laws in their place in the extreme case of a strictly antisymmetric coupling. We show numerically that the model undergoes an analogue of thermalization. We present results on the presence of conserved quantities, their diffusive spreading and a hydrodynamic picture, and the nature of the decorrelation front upon adding an initial perturbation to the system., Comment: 10 pages, 8 figures, main text and supplemental information