Search

We address the problem of extending a pretrained large language model to a new domain that was not seen at training time, like adding a language for which the original model has seen no or little training data. Popular solutions like fine-tuning or low-rank adaptation are successful at domain adaptation, but formally they do not add any extra capacity and degrade the performance in the original domain. Our paper analyzes this extension problem under three angles: data, architecture and training procedure, which are advantageously considered jointly. In particular, we improve adapters and make it possible to learn an entire new language while ensuring that the output of the neural network is almost unchanged in the original domain. For this purpose, we modify the new residual blocks in a way that leads each new residual block to output near-zeros in the original domain. This solution of neutral residues, which borrows architectural components from mixture of experts, is effective: with only 20% extra learnable weights compared to an original model trained on English, we get results that are significantly better than concurrent approaches (fine-tuning, low-rank or vanilla adapters) in terms of the trade-off between learning a new language and not forgetting English.

We present RAISE-LPBF, a large dataset on the effect of laser power and laser dot speed in powder bed fusion (LPBF) of 316L stainless steel bulk material, monitored by on-axis 20k FPS video. Both process parameters are independently sampled for each scan line from a continuous distribution, so interactions of different parameter choices can be investigated. The data can be used to derive statistical properties of LPBF, as well as to build anomaly detectors. We provide example source code for loading the data, baseline machine learning models and results, and a public benchmark to evaluate predictive models., Comment: Dataset download: https://www.makebench.eu/

We introduce Moshi, a speech-text foundation model and full-duplex spoken dialogue framework. Current systems for spoken dialogue rely on pipelines of independent components, namely voice activity detection, speech recognition, textual dialogue and text-to-speech. Such frameworks cannot emulate the experience of real conversations. First, their complexity induces a latency of several seconds between interactions. Second, text being the intermediate modality for dialogue, non-linguistic information that modifies meaning -- such as emotion or non-speech sounds -- is lost in the interaction. Finally, they rely on a segmentation into speaker turns, which does not take into account overlapping speech, interruptions and interjections. Moshi solves these independent issues altogether by casting spoken dialogue as speech-to-speech generation. Starting from a text language model backbone, Moshi generates speech as tokens from the residual quantizer of a neural audio codec, while modeling separately its own speech and that of the user into parallel streams. This allows for the removal of explicit speaker turns, and the modeling of arbitrary conversational dynamics. We moreover extend the hierarchical semantic-to-acoustic token generation of previous work to first predict time-aligned text tokens as a prefix to audio tokens. Not only this "Inner Monologue" method significantly improves the linguistic quality of generated speech, but we also illustrate how it can provide streaming speech recognition and text-to-speech. Our resulting model is the first real-time full-duplex spoken large language model, with a theoretical latency of 160ms, 200ms in practice, and is available at https://github.com/kyutai-labs/moshi.

Large Language Models (LLMs) have demonstrated impressive capability in different tasks and are bringing transformative changes to many domains. However, keeping the knowledge in LLMs up-to-date remains a challenge once pretraining is complete. It is thus essential to design effective methods to both update obsolete knowledge and induce new knowledge into LLMs. Existing locate-and-edit knowledge editing (KE) method suffers from two limitations. First, the post-edit LLMs by such methods generally have poor capability in answering complex queries that require multi-hop reasoning. Second, the long run-time of such locate-and-edit methods to perform knowledge edits make it infeasible for large scale KE in practice. In this paper, we explore Parameter-Efficient Fine-Tuning (PEFT) techniques as an alternative for KE. We curate a more comprehensive temporal KE dataset with both knowledge update and knowledge injection examples for KE performance benchmarking. We further probe the effect of fine-tuning on a range of layers in an LLM for the multi-hop QA task. We find that PEFT performs better than locate-and-edit techniques for time-sensitive knowledge edits., Comment: ACL 2024 main

Representations of AI agents in user interfaces and robotics are predominantly White, not only in terms of facial and skin features, but also in the synthetic voices they use. In this paper we explore some unexpected challenges in the representation of race we found in the process of developing an U.S. English Text-to-Speech (TTS) system aimed to sound like an educated, professional, regional accent-free African American woman. The paper starts by presenting the results of focus groups with African American IT professionals where guidelines and challenges for the creation of a representative and appropriate TTS system were discussed and gathered, followed by a discussion about some of the technical difficulties faced by the TTS system developers. We then describe two studies with U.S. English speakers where the participants were not able to attribute the correct race to the African American TTS voice while overwhelmingly correctly recognizing the race of a White TTS system of similar quality. A focus group with African American IT workers not only confirmed the representativeness of the African American voice we built, but also suggested that the surprising recognition results may have been caused by the inability or the latent prejudice from non-African Americans to associate educated, non-vernacular, professionally-sounding voices to African American people., Comment: Full version including appendixes

High-dimensional expanders are a generalization of the notion of expander graphs to simplicial complexes and give rise to a variety of applications in computer science and other fields. We provide a general tool to construct families of bounded degree high-dimensional spectral expanders. Inspired by the work of Kaufman and Oppenheim, we use coset complexes over quotients of Kac-Moody-Steinberg groups of rank $d+1$, $d$-spherical and purely $d$-spherical. We prove that infinite families of such quotients exist provided that the underlying field is of size at least 4 and the Kac-Moody-Steinberg group is 2-spherical, giving rise to new families of bounded degree high-dimensional expanders. In the case the generalized Cartan matrix we consider is affine, we recover the construction of O'Donnell and Pratt from 2022, (and thus also the one of Kaufman and Oppenheim) by considering Chevalley groups as quotients of affine Kac-Moody-Steinberg groups. Moreover, our construction applies to the case where the root system is of type $\tilde{G}_2$, a case that was not covered in earlier works., Comment: 31 pages. Fixed a typo in section 4.3. To appear in the European Journal of Combinatorics

Scaling the size of language models to tens of billions of parameters has led to impressive performance on a wide range of tasks. At generation, these models are used auto-regressively, requiring a forward pass for each generated token, and thus reading the full set of parameters from memory. This memory access forms the primary bottleneck for generation and it worsens as the model size increases. Moreover, executing a forward pass for multiple tokens in parallel often takes nearly the same time as it does for just one token. These two observations lead to the development of speculative sampling, where a second smaller model is used to draft a few tokens, that are then validated or rejected using a single forward pass of the large model. Unfortunately, this method requires two models that share the same tokenizer and thus limits its adoption. As an alternative, we propose to use parallel decoding as a way to draft multiple tokens from a single model with no computational cost, nor the need for a second model. Our approach only requires an additional input token that marks the words that will be generated simultaneously. We show promising performance (up to $30\%$ speed-up) while requiring only as few as $O(d_{emb})$ additional parameters., Comment: Accepted at the 3rd workshop on Efficient Natural Language and Speech Processing (ENLSP, NeurIPS 2023)

Scientific data collected at ESO's observatories are freely and openly accessible online through the ESO Science Archive Facility. In addition to the raw data straight out of the instruments, the ESO Science Archive also contains four million processed science files available for use by scientists and astronomy enthusiasts worldwide. ESO subscribes to the FAIR (Findable, Accessible, Interoperable, Reusable) guiding principles for scientific data management and stewardship. All data in the ESO Science Archive are distributed according to the terms of the Creative Commons Attribution 4.0 International licence (CC BY 4.0)., Comment: 5 pages, 4 figures. arXiv admin note: text overlap with arXiv:2209.11605

The AGATA tracking detector array represents a significant improvement over previous Compton suppressed arrays. The construction of AGATA led to numerous technological breakthroughs in order to meet the requirements and the challenges of building a mobile detector across Europe. This paper focuses on the design and implementation of the data acquisition system responsible of the readout and control of the germanium detectors of AGATA. Our system is highly versatile, capable of instrumenting AGATA and seamlessly adapting it to various configurations with a wide range of ancillary detectors and/or spectrometers. It consists of three main components: an autonomous and independent infrastructure, a dedicated application core ensuring overall consistency, and a high--performance software package providing a fully integrated data flow management including the setting-up, the supervision and the slow control of the instrument. In this paper, we present a comprehensive analysis of the system's design and performance, particularly under high-counting rate conditions.

The present note studies the "graph of graphs" that has cubic graphs as vertices connected by edges represented by the so-called Whitehead moves. We study its conductance and expansion properties, and the adjacency spectrum.

Presently, gamma-ray tracking in germanium segmented detectors is realised by applying two advanced, complex algorithms. While they have already triggered an intensive R&D, they are still subject to further improvements. Making such algorithms effective, online in real time conditions and/or offline for deeper analysis, in data pipelines do require many additional software developments. This review paper gives an overview of the various bricks of software produced so far by the AGATA collaboration. It provides hints of what is foreseen for the next phases of the project up to its full configuration namely with 180 capsules in the array.

We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. We release all our models to the research community.

This survey reviews works in which language models (LMs) are augmented with reasoning skills and the ability to use tools. The former is defined as decomposing a potentially complex task into simpler subtasks while the latter consists in calling external modules such as a code interpreter. LMs can leverage these augmentations separately or in combination via heuristics, or learn to do so from demonstrations. While adhering to a standard missing tokens prediction objective, such augmented LMs can use various, possibly non-parametric external modules to expand their context processing ability, thus departing from the pure language modeling paradigm. We therefore refer to them as Augmented Language Models (ALMs). The missing token objective allows ALMs to learn to reason, use tools, and even act, while still performing standard natural language tasks and even outperforming most regular LMs on several benchmarks. In this work, after reviewing current advance in ALMs, we conclude that this new research direction has the potential to address common limitations of traditional LMs such as interpretability, consistency, and scalability issues.

Evaluation of text generation to date has primarily focused on content created sequentially, rather than improvements on a piece of text. Writing, however, is naturally an iterative and incremental process that requires expertise in different modular skills such as fixing outdated information or making the style more consistent. Even so, comprehensive evaluation of a model's capacity to perform these skills and the ability to edit remains sparse. This work presents EditEval: An instruction-based, benchmark and evaluation suite that leverages high-quality existing and new datasets for automatic evaluation of editing capabilities such as making text more cohesive and paraphrasing. We evaluate several pre-trained models, which shows that InstructGPT and PEER perform the best, but that most baselines fall below the supervised SOTA, particularly when neutralizing and updating information. Our analysis also shows that commonly used metrics for editing tasks do not always correlate well, and that optimization for prompts with the highest performance does not necessarily entail the strongest robustness to different models. Through the release of this benchmark and a publicly available leaderboard challenge, we hope to unlock future research in developing models capable of iterative and more controllable editing.

Textual content is often the output of a collaborative writing process: We start with an initial draft, ask for suggestions, and repeatedly make changes. Agnostic of this process, today's language models are trained to generate only the final result. As a consequence, they lack several abilities crucial for collaborative writing: They are unable to update existing texts, difficult to control and incapable of verbally planning or explaining their actions. To address these shortcomings, we introduce PEER, a collaborative language model that is trained to imitate the entire writing process itself: PEER can write drafts, add suggestions, propose edits and provide explanations for its actions. Crucially, we train multiple instances of PEER able to infill various parts of the writing process, enabling the use of self-training techniques for increasing the quality, amount and diversity of training data. This unlocks PEER's full potential by making it applicable in domains for which no edit histories are available and improving its ability to follow instructions, to write useful comments, and to explain its actions. We show that PEER achieves strong performance across various domains and editing tasks.

Modern computational science and engineering applications are being improved by the advances in scientific machine learning. Data-driven methods such as Dynamic Mode Decomposition (DMD) can extract coherent structures from spatio-temporal data generated from dynamical systems and infer different scenarios for said systems. The spatio-temporal data comes as snapshots containing spatial information for each time instant. In modern engineering applications, the generation of high-dimensional snapshots can be time and/or resource-demanding. In the present study, we consider two strategies for enhancing DMD workflow in large numerical simulations: (i) snapshots compression to relieve disk pressure; (ii) the use of in situ visualization images to reconstruct the dynamics (or part of) in runtime. We evaluate our approaches with two 3D fluid dynamics simulations and consider DMD to reconstruct the solutions. Results reveal that snapshot compression considerably reduces the required disk space. We have observed that lossy compression reduces storage by almost $50\%$ with low relative errors in the signal reconstructions and other quantities of interest. We also extend our analysis to data generated on-the-fly, using in-situ visualization tools to generate image files of our state vectors during runtime. On large simulations, the generation of snapshots may be slow enough to use batch algorithms for inference. Streaming DMD takes advantage of the incremental SVD algorithm and updates the modes with the arrival of each new snapshot. We use streaming DMD to reconstruct the dynamics from in-situ generated images. We show that this process is efficient, and the reconstructed dynamics are accurate., Comment: 24 pages, 19 figures, 5 tables

Large language models have shown impressive few-shot results on a wide range of tasks. However, when knowledge is key for such results, as is the case for tasks such as question answering and fact checking, massive parameter counts to store knowledge seem to be needed. Retrieval augmented models are known to excel at knowledge intensive tasks without the need for as many parameters, but it is unclear whether they work in few-shot settings. In this work we present Atlas, a carefully designed and pre-trained retrieval augmented language model able to learn knowledge intensive tasks with very few training examples. We perform evaluations on a wide range of tasks, including MMLU, KILT and NaturalQuestions, and study the impact of the content of the document index, showing that it can easily be updated. Notably, Atlas reaches over 42% accuracy on Natural Questions using only 64 examples, outperforming a 540B parameters model by 3% despite having 50x fewer parameters.

Verifiability is a core content policy of Wikipedia: claims that are likely to be challenged need to be backed by citations. There are millions of articles available online and thousands of new articles are released each month. For this reason, finding relevant sources is a difficult task: many claims do not have any references that support them. Furthermore, even existing citations might not support a given claim or become obsolete once the original source is updated or deleted. Hence, maintaining and improving the quality of Wikipedia references is an important challenge and there is a pressing need for better tools to assist humans in this effort. Here, we show that the process of improving references can be tackled with the help of artificial intelligence (AI). We develop a neural network based system, called Side, to identify Wikipedia citations that are unlikely to support their claims, and subsequently recommend better ones from the web. We train this model on existing Wikipedia references, therefore learning from the contributions and combined wisdom of thousands of Wikipedia editors. Using crowd-sourcing, we observe that for the top 10% most likely citations to be tagged as unverifiable by our system, humans prefer our system's suggested alternatives compared to the originally cited reference 70% of the time. To validate the applicability of our system, we built a demo to engage with the English-speaking Wikipedia community and find that Side's first citation recommendation collects over 60% more preferences than existing Wikipedia citations for the same top 10% most likely unverifiable claims according to Side. Our results indicate that an AI-based system could be used, in tandem with humans, to improve the verifiability of Wikipedia. More generally, we hope that our work can be used to assist fact checking efforts and increase the general trustworthiness of information online.

The outbreak of COVID-19, beginning in 2019 and continuing through the time of writing, has led to renewed interest in the mathematical modeling of infectious disease. Recent works have focused on partial differential equation (PDE) models, particularly reaction-diffusion models, able to describe the progression of an epidemic in both space and time. These studies have shown generally promising results in describing and predicting COVID-19 progression. However, people often travel long distances in short periods of time, leading to nonlocal transmission of the disease. Such contagion dynamics are not well-represented by diffusion alone. In contrast, ordinary differential equation (ODE) models may easily account for this behavior by considering disparate regions as nodes in a network, with the edges defining nonlocal transmission. In this work, we attempt to combine these modeling paradigms via the introduction of a network structure within a reaction-diffusion PDE system. This is achieved through the definition of a population-transfer operator, which couples disjoint and potentially distant geographic regions, facilitating nonlocal population movement between them. We provide analytical results demonstrating that this operator does not disrupt the physical consistency or mathematical well-posedness of the system, and verify these results through numerical experiments. We then use this technique to simulate the COVID-19 epidemic in the Brazilian region of Rio de Janeiro, showcasing its ability to capture important nonlocal behaviors, while maintaining the advantages of a reaction-diffusion model for describing local dynamics.

The computer simulation of organ-scale biomechanistic models of cancer personalized via routinely collected clinical and imaging data enables to obtain patient-specific predictions of tumor growth and treatment response over the anatomy of the patient's affected organ. These patient-specific computational forecasts have been regarded as a promising approach to personalize the clinical management of cancer and derive optimal treatment plans for individual patients, which constitute timely and critical needs in clinical oncology. However, the computer simulation of the underlying spatiotemporal models can entail a prohibitive computational cost, which constitutes a barrier to the successful development of clinically-actionable computational technologies for personalized tumor forecasting. To address this issue, here we propose to utilize Dynamic-Mode Decomposition (DMD) to construct a low-dimensional representation of cancer models and accelerate their simulation. DMD is an unsupervised machine learning method based on the singular value decomposition that has proven useful in many applications as both a predictive and a diagnostic tool. We show that DMD may be applied to Fisher-Kolmogorov models, which constitute an established formulation to represent untreated solid tumor growth that can further accommodate other relevant cancer phenomena. Our results show that a DMD implementation of this model over a clinically-relevant parameter space can yield impressive predictions, with short to medium-term errors remaining under 1% and long-term errors remaining under 20%, despite very short training periods. We posit that this data-driven approach has the potential to greatly reduce the computational overhead of personalized simulations of cancer models, thereby facilitating tumor forecasting, parameter identification, uncertainty quantification, and treatment optimization.

The modeling and simulation of two-phase flows is still an active research area, mainly when surface tension is present. One way to model the different phases is with interface capturing methods. Two well-established interface capturing approaches are the level-set and phase-field methods. The level-set method is known for its ability to compute the surface tension accurately, and phase-field models satisfy the second law of thermodynamics. This paper reviews and compares a level-set and a phase-field approach to simulate two-phase flows. We use a modified level-set method, called convected level-set. The difference to the standard level-set method is that the re-initialization step is embedded in the convection equation, avoiding a separate step during the calculation. We also apply a global mass conservation procedure to enforce the mass balance between phases. On the other hand, the phase-field approach uses a conservative Allen-Cahn equation with a Lagrange multiplier to conserve the mass. The methods are implemented in libMesh, a parallel adaptive finite element library, using the same finite element formulations, time-marching schemes, solvers, and mesh adaptivity strategies. We present numerical solutions for the two methods considering adaptive mesh refinement and coarsening (AMR/C). Results are analyzed and discussed., Comment: 26 pages, 21 figures. arXiv admin note: text overlap with arXiv:1710.09831, arXiv:2007.15887 by other authors

Resonant Inelastic X-Ray Scattering (RIXS) spectra of hematite were measured at the Fe L3-edge for heteroepitaxial thin films which were undoped and doped with 1% Ti, Sn or Zn, in the energy loss range in excess of 1 eV to study electronic transitions. The spectra were measured for several momentum transfers (q), conducted at both low temperature (T=14K) and room temperature. While we can not rule out dispersive features possibly owing to propagating excitations, the coarse envelopes of the general spectra did not appreciably change shape with q, implying that the bulk of the observed L-edge RIXS intensity originates from (mostly) non-dispersive ligand field (LF) excitations. Summing the RIXS spectra over q and comparing the results at T=14 K to those at T=300 K, revealed pronounced temperature effects, including an intensity change and energy shift of the 1.4 eV peak, a broadband intensity increase of the 3-4 eV range, and higher energy features. The q-summed spectra and their temperature dependences are virtually identical for nearly all of the samples with different dopants, save for the temperature dependence of the Ti-doped sample's spectrum, which we attribute to being affected by a large number of free charge carriers. Comparing with magnetization measurements for different temperatures and dopings likewise did not show a clear correlation between the RIXS spectra and the magnetic ordering states. To clarify the excited states, we performed spin multiplet calculations which were in excellent agreement with the RIXS spectra over a wide energy range and provide detailed electronic descriptions of the excited states. The implications of these findings to the photoconversion efficiency of hematite photoanodes is discussed., Comment: 11 pages, 5 figures, and .pdf supplementary file

As the computational requirements for machine learning systems and the size and complexity of machine learning frameworks increases, essential framework innovation has become challenging. While computational needs have driven recent compiler, networking, and hardware advancements, utilization of those advancements by machine learning tools is occurring at a slower pace. This is in part due to the difficulties involved in prototyping new computational paradigms with existing frameworks. Large frameworks prioritize machine learning researchers and practitioners as end users and pay comparatively little attention to systems researchers who can push frameworks forward -- we argue that both are equally important stakeholders. We introduce Flashlight, an open-source library built to spur innovation in machine learning tools and systems by prioritizing open, modular, customizable internals and state-of-the-art, research-ready models and training setups across a variety of domains. Flashlight allows systems researchers to rapidly prototype and experiment with novel ideas in machine learning computation and has low overhead, competing with and often outperforming other popular machine learning frameworks. We see Flashlight as a tool enabling research that can benefit widely used libraries downstream and bring machine learning and systems researchers closer together. Flashlight is available at https://github.com/flashlight/flashlight ., Comment: Presented at ICML 2022

Pre-training models on large scale datasets, like ImageNet, is a standard practice in computer vision. This paradigm is especially effective for tasks with small training sets, for which high-capacity models tend to overfit. In this work, we consider a self-supervised pre-training scenario that only leverages the target task data. We consider datasets, like Stanford Cars, Sketch or COCO, which are order(s) of magnitude smaller than Imagenet. Our study shows that denoising autoencoders, such as BEiT or a variant that we introduce in this paper, are more robust to the type and size of the pre-training data than popular self-supervised methods trained by comparing image embeddings.We obtain competitive performance compared to ImageNet pre-training on a variety of classification datasets, from different domains. On COCO, when pre-training solely using COCO images, the detection and instance segmentation performance surpasses the supervised ImageNet pre-training in a comparable setting.

In order to address increasing demands of real-world applications, the research for knowledge-intensive NLP (KI-NLP) should advance by capturing the challenges of a truly open-domain environment: web-scale knowledge, lack of structure, inconsistent quality and noise. To this end, we propose a new setup for evaluating existing knowledge intensive tasks in which we generalize the background corpus to a universal web snapshot. We investigate a slate of NLP tasks which rely on knowledge - either factual or common sense, and ask systems to use a subset of CCNet - the Sphere corpus - as a knowledge source. In contrast to Wikipedia, otherwise a common background corpus in KI-NLP, Sphere is orders of magnitude larger and better reflects the full diversity of knowledge on the web. Despite potential gaps in coverage, challenges of scale, lack of structure and lower quality, we find that retrieval from Sphere enables a state of the art system to match and even outperform Wikipedia-based models on several tasks. We also observe that while a dense index can outperform a sparse BM25 baseline on Wikipedia, on Sphere this is not yet possible. To facilitate further research and minimise the community's reliance on proprietary, black-box search engines, we share our indices, evaluation metrics and infrastructure.

Recently, information retrieval has seen the emergence of dense retrievers, using neural networks, as an alternative to classical sparse methods based on term-frequency. These models have obtained state-of-the-art results on datasets and tasks where large training sets are available. However, they do not transfer well to new applications with no training data, and are outperformed by unsupervised term-frequency methods such as BM25. In this work, we explore the limits of contrastive learning as a way to train unsupervised dense retrievers and show that it leads to strong performance in various retrieval settings. On the BEIR benchmark our unsupervised model outperforms BM25 on 11 out of 15 datasets for the Recall@100. When used as pre-training before fine-tuning, either on a few thousands in-domain examples or on the large MS~MARCO dataset, our contrastive model leads to improvements on the BEIR benchmark. Finally, we evaluate our approach for multi-lingual retrieval, where training data is even scarcer than for English, and show that our approach leads to strong unsupervised performance. Our model also exhibits strong cross-lingual transfer when fine-tuned on supervised English data only and evaluated on low resources language such as Swahili. We show that our unsupervised models can perform cross-lingual retrieval between different scripts, such as retrieving English documents from Arabic queries, which would not be possible with term matching methods.

Dynamic Mode Decomposition (DMD) is an unsupervised machine learning method that has attracted considerable attention in recent years owing to its equation-free structure, ability to easily identify coherent spatio-temporal structures in data, and effectiveness in providing reasonably accurate predictions for certain problems. Despite these successes, the application of DMD to certain problems featuring highly nonlinear transient dynamics remains challenging. In such cases, DMD may not only fail to provide acceptable predictions but may indeed fail to recreate the data in which it was trained, restricting its application to diagnostic purposes. For many problems in the biological and physical sciences, the structure of the system obeys a compartmental framework, in which the transfer of mass within the system moves within states. In these cases, the behavior of the system may not be accurately recreated by applying DMD to a single quantity within the system, as proper knowledge of the system dynamics, even for a single compartment, requires that the behavior of other compartments is taken into account in the DMD process. In this work, we demonstrate, theoretically and numerically, that, when performing DMD on a fully coupled PDE system with compartmental structure, one may recover useful predictive behavior, even when DMD performs poorly when acting compartment-wise. We also establish that important physical quantities, as mass conservation, are maintained in the coupled-DMD extrapolation. The mathematical and numerical analysis suggests that DMD may be a powerful tool when applied to this common class of problems. In particular, we show interesting numerical applications to a continuous delayed-SIRD model for Covid-19, and to a problem from additive manufacturing considering a nonlinear temperature field and the resulting change of material phase from powder, liquid, and solid states.

The Dzyaloshinskii-Moriya interaction (DMI) is at the heart of many modern developments in the research field of spintronics. DMI is known to generate noncollinear magnetic textures, and can take two forms in antiferromagnets: homogeneous or inter-sublattice, leading to small, canted moments and inhomogeneous or intra-sublattice, leading to formation of chiral structures. In this work, we first determine the strength of the effective field created by the DMI, using SQUID based magnetometry and transport measurements, in thin films of the antiferromagnetic iron oxide hematite, $\alpha$-Fe$_2$O$_3$. We demonstrate that DMI additionally introduces reconfigurability in the long distance magnon transport in these films under different orientations of a magnetic field. This arises as a hysteresis centred around the easy-axis direction for an external field rotated in opposing directions whose width decreases with increasing magnetic field as the Zeeman energy competes with the effective field created by the DMI.

We present ResMLP, an architecture built entirely upon multi-layer perceptrons for image classification. It is a simple residual network that alternates (i) a linear layer in which image patches interact, independently and identically across channels, and (ii) a two-layer feed-forward network in which channels interact independently per patch. When trained with a modern training strategy using heavy data-augmentation and optionally distillation, it attains surprisingly good accuracy/complexity trade-offs on ImageNet. We also train ResMLP models in a self-supervised setup, to further remove priors from employing a labelled dataset. Finally, by adapting our model to machine translation we achieve surprisingly good results. We share pre-trained models and our code based on the Timm library.

Dynamic Mode Decomposition (DMD) is a powerful data-driven method used to extract spatio-temporal coherent structures that dictate a given dynamical system. The method consists of stacking collected temporal snapshots into a matrix and mapping the nonlinear dynamics using a linear operator. The standard procedure considers that snapshots possess the same dimensionality for all the observable data. However, this often does not occur in numerical simulations with adaptive mesh refinement/coarsening schemes (AMR/C). This paper proposes a strategy to enable DMD to extract features from observations with different mesh topologies and dimensions, such as those found in AMR/C simulations. For this purpose, the adaptive snapshots are projected onto the same reference function space, enabling the use of snapshot-based methods such as DMD. The present strategy is applied to challenging AMR/C simulations: a continuous diffusion-reaction epidemiological model for COVID-19, a density-driven gravity current simulation, and a bubble rising problem. We also evaluate the DMD efficiency to reconstruct the dynamics and some relevant quantities of interest. In particular, for the SEIRD model and the bubble rising problem, we evaluate DMD's ability to extrapolate in time (short-time future estimates).

The Di{\'o}si-Penrose model is explored in a relativistic context. Relativistic effects were considered within a recently proposed Grave de Peralta approach [L. Grave de Peralta, {\em Results Phys.} {\bf 18} (2020) 103318], which parametrize the Schr{\"o}dinger-like hamiltonian so as to impose that the average kinetic energy of the system coincide with its relativistic kinetic energy. As a case of study, the method is applied to a particle in a box with good results. In the Di{\'o}si-Penrose model we observed that the width of a quantum matter field confined by its own gravitational field [L. Di{\'o}si, {\em Phys. Lett}. {\bf 105A} (1984) 199], sharply drop to zero for a mass of the order of the Planck mass, indicating a breakdown of the model at the Planck scale.