Search

In this study, we investigated the oxidation of the Mg-11Y-1Al alloy at 500{\deg}C in an Ar-20%O2 environment. Multiscale analysis showed the network-like long-period stacking ordered (LPSO) phase transformed into needle-like LPSO and polygonal Mg24Y5 phases, leading to the formation of a high-dense network of needle-like oxides at the oxidation front. These oxides grew laterally along the oxide/matrix interfaces, forming a thicker, continuous scale that effectively blocked elemental diffusion. Hence, the preferential oxidation along the needle-like LPSO is believed to accelerate the formation of a thicker and continuous oxide scale, further improving the oxidation resistance of the Mg-11Y-1Al alloy.

FP8 formats are gaining popularity to boost the computational efficiency for training and inference of large deep learning models. Their main challenge is that a careful choice of scaling is needed to prevent degradation due to the reduced dynamic range compared to higher-precision formats. Although there exists ample literature about selecting such scalings for INT formats, this critical aspect has yet to be addressed for FP8. This paper presents a methodology to select the scalings for FP8 linear layers, based on dynamically updating per-tensor scales for the weights, gradients and activations. We apply this methodology to train and validate large language models of the type of GPT and Llama 2 using FP8, for model sizes ranging from 111M to 70B. To facilitate the understanding of the FP8 dynamics, our results are accompanied by plots of the per-tensor scale distribution for weights, activations and gradients during both training and inference.

System identification, also known as learning forward models, transfer functions, system dynamics, etc., has a long tradition both in science and engineering in different fields. Particularly, it is a recurring theme in Reinforcement Learning research, where forward models approximate the state transition function of a Markov Decision Process by learning a mapping function from current state and action to the next state. This problem is commonly defined as a Supervised Learning problem in a direct way. This common approach faces several difficulties due to the inherent complexities of the dynamics to learn, for example, delayed effects, high non-linearity, non-stationarity, partial observability and, more important, error accumulation when using bootstrapped predictions (predictions based on past predictions), over large time horizons. Here we explore the use of Reinforcement Learning in this problem. We elaborate on why and how this problem fits naturally and sound as a Reinforcement Learning problem, and present some experimental results that demonstrate RL is a promising technique to solve these kind of problems., Comment: Accepted in Neurips Deep Reinforcement Learning Workshop 2022: https://openreview.net/forum?id=fGcbpWQIJZV

Effective training of today's large language models (LLMs) depends on large batches and long sequences for throughput and accuracy. To handle variable-length sequences on hardware accelerators, it is common practice to introduce padding tokens, so that all sequences in a batch have the same length. We show in this paper that the variation in sequence lengths in common NLP datasets is such that up to 50% of all tokens can be padding. In less common, but not extreme, cases (e.g. GLUE-cola with sequence length 128), the ratio is up to 89%. Existing methods to address the resulting inefficiency are complicated by the need to avoid cross-contamination in self-attention, by a reduction in accuracy when sequence ordering information is lost, or by customized kernel implementations only valid for specific accelerators. This paper introduces a new formalization of sequence packing in the context of the well-studied bin packing problem, and presents new algorithms based on this formulation which, for example, confer a 2x speedup for phase 2 pre-training in BERT. We show how existing models can be adapted to ensure mathematical equivalence between the original and packed models, meaning that packed models can be trained with existing pre-training and fine-tuning practices., Comment: Significantly new version with different authors and much more content. Much larger variety in experiments and exhaustive SOTA analysis

We propose finite-volume schemes for the Cahn-Hilliard equation which unconditionally and discretely preserve the boundedness of the phase field and the dissipation of the free energy. Our numerical framework is applicable to a variety of free-energy potentials, including Ginzburg-Landau and Flory-Huggins, to general wetting boundary conditions, and to degenerate mobilities. Its central thrust is the upwind methodology, which we combine with a semi-implicit formulation for the free-energy terms based on the classical convex-splitting approach. The extension of the schemes to an arbitrary number of dimensions is straightforward thanks to their dimensionally split nature, which allows to efficiently solve higher-dimensional problems with a simple parallelisation. The numerical schemes are validated and tested through a variety of examples, in different dimensions, and with various contact angles between droplets and substrates.

We assess the benefit of including an image inpainting filter before passing damaged images into a classification neural network. For this we employ a modified Cahn-Hilliard equation as an image inpainting filter, which is solved via a finite volume scheme with reduced computational cost and adequate properties for energy stability and boundedness. The benchmark dataset employed here is MNIST, which consists of binary images of handwritten digits and is a standard dataset to validate image-processing methodologies. We train a neural network based of dense layers with the training set of MNIST, and subsequently we contaminate the test set with damage of different types and intensities. We then compare the prediction accuracy of the neural network with and without applying the Cahn-Hilliard filter to the damaged images test. Our results quantify the significant improvement of damaged-image prediction due to applying the Cahn-Hilliard filter, which for specific damages can increase up to 50% and is in general advantageous for low to moderate damage., Comment: An interactive jupyter notebook with the code of this work is available at https://github.com/sergiopperez/Image_Inpainting. The MNIST dataset employed in this work can be downloaded from http://yann.lecun.com/exdb/mnist/

In this work, strain and geometric necessary dislocations (GNDs) have been mapped with nm-resolution around grain boundaries affected by stress corrosion cracking (SCC) or intergranular oxidation with the aim of clarifying which local conditions trigger SCC initiation., Comment: 36 pages, 19 figures

We propose high-order well-balanced finite-volume schemes for a broad class of hydrodynamic systems with attractive-repulsive interaction forces and linear and nonlinear damping. Our schemes are suitable for free energies containing convolutions of an interaction potential with the density, which are essential for applications such as the Keller-Segel model, more general Euler-Poisson systems, or dynamic-density functional theory. Our schemes are also equipped with a nonnegative-density reconstruction which allows for vacuum regions during the simulation. We provide several prototypical examples from relevant applications highlighting the benefit of our algorithms elucidate also some of our analytical results.

We introduce a finite-volume numerical scheme for solving stochastic gradient-flow equations. Such equations are of crucial importance within the framework of fluctuating hydrodynamics and dynamic density functional theory. Our proposed scheme deals with general free-energy functionals, including, for instance, external fields or interaction potentials. This allows us to simulate a range of physical phenomena where thermal fluctuations play a crucial role, such as nucleation and other energy-barrier crossing transitions. A positivity-preserving algorithm for the density is derived based on a hybrid space discretization of the deterministic and the stochastic terms and different implicit and explicit time integrators. We show through numerous applications that not only our scheme is able to accurately reproduce the statistical properties (structure factor and correlations) of the physical system, but, because of the multiplicative noise, it allows us to simulate energy barrier crossing dynamics, which cannot be captured by mean-field approaches.

In this study, we have used a Zr-Nb alloy containing well-defined nano-precipitates as a model material in which to study imaging contrast inversions (atomic number or diffraction contrast) observed with the forescattered electron imaging system, ARGUSTM, in a scanning electron microscope (SEM) when imaging a thin foil in a transmission geometry. The study is based on Monte Carlo simulations and analysis of micrographs experimentally acquired under different imaging conditions. Based on the results, imaging conditions that enhance atomic number or diffraction contrast have been proposed. Data acquired from the ARGUSTM imaging system in SEM has also been compared with results from standard transmission electron microscopy and scanning transmission electron microscopy imaging of the same material. These results demonstrate the capability of the ARGUSTM system to investigate microstructures in nano-scale materials.