Search

In this study, we investigated the oxidation of the Mg-11Y-1Al alloy at 500{\deg}C in an Ar-20%O2 environment. Multiscale analysis showed the network-like long-period stacking ordered (LPSO) phase transformed into needle-like LPSO and polygonal Mg24Y5 phases, leading to the formation of a high-dense network of needle-like oxides at the oxidation front. These oxides grew laterally along the oxide/matrix interfaces, forming a thicker, continuous scale that effectively blocked elemental diffusion. Hence, the preferential oxidation along the needle-like LPSO is believed to accelerate the formation of a thicker and continuous oxide scale, further improving the oxidation resistance of the Mg-11Y-1Al alloy.

FP8 formats are gaining popularity to boost the computational efficiency for training and inference of large deep learning models. Their main challenge is that a careful choice of scaling is needed to prevent degradation due to the reduced dynamic range compared to higher-precision formats. Although there exists ample literature about selecting such scalings for INT formats, this critical aspect has yet to be addressed for FP8. This paper presents a methodology to select the scalings for FP8 linear layers, based on dynamically updating per-tensor scales for the weights, gradients and activations. We apply this methodology to train and validate large language models of the type of GPT and Llama 2 using FP8, for model sizes ranging from 111M to 70B. To facilitate the understanding of the FP8 dynamics, our results are accompanied by plots of the per-tensor scale distribution for weights, activations and gradients during both training and inference.

System identification, also known as learning forward models, transfer functions, system dynamics, etc., has a long tradition both in science and engineering in different fields. Particularly, it is a recurring theme in Reinforcement Learning research, where forward models approximate the state transition function of a Markov Decision Process by learning a mapping function from current state and action to the next state. This problem is commonly defined as a Supervised Learning problem in a direct way. This common approach faces several difficulties due to the inherent complexities of the dynamics to learn, for example, delayed effects, high non-linearity, non-stationarity, partial observability and, more important, error accumulation when using bootstrapped predictions (predictions based on past predictions), over large time horizons. Here we explore the use of Reinforcement Learning in this problem. We elaborate on why and how this problem fits naturally and sound as a Reinforcement Learning problem, and present some experimental results that demonstrate RL is a promising technique to solve these kind of problems., Comment: Accepted in Neurips Deep Reinforcement Learning Workshop 2022: https://openreview.net/forum?id=fGcbpWQIJZV

Effective training of today's large language models (LLMs) depends on large batches and long sequences for throughput and accuracy. To handle variable-length sequences on hardware accelerators, it is common practice to introduce padding tokens, so that all sequences in a batch have the same length. We show in this paper that the variation in sequence lengths in common NLP datasets is such that up to 50% of all tokens can be padding. In less common, but not extreme, cases (e.g. GLUE-cola with sequence length 128), the ratio is up to 89%. Existing methods to address the resulting inefficiency are complicated by the need to avoid cross-contamination in self-attention, by a reduction in accuracy when sequence ordering information is lost, or by customized kernel implementations only valid for specific accelerators. This paper introduces a new formalization of sequence packing in the context of the well-studied bin packing problem, and presents new algorithms based on this formulation which, for example, confer a 2x speedup for phase 2 pre-training in BERT. We show how existing models can be adapted to ensure mathematical equivalence between the original and packed models, meaning that packed models can be trained with existing pre-training and fine-tuning practices., Comment: Significantly new version with different authors and much more content. Much larger variety in experiments and exhaustive SOTA analysis

We propose finite-volume schemes for the Cahn-Hilliard equation which unconditionally and discretely preserve the boundedness of the phase field and the dissipation of the free energy. Our numerical framework is applicable to a variety of free-energy potentials, including Ginzburg-Landau and Flory-Huggins, to general wetting boundary conditions, and to degenerate mobilities. Its central thrust is the upwind methodology, which we combine with a semi-implicit formulation for the free-energy terms based on the classical convex-splitting approach. The extension of the schemes to an arbitrary number of dimensions is straightforward thanks to their dimensionally split nature, which allows to efficiently solve higher-dimensional problems with a simple parallelisation. The numerical schemes are validated and tested through a variety of examples, in different dimensions, and with various contact angles between droplets and substrates.

We assess the benefit of including an image inpainting filter before passing damaged images into a classification neural network. For this we employ a modified Cahn-Hilliard equation as an image inpainting filter, which is solved via a finite volume scheme with reduced computational cost and adequate properties for energy stability and boundedness. The benchmark dataset employed here is MNIST, which consists of binary images of handwritten digits and is a standard dataset to validate image-processing methodologies. We train a neural network based of dense layers with the training set of MNIST, and subsequently we contaminate the test set with damage of different types and intensities. We then compare the prediction accuracy of the neural network with and without applying the Cahn-Hilliard filter to the damaged images test. Our results quantify the significant improvement of damaged-image prediction due to applying the Cahn-Hilliard filter, which for specific damages can increase up to 50% and is in general advantageous for low to moderate damage., Comment: An interactive jupyter notebook with the code of this work is available at https://github.com/sergiopperez/Image_Inpainting. The MNIST dataset employed in this work can be downloaded from http://yann.lecun.com/exdb/mnist/

In this work, strain and geometric necessary dislocations (GNDs) have been mapped with nm-resolution around grain boundaries affected by stress corrosion cracking (SCC) or intergranular oxidation with the aim of clarifying which local conditions trigger SCC initiation., Comment: 36 pages, 19 figures

We propose high-order well-balanced finite-volume schemes for a broad class of hydrodynamic systems with attractive-repulsive interaction forces and linear and nonlinear damping. Our schemes are suitable for free energies containing convolutions of an interaction potential with the density, which are essential for applications such as the Keller-Segel model, more general Euler-Poisson systems, or dynamic-density functional theory. Our schemes are also equipped with a nonnegative-density reconstruction which allows for vacuum regions during the simulation. We provide several prototypical examples from relevant applications highlighting the benefit of our algorithms elucidate also some of our analytical results.

We introduce a finite-volume numerical scheme for solving stochastic gradient-flow equations. Such equations are of crucial importance within the framework of fluctuating hydrodynamics and dynamic density functional theory. Our proposed scheme deals with general free-energy functionals, including, for instance, external fields or interaction potentials. This allows us to simulate a range of physical phenomena where thermal fluctuations play a crucial role, such as nucleation and other energy-barrier crossing transitions. A positivity-preserving algorithm for the density is derived based on a hybrid space discretization of the deterministic and the stochastic terms and different implicit and explicit time integrators. We show through numerous applications that not only our scheme is able to accurately reproduce the statistical properties (structure factor and correlations) of the physical system, but, because of the multiplicative noise, it allows us to simulate energy barrier crossing dynamics, which cannot be captured by mean-field approaches.

In this study, we have used a Zr-Nb alloy containing well-defined nano-precipitates as a model material in which to study imaging contrast inversions (atomic number or diffraction contrast) observed with the forescattered electron imaging system, ARGUSTM, in a scanning electron microscope (SEM) when imaging a thin foil in a transmission geometry. The study is based on Monte Carlo simulations and analysis of micrographs experimentally acquired under different imaging conditions. Based on the results, imaging conditions that enhance atomic number or diffraction contrast have been proposed. Data acquired from the ARGUSTM imaging system in SEM has also been compared with results from standard transmission electron microscopy and scanning transmission electron microscopy imaging of the same material. These results demonstrate the capability of the ARGUSTM system to investigate microstructures in nano-scale materials.

In this paper, we have analysed the depth resolution that can be achieved by on-axis transmission Kikuchi diffraction (TKD) using a Zr-Nb alloy. The results indicate that the signals contributing to detectable Kikuchi bands originate from a depth of approximately the mean free path of thermal diffuse scattering (lambdaTDS) from the bottom surface of a thin foil sample. This existing surface sensitivity can thus lead to the observation of different grain structures when opposite sides of a nano-crystalline foil are facing the incident electron beam. These results also provide a guideline for the ideal sample thickness for TKD analysis of 6lambdaTDS, or 21 times the elastic scattering mean free path (lambdaMFP) for samples of high crystal symmetry. For samples of lower symmetry, a smaller thickness 3lambdaTDS, or 10lambdaMFP is suggested.

Well balanced and free energy dissipative first- and second-order accurate finite volume schemes are proposed for a general class of hydrodynamic systems with linear and nonlinear damping. The natural Liapunov functional of the system, given by its free energy, allows for a characterization of the stationary states by its variation. An analog property at the discrete level enables us to preserve stationary states at machine precision while keeping the dissipation of the discrete free energy. These schemes allow for analysing accurately the stability properties of stationary states in challeging problems such as: phase transitions in collective behavior, generalized Euler-Poisson systems in chemotaxis and astrophysics, and models in dynamic density functional theories; having done a careful validation in a battery of relevant test cases., Comment: Videos from the simulations of this work are available at https://figshare.com/projects/Well-balanced_finite_volume_schemes_for_hydrodynamic_equations_with_general_free_energy/60122

We give a detailed review of two algorithms that solve the minimization case of the assignment problem. The Bertsekas' auction algorithm and the Goldberg & Kennedy algorithm. We will show that these algorithms are equivalent in the sense that both perform equivalent steps in the same order. We also present experimental results comparing the performance of three algorithms for the assignment problem. They show the auction algorithm performs and scales better in practice than algorithms that are harder to implement but have better theoretical time complexity., Comment: 21 pages, 6 figures

Recently, a new approach to perform regression testing has been defined: the point of interest (POI) testing. A POI, in this context, is any expression of a program. The approach receives as input a set of relations between POIs from a version of a program and POIs from another version, and also a sequence of input functions, i.e. test cases. Then, a program instrumentation, an input test case generation and different comparison functions are used to obtain the final report which indicates whether the alternative version of the program behaves as expected, e.g. it produces the same values or it uses less CPU/memory. In this paper, we explain how we can improve the POI testing approach through the use of common stack traces and a more sophisticated tracing for calls. These enhancements of the approach allow users to identify errors earlier and easier. Additionally, they enable new comparison modes and new categories of reported unexpected behaviours., Comment: 29 pages, Accepted for presentation in WFLP 2018

During its lifetime, a program suffers several changes that seek to improve or to augment some parts of its functionality. However, these modifications usually also introduce errors that affect the already-working code. There are several approaches and tools that help to spot and find the source of these errors. However, most of these errors could be avoided beforehand by using some of the knowledge that the programmers had when they were writing the code. This is the idea behind the design-by-contract approach, where users can define contracts that can be checked during runtime. In this paper, we apply the principles of this approach to Erlang, enabling, in this way, a runtime verification system in this language. We define two types of contracts. One of them can be used in any Erlang program, while the second type is intended to be used only in concurrent programs. We provide the details of the implementation of both types of contracts. Moreover, we provide an extensive explanation of each contract as well as examples of their usage. All the ideas presented in this paper have been implemented in a contract-based runtime verification system named EDBC. Its source code is available at GitHub as an open-source and free project., Comment: 19 pages, Accepted for presentation in WFLP 2018

The main goal of this work is to show how SecEr can be used in different scenarios. Concretely, we demonstrate how a user can run SecEr to obtain reports about the behaviour preservation between versions as well as how a user can use SecEr to find the source of a discrepancy. The use cases presented are three: two completely different versions of the same program, an improvement in the performance of a function and a program where an error has been introduced. A complete description of the technique and the tool is available at [1] and [2].

In this paper, we present an automated feature engineering based approach to dramatically reduce false positives in fraud prediction. False positives plague the fraud prediction industry. It is estimated that only 1 in 5 declared as fraud are actually fraud and roughly 1 in every 6 customers have had a valid transaction declined in the past year. To address this problem, we use the Deep Feature Synthesis algorithm to automatically derive behavioral features based on the historical data of the card associated with a transaction. We generate 237 features (>100 behavioral patterns) for each transaction, and use a random forest to learn a classifier. We tested our machine learning model on data from a large multinational bank and compared it to their existing solution. On an unseen data of 1.852 million transactions, we were able to reduce the false positives by 54% and provide a savings of 190K euros. We also assess how to deploy this solution, and whether it necessitates streaming computation for real time scoring. We found that our solution can maintain similar benefits even when historical features are computed once every 7 days.

During the software lifecycle, a program can evolve several times for different reasons such as the optimisation of a bottle-neck, the refactoring of an obscure function, etc. These code changes often involve several functions or modules, so it can be difficult to know whether the correct behaviour of the previous releases has been preserved in the new release. Most developers rely on a previously defined test suite to check this behaviour preservation. We propose here an alternative approach to automatically obtain a test suite that specifically focusses on comparing the old and new versions of the code. Our test case generation is directed by a sophisticated combination of several already existing tools such as TypEr, CutEr, and PropEr; and other ideas such as allowing the programmer to chose an expression of interest that must preserve the behaviour, or the recording of the sequences of values to which this expression is evaluated. All the presented work has been implemented in an open-source tool that is publicly available on GitHub., Comment: Pre-proceedings paper presented at the 27th International Symposium on Logic-Based Program Synthesis and Transformation (LOPSTR 2017), Namur, Belgium, 10-12 October 2017 (arXiv:1708.07854)

In this paper we prove that if $E$ and $F$ are reflexive Banach spaces and $G$ is a closed linear subspace of the space $\mathcal{P}_{w}(^{n}E;F)$ of all $n$-homogeneous polynomials from $E$ to $F$ which are weakly continuous on bounded sets, then $G$ is either reflexive or non-isomorphic to a dual space. This result generalizes \cite[Theorem 2]{FEDER} and gives the solution to a problem posed by Feder \cite[Problem 1]{FED}., Comment: 4 pages