Search

Various reversible deep neural networks (DNN) models have been proposed to reduce memory consumption in the training process. However, almost all existing reversible DNNs either require special non-standard architectures or are constructed by modifying existing DNN architectures considerably to enable reversibility. In this work we present the BDIA-transformer, which is an exact bit-level reversible transformer that uses an unchanged standard architecture for inference. The basic idea is to first treat each transformer block as the Euler integration approximation for solving an ordinary differential equation (ODE) and then incorporate the technique of bidirectional integration approximation (BDIA) into the neural architecture, together with activation quantization to make it exactly bit-level reversible. In the training process, we let a hyper-parameter $\gamma$ in BDIA-transformer randomly take one of the two values $\{0.5, -0.5\}$ per training sample per transformer block for averaging every two consecutive integration approximations. As a result, BDIA-transformer can be viewed as training an ensemble of ODE solvers parameterized by a set of binary random variables, which regularizes the model and results in improved validation accuracy. Lightweight side information per transformer block is required to be stored in the forward process to account for binary quantization loss to enable exact bit-level reversibility. In the inference procedure, the expectation $\mathbb{E}(\gamma)=0$ is taken to make the resulting architectures of BDIA-transformer identical to transformers up to activation quantization. Our experiments in both image classification and language translation show that BDIA-transformers outperform their conventional counterparts significantly in terms of validation performance while also requiring considerably less training memory.

Following long-standing predictions associated with hydrogen, high-temperature superconductivity has recently been observed in several hydride-based materials. Nevertheless, these high-$T_c$ phases only exist at extremely high pressures, and achieving high transition temperatures at ambient pressure remains a major challenge. Recent predictions of the complex hydride Mg$_{2}$IrH$_{6}$ may help overcome this challenge with calculations of high-$T_c$ superconductivity (65 K$~<~T_c~<~$ 170 K) in a material that is stable at atmospheric pressure. In this work, the synthesis of Mg$_{2}$IrH$_{6}$ was targeted over a broad range of $P$-$T$ conditions, and the resulting products were characterized using X-ray diffraction (XRD) and vibrational spectroscopy, in concert with first-principles calculations. The results indicate that the charge-balanced complex hydride Mg$_{2}$IrH$_{5}$ is more stable over all conditions tested up to ca 28 GPa. The resulting hydride is isostructural with the predicted superconducting Mg$_{2}$IrH$_{6}$ phase except for a single hydrogen vacancy, which shows a favorable replacement barrier upon insertion of hydrogen into the lattice. Bulk Mg$_{2}$IrH$_{5}$ is readily accessible at mild $P$-$T$ conditions and may thus represent a convenient platform to access superconducting Mg$_{2}$IrH$_{6}$ via non-equilibrium processing methods., Comment: 18 pages, 17 Figures

The LBNF Absorber consists of thirteen 6061-T6 aluminum core blocks. The core blocks are water cooled with de-ionized (DI) water which becomes radioactive during beam operations. The cooling water flows through gun-drilled channels in the core blocks. The cooling water is supplied by the LBNF Absorber Radioactive Water (RAW) cooling system which is designed as per ASME B31.3 Normal Fluid Service [1]. An uninhibited beam accident pulse striking the water channels was identified as a credible accident scenario. In this study, it is assumed that the beam pulse hits the Absorber directly without interacting with any of the other upstream beamline components. The beam parameters used for the LBNF beam are 120 GeV, 2.4 MW with a 1.2 s cycle time. The accident pulse lasts for 10 {\mu}s. The maximum energy is deposited in the 3rd aluminum core block. For the sake of simplicity, it is assumed that the accident pulse strikes the 1 in. ID water channel directly. The analysis here simulates the pressure rise in the water during and after the beam pulse and its effects on the aluminum piping components that deliver water to the core blocks. The weld strengths as determined by the Load and Resistance Factor Design (LRDF) and the Allowable Strength Design (ASD) are compared to the forces generated in the weld owing to the pressure spike. A transient structural analysis was used to determine the equivalent membrane, peak, and bending stresses and they were com-pared to allowable limits., Comment: IPAC'24 - 15th International Particle Accelerator Conference

The Gaia-ESO Survey is an European Southern Observatory (ESO) public spectroscopic survey that targeted $10^5$ stars in the Milky Way covering the major populations of the disk, bulge and halo. The observations were made using FLAMES on the VLT obtaining both UVES high ($R\sim47,000$) and GIRAFFE medium ($R\sim20,000$) resolution spectra. The analysis of the Gaia-ESO spectra was the work of multiple analysis teams (nodes) within five working groups (WG). The homogenisation of the stellar parameters within WG11 (high resolution observations of FGK stars) and the homogenisation of the stellar parameters within WG10 (medium resolution observations of FGK stars) is described here. In both cases, the homogenisation was carried out using a bayesian Inference method developed specifically for the Gaia-ESO Survey by WG11. The WG10 homogenisation primarily used the cross-match of stars with WG11 as the reference set in both the stellar parameter and chemical abundance homogenisation. In this way the WG10 homogenised results have been placed directly onto the WG11 stellar parameter and chemical abundance scales. The reference set for the metal-poor end was sparse which limited the effectiveness of the homogenisation in that regime. For WG11, the total number of stars for which stellar parameters were derived was 6,231 with typical uncertainties for Teff, log g and [Fe/H] of 32~K, 0.05 and 0.05 respectively. One or more chemical abundances out of a possible 39 elements were derived for 6,188 of the stars. For WG10, the total number of stars for which stellar parameters were derived was 76,675 with typical uncertainties for Teff, log g and [Fe/H] of 64~K, 0.15 and 0.07 respectively. One or more chemical abundances out of a possible 30 elements were derived for 64,177 of the stars., Comment: 31 pages, 19 figures

Within recent approaches to text-to-video (T2V) generation, achieving controllability in the synthesized video is often a challenge. Typically, this issue is addressed by providing low-level per-frame guidance in the form of edge maps, depth maps, or an existing video to be altered. However, the process of obtaining such guidance can be labor-intensive. This paper focuses on enhancing controllability in video synthesis by employing straightforward bounding boxes to guide the subject in various ways, all without the need for neural network training, finetuning, optimization at inference time, or the use of pre-existing videos. Our algorithm, TrailBlazer, is constructed upon a pre-trained (T2V) model, and easy to implement. The subject is directed by a bounding box through the proposed spatial and temporal attention map editing. Moreover, we introduce the concept of keyframing, allowing the subject trajectory and overall appearance to be guided by both a moving bounding box and corresponding prompts, without the need to provide a detailed mask. The method is efficient, with negligible additional computation relative to the underlying pre-trained model. Despite the simplicity of the bounding box guidance, the resulting motion is surprisingly natural, with emergent effects including perspective and movement toward the virtual camera as the box size increases., Comment: 14 pages, 18 figures, Project Page: https://hohonu-vicml.github.io/Trailblazer.Page/

A key challenge in materials discovery is to find high-temperature superconductors. Hydrogen and hydride materials have long been considered promising materials displaying conventional phonon-mediated superconductivity. However, the high pressures required to stabilize these materials have restricted their application. Here, we present results from high-throughput computation, considering a wide range of high-symmetry ternary hydrides from across the periodic table at ambient pressure. This large composition space is then reduced by considering thermodynamic, dynamic, and magnetic stability, before direct estimations of the superconducting critical temperature. This approach has revealed a metastable ambient-pressure hydride superconductor, Mg$_2$IrH$_6$, with a predicted critical temperature of 160 K, comparable to the highest temperature superconducting cuprates. We propose a synthesis route via a structurally related insulator, Mg$_2$IrH$_7$, which is thermodynamically stable above 15 GPa and discuss the potential challenges in doing so., Comment: 30 pages, 30 figures, including SM

Visual effects commonly requires both the creation of realistic synthetic humans as well as retargeting actors' performances to humanoid characters such as aliens and monsters. Achieving the expressive performances demanded in entertainment requires manipulating complex models with hundreds of parameters. Full creative control requires the freedom to make edits at any stage of the production, which prohibits the use of a fully automatic ``black box'' solution with uninterpretable parameters. On the other hand, producing realistic animation with these sophisticated models is difficult and laborious. This paper describes FDLS (Facial Deep Learning Solver), which is Weta Digital's solution to these challenges. FDLS adopts a coarse-to-fine and human-in-the-loop strategy, allowing a solved performance to be verified and edited at several stages in the solving process. To train FDLS, we first transform the raw motion-captured data into robust graph features. Secondly, based on the observation that the artists typically finalize the jaw pass animation before proceeding to finer detail, we solve for the jaw motion first and predict fine expressions with region-based networks conditioned on the jaw position. Finally, artists can optionally invoke a non-linear finetuning process on top of the FDLS solution to follow the motion-captured virtual markers as closely as possible. FDLS supports editing if needed to improve the results of the deep learning solution and it can handle small daily changes in the actor's face shape. FDLS permits reliable and production-quality performance solving with minimal training and little or no manual effort in many cases, while also allowing the solve to be guided and edited in unusual and difficult cases. The system has been under development for several years and has been used in major movies., Comment: DigiPro '22: The Digital Production Symposium

The recent claim of room temperature superconductivity in a copper-doped lead apatite compound, called LK-99, has sparked remarkable interest and controversy. Subsequent experiments have largely failed to reproduce the claimed superconductivity, while theoretical works have identified multiple key features including strong electronic correlation, structural instabilities, and dopability constraints. A puzzling claim of several recent theoretical studies is that both parent and copper-doped lead apatite structures are dynamically unstable at the harmonic level, questioning decades of experimental reports of the parent compound structures and the recently proposed copper-doped structures. In this work, we demonstrate that both parent and copper-doped lead apatite structures are dynamically stable at room temperature. Anharmonic phonon-phonon interactions play a key role in stabilizing some copper-doped phases, while most phases are largely stable even at the harmonic level. We also show that dynamical stability depends on both volume and correlation strength, suggesting controllable ways of exploring the copper-doped lead apatite structural phase diagram. Our results fully reconcile the theoretical description of the structures of both parent and copper-doped lead apatite with experiment., Comment: 4 figures + Supplementary Information, published version

The Epoch of Reionization (EoR) began when galaxies grew in abundance and luminosity, so their escaping Lyman continuum (LyC) radiation started ionizing the surrounding neutral intergalactic medium (IGM). Despite significant recent progress, the nature and role of cosmic reionizers are still unclear: in order to define them, it would be necessary to directly measure their LyC escape fraction ($f_{esc}$). However, this is impossible during the EoR due to the opacity of the IGM. Consequently, many efforts at low and intermediate redshift have been made to determine measurable indirect indicators in high-redshift galaxies so that their $f_{esc}$ can be predicted. This work presents the analysis of the indirect indicators of 62 spectroscopically confirmed star-forming galaxies at $6 \leq z \leq 9$ from the Cosmic Evolution Early Release Science (CEERS) survey, combined with 12 sources with public data from other JWST-ERS campaigns. From the NIRCam and NIRSpec observations, we measured their physical and spectroscopic properties. We discovered that on average $6

Controlling the formation and stoichiometric content of desired phases of materials has become a central interest for the study of a variety of fields, notably high temperature superconductivity under extreme pressures. The further possibility of accessing metastable states by initiating reactions by x-ray triggered mechanisms over ultra-short timescales is enabled with the development of x-ray free electron lasers (XFEL). Utilizing the exceptionally high brilliance x-ray pulses from the EuXFEL, we report the synthesis of a previously unobserved yttrium hydride under high pressure, along with non-stoichiometric changes in hydrogen content as probed at a repetition rate of 4.5\,MHz using time-resolved x-ray diffraction. Exploiting non-equilibrium pathways we synthesize and characterize a hydride with yttrium cations in an \textit{A}15 structure type at 125\,GPa, predicted using crystal structure searches, with a hydrogen content between 4.0--5.75 hydrogens per cation, that is enthalpically metastable on the convex hull. We demonstrate a tailored approach to changing hydrogen content using changes in x-ray fluence that is not accessible using conventional synthesis methods, and reveals a new paradigm in metastable chemical physics.

Recently, various methods have been proposed to address the inconsistency issue of DDIM inversion to enable image editing, such as EDICT [36] and Null-text inversion [22]. However, the above methods introduce considerable computational overhead. In this paper, we propose a new technique, named \emph{bi-directional integration approximation} (BDIA), to perform exact diffusion inversion with neglible computational overhead. Suppose we would like to estimate the next diffusion state $\boldsymbol{z}_{i-1}$ at timestep $t_i$ with the historical information $(i,\boldsymbol{z}_i)$ and $(i+1,\boldsymbol{z}_{i+1})$. We first obtain the estimated Gaussian noise $\hat{\boldsymbol{\epsilon}}(\boldsymbol{z}_i,i)$, and then apply the DDIM update procedure twice for approximating the ODE integration over the next time-slot $[t_i, t_{i-1}]$ in the forward manner and the previous time-slot $[t_i, t_{t+1}]$ in the backward manner. The DDIM step for the previous time-slot is used to refine the integration approximation made earlier when computing $\boldsymbol{z}_i$. A nice property of BDIA-DDIM is that the update expression for $\boldsymbol{z}_{i-1}$ is a linear combination of $(\boldsymbol{z}_{i+1}, \boldsymbol{z}_i, \hat{\boldsymbol{\epsilon}}(\boldsymbol{z}_i,i))$. This allows for exact backward computation of $\boldsymbol{z}_{i+1}$ given $(\boldsymbol{z}_i, \boldsymbol{z}_{i-1})$, thus leading to exact diffusion inversion. It is demonstrated with experiments that (round-trip) BDIA-DDIM is particularly effective for image editing. Our experiments further show that BDIA-DDIM produces markedly better image sampling qualities than DDIM for text-to-image generation. BDIA can also be applied to improve the performance of other ODE solvers in addition to DDIM. In our work, it is found that applying BDIA to the EDM sampling procedure produces consistently better performance over four pre-trained models., Comment: arXiv admin note: text overlap with arXiv:2304.11328. Our code is available at https://github.com/guoqiang-zhang-x/BDIA

Alpha matting is widely used in video conferencing as well as in movies, television, and social media sites. Deep learning approaches to the matte extraction problem are well suited to video conferencing due to the consistent subject matter (front-facing humans), however training-based approaches are somewhat pointless for entertainment videos where varied subjects (spaceships, monsters, etc.) may appear only a few times in a single movie -- if a method of creating ground truth for training exists, just use that method to produce the desired mattes. We introduce a training-free high quality neural matte extraction approach that specifically targets the assumptions of visual effects production. Our approach is based on the deep image prior, which optimizes a deep neural network to fit a single image, thereby providing a deep encoding of the particular image. We make use of the representations in the penultimate layer to interpolate coarse and incomplete "trimap" constraints. Videos processed with this approach are temporally consistent. The algorithm is both very simple and surprisingly effective.

Machine-learned interatomic potentials are fast becoming an indispensable tool in computational materials science. One approach is the ephemeral data-derived potential (EDDP), which was designed to accelerate atomistic structure prediction. The EDDP is simple and cost-efficient. It relies on training data generated in small unit cells and is fit using a lightweight neural network, leading to smooth interactions which exhibit the robust transferability essential for structure prediction. Here, we present a variety of applications of EDDPs, enabled by recent developments of the open-source EDDP software. New features include interfaces to phonon and molecular dynamics codes, as well as deployment of the ensemble deviation for estimating the confidence in EDDP predictions. Through case studies ranging from elemental carbon and lead to the binary scandium hydride and the ternary zinc cyanide, we demonstrate that EDDPs can be trained to cover wide ranges of pressures and stoichiometries, and used to evaluate phonons, phase diagrams, superionicity, and thermal expansion. These developments complement continued success in accelerated structure prediction., Comment: 22 pages, 15 figures