Search

We report on results from an experiment at the European XFEL where we measured the x-ray Thomson scattering (XRTS) spectrum of single crystal silicon with ultrahigh resolution. Compared to similar previous experiments, we consider a more complex scattering setup, in which the scattering vector changes orientation through the crystal lattice. In doing so, we are able to observe strong geometric dependencies in the inelastic scattering spectrum of silicon at low scattering angles. Furthermore, the high quality of the experimental data allows us to benchmark state-of-the-art TDDFT calculations, and demonstrate TDDFT's ability to accurately predict these geometric dependencies. Finally, we note that this experimental data was collected at a much faster rate than another recently reported dataset using the same setup, demonstrating that ultrahigh resolution XRTS data can be collected in more general experimental scenarios.

Low symmetry crystals have recently emerged as a platform for exploring novel light-matter interactions in the form of hyperbolic shear polaritons. These excitations exhibit unique optical properties such as frequency-dispersive optical axes and asymmetric light propagation and energy dissipation, which arise from the presence of non-orthogonal resonances. However, only non-vdW materials have been demonstrated to support hyperbolic shear polaritons, limiting their exotic properties and potential applications. Here we introduce for the first time novel shear phenomena in low symmetry crystal thin films by demonstrating the existence of elliptical and canalized shear phonon polaritons in gypsum, an exfoliable monoclinic sulphate mineral. Our results unveil a topological transition from hyperbolic shear to elliptical shear polaritons, passing through a canalization regime with strong field confinement. Importantly, we observe a significant slowdown of group velocity, reaching values as low as 0.0005c, highlighting the potential of gypsum for "slow light" applications and extreme light-matter interaction control. These findings expand the application scope of low-symmetry crystals with the benefits that an exfoliable material provides, such as stronger field confinement, tunability, and versatility for its incorporation in complex photonic devices that might unlock new optical phenomena at the nanoscale.

The o1 model series is trained with large-scale reinforcement learning to reason using chain of thought. These advanced reasoning capabilities provide new avenues for improving the safety and robustness of our models. In particular, our models can reason about our safety policies in context when responding to potentially unsafe prompts, through deliberative alignment. This leads to state-of-the-art performance on certain benchmarks for risks such as generating illicit advice, choosing stereotyped responses, and succumbing to known jailbreaks. Training models to incorporate a chain of thought before answering has the potential to unlock substantial benefits, while also increasing potential risks that stem from heightened intelligence. Our results underscore the need for building robust alignment methods, extensively stress-testing their efficacy, and maintaining meticulous risk management protocols. This report outlines the safety work carried out for the OpenAI o1 and OpenAI o1-mini models, including safety evaluations, external red teaming, and Preparedness Framework evaluations.

This paper presents a novel approach for simulating plasma instabilities in tokamak plasmas using the piecewise field-aligned finite element method in combination with the particle-in-cell method. Our method traditionally aligns the computational grid but defines the basis functions in piecewise field-aligned coordinates to avoid grid deformation while naturally representing the field-aligned mode structures. This scheme is formulated and implemented numerically. It also applied to the unstructured triangular meshes in principle. We have conducted linear benchmark tests, which agree well with previous results and traditional schemes. Furthermore, multiple-$n$ simulations are also carried out as a proof of principle, demonstrating the efficiency of this scheme in nonlinear turbulence simulations within the framework of the finite element method., Comment: 27 pages, 10 figures, 20th edition of the European Fusion Theory Conference (EFTC 2023), 8th Asia-Pacific Conference on Plasma Physics(AAPPS-DPP2024)

GPT-4o is an autoregressive omni model that accepts as input any combination of text, audio, image, and video, and generates any combination of text, audio, and image outputs. It's trained end-to-end across text, vision, and audio, meaning all inputs and outputs are processed by the same neural network. GPT-4o can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time in conversation. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50\% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models. In line with our commitment to building AI safely and consistent with our voluntary commitments to the White House, we are sharing the GPT-4o System Card, which includes our Preparedness Framework evaluations. In this System Card, we provide a detailed look at GPT-4o's capabilities, limitations, and safety evaluations across multiple categories, focusing on speech-to-speech while also evaluating text and image capabilities, and measures we've implemented to ensure the model is safe and aligned. We also include third-party assessments on dangerous capabilities, as well as discussion of potential societal impacts of GPT-4o's text and vision capabilities.

We investigate tearing modes (TM) driven by current density gradient in collisionless tokamak plasmas by using the electromagnetic gyrokinetic simulation code ORB5. We elucidate the TM width by simulations for flat profiles, as the absence of background diamagnetic flows implies a small rotation-speed, while finite-gradients are included to investigate the TM rotation. For flat profiles, the initial saturation width of nonlinearly driven magnetic islands is related to the TM linear growth rate; however, large islands in the initial saturation phase are prone to current density redistribution that reduces the island width in the following evolution. Island-induced $\boldsymbol{E}\times\boldsymbol{B}$ and diamagnetic sheared flows develop at the separatrix, able to destabilize the Kelvin-Helmholtz instability (KHI). The KHI turbulence enhances a strong quadrupole vortex flow that reinforces the island decay, resulting in a strong reduction of the island width in an eventual steady state. This process is enhanced by trapped electrons. For finite gradients profile, the TM usually rotates in the electron diamagnetic direction, but can change direction when the ion temperature gradient dominates the other gradients. The reduced growth of the TM by diamagnetic effects results in a moderate island size, which remains almost unchanged after the initial saturation. At steady state, strong zonal flows are nonlinearly excited and dominate the island rotation, as expected from previous theoretical and numerical studies. When the plasma beta is increased, the TM mode is suppressed and a mode with the same helicity but with twisting parity, coupled with the neighboring poloidal harmonics, is destabilized, similar to the kinetic ballooning mode.

The triangular mesh-based gyrokinetic scheme enables comprehensive axis-to-edge studies across the entire plasma volume. Our approach employs triangular finite elements with first-derivative continuity (C1), building on previous work to facilitate gyrokinetic simulations. Additionally, we have adopted the mixed variable/pullback scheme for gyrokinetic electromagnetic particle simulations. The filter-free treatment in the poloidal cross-section with triangular meshes introduces unique features and challenges compared to previous treatments using structured meshes. Our implementation has been validated through benchmarks using ITPA-TAE (Toroidicity-induced Alfv\'en Eigenmode) parameters, showing its capability in moderate to small electron skin depth regimes. Additional examinations using experimental parameters confirm its applicability to realistic plasma conditions., Comment: 31 pages, 9 figures. Joint Varenna-Lausanne International Workshop on Theory of Fusion Plasmas. Submitted to Plasma Physics and Controlled Fusion (2024)

Two-dimensional materials with flat electronic bands are promising for realizing exotic quantum phenomena such as unconventional superconductivity and nontrivial topology, but exploring their vast chemical space remains challenging. Here, we introduce an unsupervised convolutional autoencoder agent (elf) that operates on electronic band structure images and is capable of mapping band features and extracting the latent space representation as a fingerprint, enabling autonomous clustering of materials with common electronic properties beyond traditional chemical paradigms. Unsupervised visualisation of the latent space then helps to uncover hidden chemical trends and identify promising candidates based on similarities to well-studied exemplars. Our framework paves the way for the accelerated discovery of novel flat-band materials with desirable electronic characteristics. It complements high-throughput ab initio methods by rapidly screening candidates and guides further investigations into the mechanisms governing the emergence of flat-band physics. We believe the elf autoencoder will be a valuable tool for the autonomous discovery of previously unexplored flat-band materials, aiding in the unbiased identification of compounds with desirable electronic properties in vast 2D chemical space.

Ashenhurst-Curtis decomposition (ACD) is a decomposition technique used, in particular, to map combinational logic into lookup tables (LUTs) structures when synthesizing hardware designs. However, available implementations of ACD suffer from excessive complexity, search-space restrictions, and slow run time, which limit their applicability and scalability. This paper presents a novel fast and versatile technique of ACD suitable for delay optimization. We use this new formulation to compute two-level decompositions into a variable number of LUTs and enhance delay-driven LUT mapping by performing ACD on the fly. Compared to state-of-the-art technology mapping, experiments on heavily optimized benchmarks demonstrate an average delay improvement of 12.39%, and area reduction of 2.20% with affordable run time. Additionally, our method improves 4 of the best delay results in the EPFL synthesis competition without employing design-space exploration techniques., Comment: originally accepted at 33rd International Workshop on Logic & Synthesis 2024

Mosaic crystals, with their high integrated reflectivities, are widely-employed in spectrometers used to diagnose high energy density systems. X-ray Thomson scattering (XRTS) has emerged as a powerful diagnostic tool of these systems, providing in principle direct access to important properties such as the temperature via detailed balance. However, the measured XRTS spectrum is broadened by the spectrometer instrument function (IF), and without careful consideration of the IF one risks misdiagnosing system conditions. Here, we consider in detail the IF of 40 $\mu$m and 100 $\mu$m mosaic HAPG crystals, and how the broadening varies across the spectrometer in an energy range of 6.7-8.6 keV. Notably, we find a strong asymmetry in the shape of the IF towards higher energies. As an example, we consider the effect of the asymmetry in the IF on the temperature inferred via XRTS for simulated 80 eV CH plasmas, and find that the temperature can be overestimated if an approximate symmetric IF is used. We therefore expect a detailed consideration of the full IF will have an important impact on system properties inferred via XRTS in both forward modelling and model-free approaches., Comment: 19 pages, 13 figures

The auto-regressive decoding of Large Language Models (LLMs) results in significant overheads in their hardware performance. While recent research has investigated various speculative decoding techniques for multi-token generation, these efforts have primarily focused on improving processing speed such as throughput. Crucially, they often neglect other metrics essential for real-life deployments, such as memory consumption and training cost. To overcome these limitations, we propose a novel parallel prompt decoding that requires only $0.0002$% trainable parameters, enabling efficient training on a single A100-40GB GPU in just 16 hours. Inspired by the human natural language generation process, $PPD$ approximates outputs generated at future timesteps in parallel by using multiple prompt tokens. This approach partially recovers the missing conditional dependency information necessary for multi-token generation, resulting in up to a 28% higher acceptance rate for long-range predictions. Furthermore, we present a hardware-aware dynamic sparse tree technique that adaptively optimizes this decoding scheme to fully leverage the computational capacities on different GPUs. Through extensive experiments across LLMs ranging from MobileLlama to Vicuna-13B on a wide range of benchmarks, our approach demonstrates up to 2.49$\times$ speedup and maintains a minimal runtime memory overhead of just $0.0004$%. More importantly, our parallel prompt decoding can serve as an orthogonal optimization for synergistic integration with existing speculative decoding, showing up to $1.22\times$ further speed improvement. Our code is available at https://github.com/hmarkc/parallel-prompt-decoding., Comment: The code for this implementation is available at https://github.com/hmarkc/parallel-prompt-decoding

Existing learning rate schedules that do not require specification of the optimization stopping step T are greatly out-performed by learning rate schedules that depend on T. We propose an approach that avoids the need for this stopping time by eschewing the use of schedules entirely, while exhibiting state-of-the-art performance compared to schedules across a wide family of problems ranging from convex problems to large-scale deep learning problems. Our Schedule-Free approach introduces no additional hyper-parameters over standard optimizers with momentum. Our method is a direct consequence of a new theory we develop that unifies scheduling and iterate averaging. An open source implementation of our method is available at https://github.com/facebookresearch/schedule_free. Schedule-Free AdamW is the core algorithm behind our winning entry to the MLCommons 2024 AlgoPerf Algorithmic Efficiency Challenge Self-Tuning track.

Transition metal dichalcogenide (TMD) nanoscrolls (NS) exhibit significant photoluminescence (PL) signals despite their multilayer structure, which cannot be explained by the strained multilayer description of NS. Here, we investigate the interlayer interactions in NS to address this discrepancy. The reduction of interlayer interactions in NS is attributed to two factors: (1) the symmetry-broken mixed stacking order between neighbouring layers due to misalignment, and (2) the high inhomogeneity in the strain landscape resulting from the unique Archimedean spiral-like geometry with positive eccentricity. These were confirmed through transmission electron microscopy, field emission scanning electron microscopy and atomic force microscopy. To probe the effect of reduction of interlayer interactions in multilayered MoS$_2$ nanoscrolls, low-temperature PL spectroscopy was employed investigating the behaviour of K-point excitons. The effects of reduced interlayer interactions on exciton-phonon coupling (EXPC), exciton energy, and exciton oscillator strength are discussed, providing insights into the unique properties of TMD nanoscrolls., Comment: S.C. and T.C. have contributed equally to this work

Including the effect of lattice anharmonicity on electron-phonon interactions has recently garnered attention due to its role as a necessary and significant component in explaining various phenomena, including superconductivity, optical response, and temperature dependence of mobility. This study focuses on analytically treating the effects of anharmonic electron-phonon coupling on the polaron self-energy, combined with numerical Diagrammatic Monte Carlo data. Specifically, we incorporate a quadratic interaction into the method of squeezed phonon states, which has proven effective for analytically calculating the polaron parameters. Additionally, we extend this method to nonparabolic finite-width conduction bands while maintaining the periodic translation symmetry of the system. Our results are compared with those obtained from Diagrammatic Monte Carlo, partially reported in a recent study [S. Ragni et al., Phys. Rev. B 107, L121109(2023)], covering a wide range of coupling strengths for the nonlinear interaction. Remarkably, our analytic method predicts the same features as the Diagrammatic Monte Carlo simulation., Comment: 32 pages, 8 figures

Using a novel ultrahigh resolution ($\Delta E \sim 0.1\,$eV) setup to measure electronic features in x-ray Thomson scattering (XRTS) experiments at the European XFEL in Germany, we have studied the collective plasmon excitation in aluminium at ambient conditions, which we can measure very accurately even at low momentum transfers. As a result, we can resolve previously reported discrepancies between ab initio time-dependent density functional theory simulations and experimental observations. The demonstrated capability for high-resolution XRTS measurements will be a game changer for the diagnosis of experiments with matter under extreme densities, temperatures, and pressures, and unlock the full potential of state-of-the-art x-ray free electron laser (XFEL) facilities to study planetary interior conditions, to understand inertial confinement fusion applications, and for material science and discovery.

Hot electrons formed in a graphene crystal by high-intensity short-duration laser pulses can exist for a time that is less than an electron-phonon energy relaxation time. During that time, electron-electron collisions cause the electrons to thermalize to a local effective temperature that propagates (diffuses) through graphene. The non-uniform nature of the electron temperature leads to a force acting on the graphene lattice. This force is the result of the electron-lattice interaction that exists even at times that are less than the electron-phonon scattering time. The force causes the lattice to deform. A Boltzmann equation description of the transient electron-lattice deformation causes by hot electrons in graphene is presented., Comment: 10.1 pages

Equation of state measurements at Jovian or stellar conditions are currently conducted by dynamic shock compression driven by multi-kilojoule multi-beam nanosecond-duration lasers. These experiments require precise design of the target and specific tailoring of the spatial and temporal laser profiles to reach the highest pressures. At the same time, the studies are limited by the low repetition rate of the lasers. Here, we show that by the irradiation of a thin wire with single beam Joule-class short-pulse laser, a converging cylindrical shock is generated compressing the wire material to conditions relevant for the above applications. The shockwave was observed using Phase Contrast Imaging employing a hard X-ray Free Electron Laser with unprecedented temporal and spatial sensitivity. The data collected for Cu wires is in agreement with hydrodynamic simulations of an ablative shock launched by a highly-impulsive and transient resistive heating of the wire surface. The subsequent cylindrical shockwave travels towards the wire axis and is predicted to reach a compression factor of 9 and pressures above 800 Mbar. Simulations for astrophysical relevant materials underline the potential of this compression technique as a new tool for high energy density studies at high repetition rates.

In this work, we use the global electromagnetic and electrostatic gyro kinetic approaches to investigate the effects of zonal flows forced-driven by Alfv\'en modes due to their excitation by energetic particles (EPs), on the dynamics of ITG (Ion temperature gradient) instabilities. The equilibrium of the 92416 JET tokamak shot is considered. The linear and nonlinear Alfv\'en modes dynamics, as well as the zonal flow dynamics, are investigated and their respective radial structures and saturation levels are reported. ITG dynamics in the presence of the zonal flows excited by these Alfv\'en modes are also investigated. We find that, the zonal flows forced-driven by Alfv\'en modes can significantly impact the ITG dynamics. A zonal flow amplitude scan reveals the existence of an inverse relation between the zonal flow amplitude and the ITG growth rate. These results show that, forced-driven zonal flows can be an important indirect part of turbulence mitigation due to the injection of energetic particles., Comment: 13 pages, 10 figures

We investigate collective spin excitations of graphene electrons with short-ranged interactions and subject to the external Zeeman magnetic field. We find that in addition to the familiar Silin spin wave, a collective spin-flip excitation that reduces to the uniform precession when the wave's momentum approaches zero, the magnetized graphene supports another collective mode visible in the transverse spin susceptibility: a collective spin-current mode. Unlike the Silin wave, this mode is not dictated by the spin-rotational symmetry but rather owns its existence to the pseudo-spin structure of the graphene lattice. We find the new collective excitation to become sharply defined in a finite interval of wave's momenta, the range of which is determined by the interaction and the magnetization., Comment: 15 pages, 7 figures

Transition-metal dichalcogenides (TMDs) host tightly bound quasi-particles called excitons. Based on spin and momentum selection rules, these excitons can be either optically bright or dark. In tungsten-based TMDs, momentum-forbidden dark exciton is the energy ground state and therefore it strongly affect the emission properties. In this work, we brighten the momentum forbidden dark exciton by placing WS$_2$ on top of nanotextured substrates which put the WS$_2$ layer under tensile strain, modifying electronic bandstructure. This enables phonon assisted scattering of exciton between momentum valleys, thereby brightening momentum forbidden dark excitons. Our results will pave the way to design ultrasensitive strain sensing devices based on TMDs.

Sequential logic synthesis can provide better Power-Performance-Area (PPA) than combinational logic synthesis since it explores a larger solution space. As the gate cost in advanced technologies keeps rising, sequential logic synthesis provides a powerful alternative that is gaining momentum in the EDA community. In this work, we present a new scalable algorithm for don't-care-based sequential logic synthesis. Our new approach is based on sequential k-step induction and can apply both redundancy removal and resubstitution transformations under Sequential Observability Don't Cares (SODCs). Using SODC-based optimizations with induction is a challenging problem due to dependencies and alignment of don't cares among the base case and the inductive case. We propose a new approach utilizing the full power of SODCs without limiting the solution space. Our algorithm is implemented as part of an industrial tool and achieves 6.9% average area improvement after technology mapping when compared to state-of-the-art sequential synthesis methods. Moreover, all the new sequential optimizations can be verified using state-of-the-art sequential verification tools., Comment: Accepted for the conference Design, Automation and Test in Europe (DATE) 2024

The complexity of modern hardware designs necessitates advanced methodologies for optimizing and analyzing modern digital systems. In recent times, machine learning (ML) methodologies have emerged as potent instruments for assessing design quality-of-results at the Register-Transfer Level (RTL) or Boolean level, aiming to expedite design exploration of advanced RTL configurations. In this presentation, we introduce an innovative open-source framework that translates RTL designs into graph representation foundations, which can be seamlessly integrated with the PyTorch Geometric graph learning platform. Furthermore, the Verilog-to-PyG (V2PYG) framework is compatible with the open-source Electronic Design Automation (EDA) toolchain OpenROAD, facilitating the collection of labeled datasets in an utterly open-source manner. Additionally, we will present novel RTL data augmentation methods (incorporated in our framework) that enable functional equivalent design augmentation for the construction of an extensive graph-based RTL design database. Lastly, we will showcase several using cases of V2PYG with detailed scripting examples. V2PYG can be found at \url{https://yu-maryland.github.io/Verilog-to-PyG/}., Comment: 8 pages, International Conference on Computer-Aided Design (ICCAD'23)

Electrostatic gyrokinetic instabilities and turbulence in the Wendelstein 7-X stellarator are studied. Particular attention is paid to the ion-temperature-gradient (ITG) instability and its character close to marginal stability [Floquet-type turbulence (Zocco et al. 2022) with no electron temperature gradient]. The flux-tube version of the $\delta$f code stella (Barnes et al. 2019) is used to run linear and nonlinear gyrokinetic simulations with kinetic electrons. The nature of the dominant instability depends on the wavelength perpendicular to the magnetic field, and the results are conveniently displayed in stability diagrams that take this dependence into account. This approach highlights the presence of universal instabilities, which are less unstable but have longer wavelengths than other modes. A quasi-linear estimate of the heat flux suggests they are relevant for transport. Close to the stability threshold, the linear eigenmodes and turbulence form highly extended structures along the computational domain if the magnetic shear is small. Numerical experiments and diagnostics are undertaken to assess the resulting radial localisation of the turbulence, which affects the interaction of the latter with zonal flows. Increasing the amplitude of the magnetic shear (e.g. through current drive) has a stabilising effect on the turbulence and thus reduces the nonlinear energy transport.

The key methodologies of modern logic synthesis techniques are conducted on multi-level technology-independent representations such as And-Inverter-Graphs (AIGs) of the digital logic via directed-acyclic-graph (DAGs) traversal based structural rewriting, resubstitution, and refactoring. Existing state-of-the-art DAG-aware logic synthesis algorithms are all designed to perform stand-alone optimizations during a single DAG traversal. However, we empirically identify and demonstrate that these algorithms are limited in quality-of-results and runtime complexity due to this design concept. This work proposes Synthesis Orchestration, which orchestrates stand-alone operations within the single traversal of AIG. Thus, orchestration method explores more optimization opportunities and results in better performance. Our experimental results are comprehensively conducted on all 104 designs collected from ISCAS'85/89/99, VTR, and EPFL benchmark suites, with consistent logic minimization improvements over rewriting, resubstitution, refactoring, leading to an average of 4% more node reduction with improved runtime efficiency for the single optimization. Moreover, we evaluate orchestration as a plug-in algorithm in resyn and resyn3 flows in ABC, which demonstrates consistent logic minimization improvements (3.8% and 10.9% more node reduction on average). The runtime analysis demonstrates the orchestration outperforms stand-alone algorithms in both AIG minimization and runtime efficiency. Finally, we integrate the orchestration into OpenROAD for end-to-end performance evaluation. Our results demonstrate the advantages of the orchestration optimization technique, even after technology mapping and post-routing in the design flow have been conducted.

Learning rate schedules used in practice bear little resemblance to those recommended by theory. We close much of this theory/practice gap, and as a consequence are able to derive new problem-adaptive learning rate schedules. Our main technical contribution is a refined analysis of learning rate schedules for a wide class of optimization algorithms (including SGD). When considering only worst-case analysis, our theory predicts that the optimal choice is the linear decay schedule where the step-size is set proportional to 1 - t/T, where t is the current iteration and T is the total number of steps. To go beyond this worst-case analysis, we use the observed gradient norms to derive schedules refined for any particular task. These refined schedules exhibit learning rate warm-up and rapid learning rate annealing near the end of training. Ours is the first systematic approach to automatically yield both of these properties. We perform the most comprehensive evaluation of learning rate schedules to date, evaluating across 10 diverse deep learning problems, a series of LLMs, and a suite of logistic regression problems. We validate that overall, the linear-decay schedule outperforms all commonly used default schedules including cosine annealing. Our adaptive schedule refinement method gives further improvements., Comment: An earlier preprint of this work appeared under the title When, Why and How Much? Adaptive Learning Rate Scheduling by Refinement

Recently, two arXiv preprints (arXiv:2307.12008, arXiv:2307.12037) reported signatures of superconductivity above room temperature (SART) and at ambient pressure, striking worldwide experimental research efforts in replicating the results3-6, as well as theoretical attempts to explain the purported superconductivity7-11. The material of interest has chemical formula Pb$_{10-x}$Cu$_x$(PO$_4$)$_6$O, where $x \approx 1$, and was named by the authors as LK-99. It belongs to lead apatite family, and was synthesised from two precursors, lanarkite (PbSO$_4\cdot$PbO) and copper phosphide (Cu$_3$P). Here we performed a systematic study on LK-99, starting from solid-state synthesis, followed by characterisation and transport measurements. We did not observe any signatures of superconductivity in our samples of LK-99.

In this paper, we explore two fundamental first-order algorithms in convex optimization, namely, gradient descent (GD) and proximal gradient method (ProxGD). Our focus is on making these algorithms entirely adaptive by leveraging local curvature information of smooth functions. We propose adaptive versions of GD and ProxGD that are based on observed gradient differences and, thus, have no added computational costs. Moreover, we prove convergence of our methods assuming only local Lipschitzness of the gradient. In addition, the proposed versions allow for even larger stepsizes than those initially suggested in [MM20].

Insights into the fundamental properties of graphene's Dirac-Weyl fermions have emerged from studies of electron tunnelling transistors in which an atomically thin layer of hexagonal boron nitride (hBN) is sandwiched between two layers of high purity graphene. Here, we show that when a single defect is present within the hBN tunnel barrier, it can inject electrons into the graphene layers and its sharply defined energy level acts as a high resolution spectroscopic probe of electron-electron interactions in graphene. We report a magnetic field dependent suppression of the tunnel current flowing through a single defect below temperatures of $\sim$ 2 K. This is attributed to the formation of a magnetically-induced Coulomb gap in the spectral density of electrons tunnelling into graphene due to electron-electron interactions.

Nonlinear simulations of Alfv\'en modes (AM) driven by energetic particles (EP) in the presence of turbulence are performed with the gyrokinetic particle-in-cell code ORB5. The AMs carry a heat flux, and consequently they nonlinearly modify the plasma temperature profiles. The isolated effect of this modification on the dynamics of turbulence is studied, by means of electrostatic simulations. We find that turbulence is reduced when the profiles relaxed by the AM are used, with respect to the simulation where the unperturbed profiles are used. This is an example of indirect interaction of EPs and turbulence. First, an analytic magnetic equilibrium with circular concentric flux surfaces is considered as a simplified example for this study. Then, an application to an experimentally relevant case of ASDEX Upgrade is discussed.

We consider the problem of estimating the learning rate in adaptive methods, such as AdaGrad and Adam. We propose Prodigy, an algorithm that provably estimates the distance to the solution $D$, which is needed to set the learning rate optimally. At its core, Prodigy is a modification of the D-Adaptation method for learning-rate-free learning. It improves upon the convergence rate of D-Adaptation by a factor of $O(\sqrt{\log(D/d_0)})$, where $d_0$ is the initial estimate of $D$. We test Prodigy on 12 common logistic-regression benchmark datasets, VGG11 and ResNet-50 training on CIFAR10, ViT training on Imagenet, LSTM training on IWSLT14, DLRM training on Criteo dataset, VarNet on Knee MRI dataset, as well as RoBERTa and GPT transformer training on BookWiki. Our experimental results show that our approach consistently outperforms D-Adaptation and reaches test accuracy values close to that of hand-tuned Adam.

We present a partially personalized formulation of Federated Learning (FL) that strikes a balance between the flexibility of personalization and cooperativeness of global training. In our framework, we split the variables into global parameters, which are shared across all clients, and individual local parameters, which are kept private. We prove that under the right split of parameters, it is possible to find global parameters that allow each client to fit their data perfectly, and refer to the obtained problem as overpersonalized. For instance, the shared global parameters can be used to learn good data representations, whereas the personalized layers are fine-tuned for a specific client. Moreover, we present a simple algorithm for the partially personalized formulation that offers significant benefits to all clients. In particular, it breaks the curse of data heterogeneity in several settings, such as training with local steps, asynchronous training, and Byzantine-robust training.

This paper proposes a new easy-to-implement parameter-free gradient-based optimizer: DoWG (Distance over Weighted Gradients). We prove that DoWG is efficient -- matching the convergence rate of optimally tuned gradient descent in convex optimization up to a logarithmic factor without tuning any parameters, and universal -- automatically adapting to both smooth and nonsmooth problems. While popular algorithms following the AdaGrad framework compute a running average of the squared gradients to use for normalization, DoWG maintains a new distance-based weighted version of the running average, which is crucial to achieve the desired properties. To complement our theory, we also show empirically that DoWG trains at the edge of stability, and validate its effectiveness on practical machine learning tasks., Comment: 22 pages, 1 table, 4 figures

We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs. While less capable than humans in many real-world scenarios, GPT-4 exhibits human-level performance on various professional and academic benchmarks, including passing a simulated bar exam with a score around the top 10% of test takers. GPT-4 is a Transformer-based model pre-trained to predict the next token in a document. The post-training alignment process results in improved performance on measures of factuality and adherence to desired behavior. A core component of this project was developing infrastructure and optimization methods that behave predictably across a wide range of scales. This allowed us to accurately predict some aspects of GPT-4's performance based on models trained with no more than 1/1,000th the compute of GPT-4., Comment: 100 pages; updated authors list; fixed author names and added citation

Graphite is one of the most chemically inert materials. Its elementary constituent, monolayer graphene, is generally expected to inherit most of the parent material's properties including chemical inertness. Here we show that, unlike graphite, defect-free monolayer graphene exhibits a strong activity with respect to splitting molecular hydrogen, which is comparable to that of metallic and other known catalysts for this reaction. We attribute the unexpected catalytic activity to surface corrugations (nanoscale ripples), a conclusion supported by theory. Nanoripples are likely to play a role in other chemical reactions involving graphene and, because nanorippling is inherent to atomically thin crystals, can be important for two dimensional materials in general.