Search

We examine the use of different randomisation policies for stochastic gradient algorithms used in sampling, based on first-order (or overdamped) Langevin dynamics, the most popular of which is known as Stochastic Gradient Langevin Dynamics. Conventionally, this algorithm is combined with a specific stochastic gradient strategy, called Robbins-Monro. In this work, we study an alternative strategy, Random Reshuffling, and show convincingly that it leads to improved performance via: a) a proof of reduced bias in the Wasserstein metric for strongly convex, gradient Lipschitz potentials; b) an analytical demonstration of reduced bias for a Gaussian model problem; and c) an empirical demonstration of reduced bias in numerical experiments for some logistic regression problems. This is especially important since Random Reshuffling is typically more efficient due to memory access and cache reasons. Such acceleration for the Random Reshuffling policy is familiar from the optimisation literature on stochastic gradient descent., Comment: 23 pages, 11 figures

Chalcogenide perovskites are lead-free materials for potential photovoltaic or thermoelectric applications. BaZrS$_3$ is the most studied member of this family due to its superior thermal and chemical stability, desirable optoelectronic properties, and low thermal conductivity. Phase transitions of the BaZrS$_3$ perovskite are under-explored in literature as most experimental characterization is performed at ambient conditions where the orthorhombic Pnma phase is reported to be stable. In this work, we study the dynamics of BaZrS$_3$ across a range of temperatures and pressures using an accurate machine-learned interatomic potential trained with data from hybrid density functional theory calculations. At 0Pa, we find a first-order phase transition from the orthorhombic to tetragonal I4/mcm phase at 610K, and a second-order transition from the tetragonal to the cubic Pm-3m phase at 880K. The tetragonal phase is stable over a larger temperature range at higher pressures. To confirm the validity of our model we report the static structure factor as a function of temperature and compare our results with published experimental data.

CG (Computer Graphics) is a popular field of CS (Computer Science), but many students find this topic difficult due to it requiring a large number of skills, such as mathematics, programming, geometric reasoning, and creativity. Over the past few years, researchers have investigated ways to harness the power of GenAI (Generative Artificial Intelligence) to improve teaching. In CS, much of the research has focused on introductory computing. A recent study evaluating the performance of an LLM (Large Language Model), GPT-4 (text-only), on CG questions, indicated poor performance and reliance on detailed descriptions of image content, which often required considerable insight from the user to return reasonable results. So far, no studies have investigated the abilities of LMMs (Large Multimodal Models), or multimodal LLMs, to solve CG questions and how these abilities can be used to improve teaching. In this study, we construct two datasets of CG questions requiring varying degrees of visual perception skills and geometric reasoning skills, and evaluate the current state-of-the-art LMM, GPT-4o, on the two datasets. We find that although GPT-4o exhibits great potential in solving questions with visual information independently, major limitations still exist to the accuracy and quality of the generated results. We propose several novel approaches for CG educators to incorporate GenAI into CG teaching despite these limitations. We hope that our guidelines further encourage learning and engagement in CG classrooms., Comment: 8 pages, 8 figures, 1 table, to be published in SIGGRAPH Asia 2024 Educator's Forum

We propose a scalable kinetic Langevin dynamics algorithm for sampling parameter spaces of big data and AI applications. Our scheme combines a symmetric forward/backward sweep over minibatches with a symmetric discretization of Langevin dynamics. For a particular Langevin splitting method (UBU), we show that the resulting Symmetric Minibatch Splitting-UBU (SMS-UBU) integrator has bias $O(h^2 d^{1/2})$ in dimension $d>0$ with stepsize $h>0$, despite only using one minibatch per iteration, thus providing excellent control of the sampling bias as a function of the stepsize. We apply the algorithm to explore local modes of the posterior distribution of Bayesian neural networks (BNNs) and evaluate the calibration performance of the posterior predictive probabilities for neural networks with convolutional neural network architectures for classification problems on three different datasets (Fashion-MNIST, Celeb-A and chest X-ray). Our results indicate that BNNs sampled with SMS-UBU can offer significantly better calibration performance compared to standard methods of training and stochastic weight averaging., Comment: 35 pages, 6 figures. The first two authors contributed equally

This paper reports on the results of a 2023 survey that explores the Work Integrated Learning (WiL) experiences of thirty recent Computer Science (CS) graduates. The graduates had all completed their undergraduate bachelors degree within the last five years and were currently employed in a CS industry role. The survey asked about the graduates' perceptions within a continuum of WiL experiences from final year capstone projects to professional development in their first industry-based role. Most respondents had taken a capstone course involving a team project. Only two respondents had participated in an internship program. Our results indicate that graduates value their capstone experiences and believe that they provide transferable skills including teamwork, managing client relations, exposure to technologies and methods, and time management. When entering their first industry role less than fifty percent of graduates were allocated a mentor. Overwhelmingly, these graduates noted the importance of those mentors in their transition from student to working professional. Very few of the surveyed graduates were provided with ongoing professional development opportunities. Those who did noted significant gains including growth of leadership skills and accelerated career progression. Our survey highlights a gap and an opportunity for tertiary institutions to work with industry to provide graduate onboarding and novice/early-career professional development opportunities., Comment: 6 pages, 2 figures

We study three kinetic Langevin samplers including the Euler discretization, the BU and the UBU splitting scheme. We provide contraction results in $L^1$-Wasserstein distance for non-convex potentials. These results are based on a carefully tailored distance function and an appropriate coupling construction. Additionally, the error in the $L^1$-Wasserstein distance between the true target measure and the invariant measure of the discretization scheme is bounded. To get an $\varepsilon$-accuracy in $L^1$-Wasserstein distance, we show complexity guarantees of order $\mathcal{O}(\sqrt{d}/\varepsilon)$ for the Euler scheme and $\mathcal{O}(d^{1/4}/\sqrt{\varepsilon})$ for the UBU scheme under appropriate regularity assumptions on the target measure. The results are applicable to interacting particle systems and provide bounds for sampling probability measures of mean-field type., Comment: 44 pages, 3 figures

A method for analyzing non-asymptotic guarantees of numerical discretizations of ergodic SDEs in Wasserstein-2 distance is presented by Sanz-Serna and Zygalakis in ``Wasserstein distance estimates for the distributions of numerical approximations to ergodic stochastic differential equations". They analyze the UBU integrator which is strong order two and only requires one gradient evaluation per step, resulting in desirable non-asymptotic guarantees, in particular $\mathcal{O}(d^{1/4}\epsilon^{-1/2})$ steps to reach a distance of $\epsilon > 0$ in Wasserstein-2 distance away from the target distribution. However, there is a mistake in the local error estimates in Sanz-Serna and Zygalakis (2021), in particular, a stronger assumption is needed to achieve these complexity estimates. This note reconciles the theory with the dimension dependence observed in practice in many applications of interest., Comment: 9 pages

Experiments in engineering are typically conducted in controlled environments where parameters can be set to any desired value. This assumes that the same applies in a real-world setting -- an assumption that is often incorrect as many experiments are influenced by uncontrollable environmental conditions such as temperature, humidity and wind speed. When optimising such experiments, the focus should lie on finding optimal values conditionally on these uncontrollable variables. This article extends Bayesian optimisation to the optimisation of systems in changing environments that include controllable and uncontrollable parameters. The extension fits a global surrogate model over all controllable and environmental variables but optimises only the controllable parameters conditional on measurements of the uncontrollable variables. The method is validated on two synthetic test functions and the effects of the noise level, the number of the environmental parameters, the parameter fluctuation, the variability of the uncontrollable parameters, and the effective domain size are investigated. ENVBO, the proposed algorithm resulting from this investigation, is applied to a wind farm simulator with eight controllable and one environmental parameter. ENVBO finds solutions for the full domain of the environmental variable that outperforms results from optimisation algorithms that only focus on a fixed environmental value in all but one case while using a fraction of their evaluation budget. This makes the proposed approach very sample-efficient and cost-effective. An off-the-shelf open-source version of ENVBO is available via the NUBO Python package., Comment: 23 pages, 10 figures

The chalcogenide perovskite BaZrS$_3$ has strong visible light absorption and high chemical stability, is nontoxic, and is made from earth-abundant elements. As such, it is a promising candidate material for application in optoelectronic technologies. However the synthesis of BaZrS$_3$ thin-films for characterisation and device integration remains a challenge. Here we use density functional theory and lattice dynamics to calculate the vibrational properties of elemental, binary and ternary materials in the Ba-Zr-S system. This is used to build a thermodynamic model for the stability of BaZrS$_3$, BaS$_x$, and ZrS$_x$ in equilibrium with sulfur gas, across a range of temperatures and sulfur partial pressures. We highlight that reaction thermodynamics are highly sensitive to sulfur allotrope and the extent of allotrope mixing. We use our model to predict the synthesis conditions in which BaZrS$_3$ and the intermediate binary compounds can form. At a moderate temperature of 500 {\deg}C we find that BaS$_3$, associated with fast reaction kinetics, is stable at pressures above 3x10$^5$ Pa. We also find BaZrS$_3$ is stable against decomposition into sulfur-rich binaries up to at least 1x10$^7$ Pa. Our work provides insights into the chemistry of this promising material and suggests the experimental conditions required for the successful synthesis of BaZrS$_3$.

Major depressive disorder (MDD) is a complex psychiatric disorder that affects the lives of hundreds of millions of individuals around the globe. Even today, researchers debate if morphological alterations in the brain are linked to MDD, likely due to the heterogeneity of this disorder. The application of deep learning tools to neuroimaging data, capable of capturing complex non-linear patterns, has the potential to provide diagnostic and predictive biomarkers for MDD. However, previous attempts to demarcate MDD patients and healthy controls (HC) based on segmented cortical features via linear machine learning approaches have reported low accuracies. Here, we used globally representative data from the ENIGMA-MDD working group containing 7,012 participants from 30 sites (N=2,772 MDD and N=4,240 HC), which allows a comprehensive analysis with generalizable results. Based on the hypothesis that integration of vertex-wise cortical features can improve classification performance, we evaluated the classification of a DenseNet and a Support Vector Machine (SVM), with the expectation that the former would outperform the latter. We found that both classifiers exhibited close to chance performance (balanced accuracy DenseNet: 51%; SVM: 53%), when estimated on unseen sites. Slightly higher classification performance (balanced accuracy DenseNet: 58%; SVM: 55%) was found when the cross-validation folds contained subjects from all sites, indicating site effect. In conclusion, the integration of vertex-wise morphometric features and the use of the non-linear classifier did not lead to the differentiability between MDD and HC. Our results support the notion that MDD classification on this combination of such features and classifiers is unfeasible. Perhaps more sophisticated integration of multimodal information may lead to a higher performance in this diagnostic task., Comment: arXiv admin note: text overlap with arXiv:2206.08122

We present an unbiased method for Bayesian posterior means based on kinetic Langevin dynamics that combines advanced splitting methods with enhanced gradient approximations. Our approach avoids Metropolis correction by coupling Markov chains at different discretization levels in a multilevel Monte Carlo approach. Theoretical analysis demonstrates that our proposed estimator is unbiased, attains finite variance, and satisfies a central limit theorem. It can achieve accuracy $\epsilon>0$ for estimating expectations of Lipschitz functions in $d$ dimensions with $\mathcal{O}(d^{1/4}\epsilon^{-2})$ expected gradient evaluations, without assuming warm start. We exhibit similar bounds using both approximate and stochastic gradients, and our method's computational cost is shown to scale independently of the size of the dataset. The proposed method is tested using a multinomial regression problem on the MNIST dataset and a Poisson regression model for soccer scores. Experiments indicate that the number of gradient evaluations per effective sample is independent of dimension, even when using inexact gradients. For product distributions, we give dimension-independent variance bounds. Our results demonstrate that in large-scale applications, the unbiased algorithm we present can be 2-3 orders of magnitude more efficient than the ``gold-standard" randomized Hamiltonian Monte Carlo., Comment: 101 Pages, 13 Figures

Personal sensing, leveraging data passively and near-continuously collected with wearables from patients in their ecological environment, is a promising paradigm to monitor mood disorders (MDs), a major determinant of worldwide disease burden. However, collecting and annotating wearable data is very resource-intensive. Studies of this kind can thus typically afford to recruit only a couple dozens of patients. This constitutes one of the major obstacles to applying modern supervised machine learning techniques to MDs detection. In this paper, we overcome this data bottleneck and advance the detection of MDs acute episode vs stable state from wearables data on the back of recent advances in self-supervised learning (SSL). This leverages unlabelled data to learn representations during pre-training, subsequently exploited for a supervised task. First, we collected open-access datasets recording with an Empatica E4 spanning different, unrelated to MD monitoring, personal sensing tasks -- from emotion recognition in Super Mario players to stress detection in undergraduates -- and devised a pre-processing pipeline performing on-/off-body detection, sleep-wake detection, segmentation, and (optionally) feature extraction. With 161 E4-recorded subjects, we introduce E4SelfLearning, the largest to date open access collection, and its pre-processing pipeline. Second, we show that SSL confidently outperforms fully-supervised pipelines using either our novel E4-tailored Transformer architecture (E4mer) or classical baseline XGBoost: 81.23% against 75.35% (E4mer) and 72.02% (XGBoost) correctly classified recording segments from 64 (half acute, half stable) patients. Lastly, we illustrate that SSL performance is strongly associated with the specific surrogate task employed for pre-training as well as with unlabelled data availability.

Photovoltaics (PVs) are a critical technology for curbing growing levels of anthropogenic greenhouse gas emissions, and meeting increases in future demand for low-carbon electricity. In order to fulfil ambitions for net-zero carbon dioxide equivalent (CO2eq) emissions worldwide, the global cumulative capacity of solar PVs must increase by an order of magnitude from 0.9 TWp in 2021 to 8.5 TWp by 2050 according to the International Renewable Energy Agency, which is considered to be a highly conservative estimate. In 2020, the Henry Royce Institute brought together the UK PV community to discuss the critical technological and infrastructure challenges that need to be overcome to address the vast challenges in accelerating PV deployment. Herein, we examine the key developments in the global community, especially the progress made in the field since this earlier roadmap, bringing together experts primarily from the UK across the breadth of the photovoltaics community. The focus is both on the challenges in improving the efficiency, stability and levelized cost of electricity of current technologies for utility-scale PVs, as well as the fundamental questions in novel technologies that can have a significant impact on emerging markets, such as indoor PVs, space PVs, and agrivoltaics. We discuss challenges in advanced metrology and computational tools, as well as the growing synergies between PVs and solar fuels, and offer a perspective on the environmental sustainability of the PV industry. Through this roadmap, we emphasize promising pathways forward in both the short- and long-term, and for communities working on technologies across a range of maturity levels to learn from each other., Comment: 160 pages, 21 figures

In previous work, we introduced a method for determining convergence rates for integration methods for the kinetic Langevin equation for $M$-$\nabla$Lipschitz $m$-log-concave densities [arXiv:2302.10684, 2023]. In this article, we exploit this method to treat several additional schemes including the method of Brunger, Brooks and Karplus (BBK) and stochastic position/velocity Verlet. We introduce a randomized midpoint scheme for kinetic Langevin dynamics, inspired by the recent scheme of Bou-Rabee and Marsden [arXiv:2211.11003, 2022]. We also extend our approach to stochastic gradient variants of these schemes under minimal extra assumptions. We provide convergence rates of $\mathcal{O}(m/M)$, with explicit stepsize restriction, which are of the same order as the stability thresholds for Gaussian targets and are valid for a large interval of the friction parameter. We compare the contraction rate estimates of many kinetic Langevin integrators from molecular dynamics and machine learning. Finally we present numerical experiments for a Bayesian logistic regression example.

NUBO, short for Newcastle University Bayesian Optimization, is a Bayesian optimization framework for optimizing expensive-to-evaluate black-box functions, such as physical experiments and computer simulators. Bayesian optimization is a cost-efficient optimization strategy that uses surrogate modeling via Gaussian processes to represent an objective function and acquisition functions to guide the selection of candidate points to approximate the global optimum of the objective function. NUBO focuses on transparency and user experience to make Bayesian optimization accessible to researchers from all disciplines. Clean and understandable code, precise references, and thorough documentation ensure transparency, while a modular and flexible design, easy-to-write syntax, and careful selection of Bayesian optimization algorithms ensure a good user experience. NUBO allows users to tailor Bayesian optimization to their problem by writing a custom optimization loop using the provided building blocks. It supports sequential single-point, parallel multi-point, and asynchronous optimization of bounded, constrained, and mixed (discrete and continuous) parameter input spaces. Only algorithms and methods extensively tested and validated to perform well are included in NUBO. This ensures that the package remains compact and does not overwhelm the user with an unnecessarily large number of options. The package is written in Python but does not require expert knowledge of Python to optimize simulators and experiments. NUBO is distributed as open-source software under the BSD 3-Clause license., Comment: Accepted for publication by the Journal of Statistical Software

We provide a framework to analyze the convergence of discretized kinetic Langevin dynamics for $M$-$\nabla$Lipschitz, $m$-convex potentials. Our approach gives convergence rates of $\mathcal{O}(m/M)$, with explicit stepsize restrictions, which are of the same order as the stability threshold for Gaussian targets and are valid for a large interval of the friction parameter. We apply this methodology to various integration schemes which are popular in the molecular dynamics and machine learning communities. Further, we introduce the property ``$\gamma$-limit convergent" (GLC) to characterize underdamped Langevin schemes that converge to overdamped dynamics in the high-friction limit and which have stepsize restrictions that are independent of the friction parameter; we show that this property is not generic by exhibiting methods from both the class and its complement. Finally, we provide asymptotic bias estimates for the BAOAB scheme, which remain accurate in the high-friction limit by comparison to a modified stochastic dynamics which preserves the invariant measure., Comment: 34 pages, 1 figure

Due to their high photovoltaic efficiency and low-cost synthesis, lead halide perovskites have attracted wide interest for application in new solar cell technologies. The most stable and efficient ABX$_3$ perovskite solar cells employ mixed A-site cations, however the impact of cation mixing on carrier trapping and recombination -- key processes that limit photovoltaic performance -- is not fully understood. Here we analyse non-radiative carrier trapping in the mixed A-cation hybrid halide perovskite MA$_{1-x}$Cs$_x$PbI$_3$. By using rigorous first-principles simulations we show that cation mixing leads to a hole trapping rate at the iodine interstitial that is eight orders of magnitude greater than in the single cation system. We demonstrate that the same defect in the same material can display a wide variety of defect activity -- from electrically inactive to recombination centre -- and, in doing so, resolve conflicting reports in the literature. Finally, we propose a new mechanism in which steric effects can be used to determine the rate of carrier trapping; this is achieved by controlling the phase and dynamical response of the lattice through the A-site composition. Our findings elucidate crucial links between chemical composition, defect activity and optoelectronic performance, and suggest a general approach that can help to rationalise the development of new crystalline materials with target defect properties.

Chalcogenide perovskites have been recently under the researchers spotlight as novel absorber materials for photovoltaic applications. BaZrS$_3$, the most investigated compound of this family, shows a high absorption coefficient, a bandgap of around 1.8 eV, and excellent environmental and thermal stability. In addition to the 3D perovskite BaZrS$_3$, the Ba-Zr-S compositional space contains various 2-D Ruddlesden-Popper phases Ba$_{x+1}$Zr$_x$S$_{3x+1}$ (with $x=$ 1, 2, 3) which have recently been reported. In this work it will be shown that at high temperature the Gibbs free energies of 3D and 2D perovskites are very close, suggesting that 2D phases can be easily formed at high temperatures. The analysis of the product of the BaS and ZrS$_2$ solid-state reaction, in different stoichiometric conditions, present a mixture of BaZrS$_3$ and Ba$_4$Zr$_3$S$_{10}$. To carefully resolve the composition, XRD, SEM and EDS analysis were complemented with Raman spectroscopy. For this purpose, the phonon modes, and the consequent Raman spectra, were calculated for the 3D and 2D chalcogenide perovskites, as well as for the binary precursors. This thorough characterization demonstrates the thermodynamic limitations and experimental difficulties in forming phase-pure chalcogenide perovskites through solid state synthesis, and the importance of using multiple techniques to soundly resolve the composition of these chalcogenide materials.