Search

Blood metabolite levels are affected by numerous factors, including preanalytical factors such as collection methods and geographical sites. These perturbations have caused deleterious consequences for many metabolomics studies and represent a major challenge in the metabolomics field. It is important to understand these factors and develop models to reduce their perturbations. However, to date, the lack of suitable mathematical models for blood metabolite levels under homeostasis has hindered progress. In this study, we develop quantitative models of blood metabolite levels in healthy adults based on multisite sample cohorts that mimic the current challenge. Five cohorts of samples obtained across four geographically distinct sites were investigated, focusing on approximately 50 metabolites that were quantified using 1H NMR spectroscopy. More than one-third of the variation in these metabolite profiles is due to cross-cohort variation. A dramatic reduction in the variation of metabolite levels (90%), especially their site-to-site variation (95%), was achieved by modeling each metabolite using demographic and clinical factors and especially other metabolites, as observed in the top principal components. The results also reveal that several metabolites contribute disproportionately to such variation, which could be explained by their association with biological pathways including biosynthesis and degradation. The study demonstrates an intriguing network effect of metabolites that can be utilized to better define homeostatic metabolite levels, which may have implications for improved health monitoring. As an example of the potential utility of the approach, we show that modeling gender-related metabolic differences retains the interesting variance while reducing unwanted (site-related) variance.

We in this article propose a novel non‐parametric estimator for the volatility function within a broad context that encompasses nonlinear time series models as a special case. The new estimator, built on the mode value, is designed to complement existing mean volatility measures to reveal distinct data features. We demonstrate that the suggested modal volatility estimator can be obtained asymptotically as well as if the conditional mean regression function were known, assuming observations are from a strictly stationary and absolutely regular process. Under mild regularity conditions, we establish that the asymptotic distributions of the resulting estimator align with those derived from independent observations, albeit with a slower convergence rate compared to non‐parametric mean regression. The theory and practice of bandwidth selection are discussed. Moreover, we put forward a variance reduction technique for the modal volatility estimator to attain asymptotic relative efficiency while maintaining the asymptotic bias unchanged. We numerically solve the modal regression model with the use of a modified modal‐expectation‐maximization algorithm. Monte Carlo simulations are conducted to assess the finite sample performance of the developed estimation procedure. Two real data analyses are presented to further illustrate the newly proposed model in practical applications. To potentially enhance the accuracy of the bias term, we in the end discuss the extension of the method to local exponential modal estimation. We showcase that the suggested exponential modal volatility estimator shares the same asymptotic variance as the non‐parametric modal volatility estimator but may exhibit a smaller bias. [ABSTRACT FROM AUTHOR]

This study compares the accuracy, efficiency, and reliability of variance reduction (VR) methods for Monte Carlo radiation transport simulations involving wide-area ground plane (i.e., "surface") and buried (i.e., "volumetric") gamma source emissions from environmental soil. The simulation models are idealized external exposure scenarios intended as a basis for deriving site-specific dose-based or carcinogenic risk–based regulatory limits in the radiological site remediation process. These simulations are computationally resource intensive since particle tracks are transported from an extremely large source region to a relatively small detector region. For each simulation, several VR methods are compared with metrics of accuracy, efficiency, and reliability. The MCNP deterministic transport (DXTRAN) VR method was most effective for problems involving sources emitting low-energy gamma rays, and a coupled multicode method was more effective for problems involving sources emitting higher-energy gamma rays that undergo significant attenuation in the soil. [ABSTRACT FROM AUTHOR]

The unstructured mesh (UM)–based Monte Carlo (MC) method can utilize modern computer-aided-design/computer-aided-engineering platforms to obtain geometric models with reduced human effort and is capable of generating high-resolution tally data. This approach presents a significant advantage over the traditional Constructive Solid Geometry (CSG)–based MC method in handling complex geometries and conducting multiphysics calculations. In this study, the UM-based MC calculation capability was developed in the MC code NECP-MCX. On this basis, an automatic UM-based Consistent Adjoint-Driven Importance Sampling (CADIS) method was further studied and implemented in which the adjoint deterministic calculation, forward MC calculation, and variance reduction (VR) parameter generation were performed on the unified UM model. To achieve this, the discrete ordinates (SN)–Discontinuous Finite Element Method (DFEM) code NECP-SUN was embedded into NECP-MCX as the adjoint transport solver. Validations of the developed code and evaluations of the VR performance of the UM-based CADIS method were conducted on the Pool Critical Assembly (PCA) Replica benchmark and H. B. Robinson Unit 2 (HBR-2) benchmark. The numerical results indicated that the developed UM-based particle tracking capability achieved comparable accuracy to the CSG-based approach. Furthermore, compared to the traditional CADIS method, the UM-based CADIS method demonstrated higher figure-of-merit (FOM) values (3.5 to 44 times higher for the PCA Replica benchmark and 2.22 to 2.92 times higher for the HBR-2 benchmark), highlighting the superior VR performance of the UM-based CADIS method over the traditional CADIS method. [ABSTRACT FROM AUTHOR]

Natural policy gradient (NPG) and its variants are widely-used policy search methods in reinforcement learning. Inspired by prior work, a new NPG variant coined NPG-HM is developed in this paper, which utilizes the Hessian-aided momentum technique for variance reduction, while the sub-problem is solved via the stochastic gradient descent method. It is shown that NPG-HM can achieve the global last iterate ε -optimality with a sample complexity of O (ε - 2) , which is the best known result for natural policy gradient type methods under the generic Fisher non-degenerate policy parameterizations. The convergence analysis is built upon a relaxed weak gradient dominance property tailored for NPG under the compatible function approximation framework, as well as a neat way to decompose the error when handling the sub-problem. Moreover, numerical experiments on Mujoco-based environments demonstrate the superior performance of NPG-HM over other state-of-the-art policy gradient methods. [ABSTRACT FROM AUTHOR]

AbstractStratified sampling is one of the powerful variance reduction methods for analyzing system performance, such as reliability, with stochastic simulation. It divides the input space into disjoint subsets, called strata, to draw samples from each stratum. Partitioning the input space properly and allocating greater computational effort to crucial strata can help accurately estimate system performance with a limited computational budget. How to create strata, however, has yet to be thoroughly examined. Strata design faces the curse of dimensionality and data scarcity as the input dimension increases. We analytically derive the optimal stratification structure that minimizes the estimation variance for univariate problems. Further, reconciling the optimal stratification into decision trees, we devise a robust algorithm for multi-dimensional problems. Numerical experiments and a wind turbine case study demonstrate the superiority of the proposed method in terms of variance reduction, leading to computational efficiency and scalability. [ABSTRACT FROM AUTHOR]

Policy Gradient (PG) method is one of the most popular algorithms in Reinforcement Learning (RL). However, distributed adaptive variants of PG are rarely studied in multi-agent. For this reason, this paper proposes a distributed adaptive policy gradient algorithm (IS-DAPGM) incorporated with Adam-type updates and importance sampling technique. Furthermore, we also establish the theoretical convergence rate of O (1 / T) , where T represents the number of iterations, it can match the convergence rate of the state-of-the-art centralized policy gradient methods. In addition, many experiments are conducted in a multi-agent environment, which is a modification on the basis of Particle world environment. By comparing with some other distributed PG methods and changing the number of agents, we verify the performance of IS-DAPGM is more efficient than the existing methods. [ABSTRACT FROM AUTHOR]

Options pricing remains a critical aspect of finance, dominated by traditional models such as Black-Scholes and binomial tree. However, as market dynamics become more complex, numerical methods such as Monte Carlo simulation are accommodating uncertainty and offering promising alternatives. In this paper, we examine how effective different options pricing methods, from traditional models to machine learning algorithms, are at predicting KOSPI200 option prices and maximizing investment returns. Using a dataset of 2023, we compare the performance of models over different time frames and highlight the strengths and limitations of each model. In particular, we find that machine learning models are not as good at predicting prices as traditional models but are adept at identifying undervalued options and producing significant returns. Our findings challenge existing assumptions about the relationship between forecast accuracy and investment profitability and highlight the potential of advanced methods in exploring dynamic financial environments. [ABSTRACT FROM AUTHOR]

Particle methods based on evolving the spatial derivatives of the solution were originally introduced to simulate reaction-diffusion processes, inspired by vortex methods for the Navier–Stokes equations. Such methods, referred to as gradient random walk methods, were extensively studied in the '90s and have several interesting features, such as being grid-free, automatically adapting to the solution by concentrating elements where the gradient is large, and significantly reducing the variance of the standard random walk approach. In this work, we revive these ideas by showing how to generalize the approach to a larger class of partial differential equations, including hyperbolic systems of conservation laws. To achieve this goal, we first extend the classical Monte Carlo method to relaxation approximation of systems of conservation laws, and subsequently consider a novel particle dynamics based on the spatial derivatives of the solution. The methodology, combined with asymptotic-preserving splitting discretization, yields a way to construct a new class of gradient-based Monte Carlo methods for hyperbolic systems of conservation laws. Several results in one spatial dimension for scalar equations and systems of conservation laws show that the new methods are very promising and yield remarkable improvements compared to standard Monte Carlo approaches, either in terms of variance reduction as well as in describing the shock structure. [ABSTRACT FROM AUTHOR]

Several variance-reduced versions of REINFORCE based on importance sampling achieve an improved O (ϵ - 3) sample complexity to find an ϵ -stationary point, under an unrealistic assumption on the variance of the importance weights. In this paper, we propose the Defensive Policy Gradient (DEF-PG) algorithm, based on defensive importance sampling, achieving the same result without any assumption on the variance of the importance weights. We also show that this is not improvable by establishing a matching Ω (ϵ - 3) lower bound, and that REINFORCE with its O (ϵ - 4) sample complexity is actually optimal under weaker assumptions on the policy class. Numerical simulations show promising results for the proposed technique compared to similar algorithms based on vanilla importance sampling. [ABSTRACT FROM AUTHOR]

Hypergraphs can represent higher-order relations among objects. Traditional hypergraph neural networks involve node-edge-node transform, leading to high computational cost and timing. The main aim of this article is to propose a new sampling technique for learning with hypergraph neural networks. The core idea is to design a layer-wise sampling scheme for nodes and hyperedges to approximate the original hypergraph convolution. We rewrite hypergraph convolution in the form of double integral and leverage Monte Carlo to achieve a discrete and consistent estimator. In addition, we use importance sampling and finally derive feasible probability mass functions for both nodes and hyperedges in consideration of variance reduction, based on some assumptions. Notably, the proposed sampling technique allows us to handle large-scale hypergraph learning, which is not feasible with traditional hypergraph neural networks. Experiment results demonstrate that our proposed model keeps a good balance between running time and prediction accuracy. [ABSTRACT FROM AUTHOR]

Probability estimation of tree topologies is one of the fundamental tasks in phylogenetic inference. The recently proposed subsplit Bayesian networks (SBNs) provide a powerful probabilistic graphical model for tree topology probability estimation by properly leveraging the hierarchical structure of phylogenetic trees. However, the expectation maximization method currently used for learning SBN parameters does not scale up to large data sets. In this paper, we introduce several computationally efficient methods for training SBNs and show that variance reduction could be the key for better performance. Furthermore, we also introduce the variance reduction technique to improve the optimization of SBN parameters for variational Bayesian phylogenetic inference. Extensive synthetic and real data experiments demonstrate that our methods outperform previous baseline methods on the tasks of tree topology probability estimation as well as Bayesian phylogenetic inference using SBNs. [ABSTRACT FROM AUTHOR]

Abstract Policy Gradient (PG) method is one of the most popular algorithms in Reinforcement Learning (RL). However, distributed adaptive variants of PG are rarely studied in multi-agent. For this reason, this paper proposes a distributed adaptive policy gradient algorithm (IS-DAPGM) incorporated with Adam-type updates and importance sampling technique. Furthermore, we also establish the theoretical convergence rate of $$\mathcal {O}(1/\sqrt{T})$$ O ( 1 / T ) , where T represents the number of iterations, it can match the convergence rate of the state-of-the-art centralized policy gradient methods. In addition, many experiments are conducted in a multi-agent environment, which is a modification on the basis of Particle world environment. By comparing with some other distributed PG methods and changing the number of agents, we verify the performance of IS-DAPGM is more efficient than the existing methods.

This paper presents two new methods for variance reduction for shielding calculation in Monte Carlo radiation transport. One method is CADIS-NEE, which combines Consistent Adjoint Driven Importance Sampling (CADIS) and next-event estimator (NEE) methods to increase the calculation efficiency of tallies at points. The other is CADIS-deterministic transport (DXTRAN), which combines CADIS and DXTRAN to obtain higher performance than using CADIS and DXTRAN separately. The combination processes are derived and implemented in the hybrid Monte-Carlo-Deterministic particle-transport code NECP-MCX. Various problems are tested to demonstrate the effectiveness of the two methods. According to the results, the two combination methods have higher efficiency than using CADIS, NEE or DXTRAN separately. In a long-distance photon-transport problem, CADIS-NEE converges faster than NEE and the figure of merit (FOM) of CADIS-NEE is 75.6 times of NEE. In a labyrinthine problem, CADIS-DXTRAN's FOM surpasses that of DXTRAN and CADIS by a factor of 45.3 and 17.7, respectively. Therefore, it is advisable to employ these two novel methods selectively in appropriate scenarios to reduce variance.

AbstractThe sampling importance resampling method is widely utilized in various fields, such as numerical integration and statistical simulation. In this paper, two modified methods are presented by incorporating two variance reduction techniques commonly used in Monte Carlo simulation, namely antithetic sampling and Latin hypercube sampling, into the process of sampling importance resampling method, respectively. Theoretical evidence is provided to demonstrate that the proposed methods significantly reduce estimation errors compared to the original approach. Furthermore, the effectiveness and advantages of the proposed methods are validated through both numerical studies and real data analysis. [ABSTRACT FROM AUTHOR]

Significant risks in project completion delay could be buried in the development of project schedules because of theoretical flaws in project schedule risk analysis. Major project delays hamper infrastructure development endeavors and cause negative consequences to project finance, the public interest, and socioeconomic growth. This research critically reviews the theoretical foundation of the Program Evaluation and Review Technique (PERT) and identifies two major flaws: (1) failing to account for variances of activities on subcritical paths; and (2) lacking the functionality of characterizing variance-criticality for activities in a project network. Further, the path variance-criticality index and the activity variance-criticality index are formalized for defining the subcritical path(s) and identifying the variance-critical activities. Reducing time duration variances in the identified variance-critical activities exerts a significant impact on mitigating project schedule delay risks. A case study is given to illustrate the analytical steps for identifying variance-critical activities in a project. Monte Carlo simulations were conducted to validate the effectiveness of the proposed analytical approach. The enhanced PERT for project schedule risk analysis is instrumental in (1) identifying critical and subcritical paths in the project network model; (2) clarifying the notion of variance-criticality for prioritization of activities based on the impact of activity time variance on project time variance; and (3) reining in project schedule risks by reducing time duration variances on those variance-critical activities in project planning. [ABSTRACT FROM AUTHOR]

In this paper, we investigate a variance reduced forward-reflected-backward algorithm for solving nonconvex finite-sum mixed variational inequalities, where $ F $ is Lipschitz continuous, $ h $ is proper lower semicontinuous prox-convex, and they satisfy a generalized monotonicity condition on $ C $. We prove that the algorithm is almost surely convergent and exhibits $ R $-linear rate when $ F $ is strongly monotone. [ABSTRACT FROM AUTHOR]

The blocky optimization has gained a significant amount of attention in far-reaching practical applications. Following the recent work (M. Nikolova and P. Tan [SIAM J. Optim. 29 (2019), pp. 2053–2078]) on solving a class of nonconvex nonsmooth optimization, we develop a stochastic alternating structure-adapted proximal (s-ASAP) gradient descent method for solving blocky optimization problems. By deploying some state-of-the-art variance reduced gradient estimators (rather than full gradient) in stochastic optimization, the s-ASAP method is applicable to nonconvex optimization whose objective is the sum of a nonsmooth data-fitting term and a finite number of differentiable functions. The sublinear convergence rate of s-ASAP is built upon the proximal point algorithmic framework, whilst the linear convergence rate of s-ASAP is achieved under the error bound condition. Furthermore, the convergence of the sequence produced by s-ASAP is established under the Kurdyka-Łojasiewicz property. Preliminary numerical simulations on some image processing applications demonstrate the compelling performance of the proposed method. [ABSTRACT FROM AUTHOR]

AbstractIn the context of designing radiotherapy facilities, typical dose estimation methods involve analytical approaches, as outlined in International Atomic Energy Agency (IAEA) Safety Reports Series No. 47 (IAEA 47). These methods are known for their ease of use and rapid calculations, but they could lead to either overestimation or underestimation of radiation doses. Hence, the integration of Monte Carlo (MC) methods is considered valuable. In this particular study, a radiotherapy facility was modeled using MCNP version 6.2, and dose calculations were conducted using analytical techniques following both IAEA 47 guidelines and MC simulations. The study focused on monoenergetic photon cone beams with energies of 10 and 15 MeV. Notably, the beam’s orientation prevented primary radiation from reaching the dose location at the entrance of the maze, allowing only scatter radiation to contribute to the tally. Given the challenges associated with obtaining reliable and accurate results through standard MCNP calculations, the investigation focused on the use of weight windows as a variance reduction technique. The findings revealed that the IAEA method tends to provide conservative results only when the same conditions were replicated in the MC simulations. In fact, approximately 50% of the final dose estimated through MC methods accounted for factors that were not considered in the analytical calculations. The primary contributor to scattering (averaging around 30%) was identified as the floor and ceiling. This study underscores the need for caution when relying solely on the analytical approach, as it may not consistently yield conservative outcomes. [ABSTRACT FROM AUTHOR]

Objective. We provide optimal particle split numbers for speeding up TOPAS Monte Carlo simulations of linear accelerator (linac) treatment heads while maintaining accuracy. In addition, we provide a new TOPAS physics module for simulating photoneutron production and transport. Approach. TOPAS simulation of a Siemens Oncor linac was used to determine the optimal number of splits for directional bremsstrahlung splitting as a function of the field size for 6 MV and 18 MV x-ray beams. The linac simulation was validated against published data of lateral dose profiles and percentage depth-dose curves (PDD) for the largest square field (40 cm side). In separate simulations, neutron particle split and the custom TOPAS physics module was used to generate and transport photoneutrons, called 'TsPhotoNeutron'. Verification of accuracy was performed by comparing simulations with published measurements of: (1) neutron yields as a function of beam energy for thick targets of Al, Cu, Ta, W, Pb and concrete; and (2) photoneutron energy spectrum at 40 cm laterally from the isocenter of the Oncor linac from an 18 MV beam with closed jaws and MLC. Main results. The optimal number of splits obtained for directional bremsstrahlung splitting enhanced the computational efficiency by two orders of magnitude. The efficiency decreased with increasing beam energy and field size. Calculated lateral profiles in the central region agreed within 1 mm/2% from measured data, PDD curves within 1 mm/1%. For the TOPAS physics module, at a split number of 146, the efficiency of computing photoneutron yields was enhanced by a factor of 27.6, whereas it improved the accuracy over existing Geant4 physics modules. Significance. This work provides simulation parameters and a new TOPAS physics module to improve the efficiency and accuracy of TOPAS simulations that involve photonuclear processes occurring in high- Z materials found in linac components, patient devices, and treatment rooms, as well as to explore new therapeutic modalities such as very-high energy electron therapy. [ABSTRACT FROM AUTHOR]

Robust face clustering enjoys a wide range of applications for gate passes, surveillance systems and security analysis in embedded sensors. Nevertheless, existing algorithms have limitations in finding accurate clusters when data contain noise (e.g., occluded face clustering and recognition). It is known that in subspace clustering, the ℓ 1 - and ℓ 2 -norm regularizers can improve subspace preservation and connectivity, respectively, and the elastic net regularizer (i.e., the mixture of the ℓ 1 - and ℓ 2 -norms) provides a balance between the two properties. However, existing deterministic methods have high per iteration computational complexities, making them inapplicable to large-scale problems. To address this issue, this paper proposes the first accelerated stochastic variance reduction gradient (RASVRG) algorithm for robust subspace clustering. We also introduce a new momentum acceleration technique for the RASVRG algorithm. As a result of the involvement of this momentum, the RASVRG algorithm achieves both the best oracle complexity and the fastest convergence rate, and it reaches higher efficiency in practice for both strongly convex and not strongly convex models. Various experimental results show that the RASVRG algorithm outperformed existing state-of-the-art methods with elastic net and ℓ 1 -norm regularizers in terms of accuracy in most cases. As demonstrated on real-world face datasets with different manually added levels of pixel corruption and occlusion situations, the RASVRG algorithm achieved much better performance in terms of accuracy and robustness. [ABSTRACT FROM AUTHOR]

For large-scale optimization that covers a wide range of optimization problems encountered frequently in machine learning and deep neural networks, stochastic optimization has become one of the most used methods thanks to its low computational complexity. In machine learning and deep learning problems, nonconvex problems are common, while convex problems are rare. How to find the global minimum for nonconvex optimization and reduce the computational complexity are challenges. Inspired by the phenomenon that the stagewise stepsize tuning strategy can empirically improve the convergence speed in deep neural networks, we incorporate the stagewise stepsize tuning strategy into the iterative framework of Nesterov's acceleration- and variance reduction-based methods to reduce the computational complexity, i.e., the stagewise stepsize tuning strategy is incorporated into randomized stochastic accelerated gradient and stochastic variance-reduced gradient. The proposed methods are theoretically derived to reduce the complexity of the nonconvex and convex problems and improve the convergence rate of the frameworks, which have the complexity O (1 / μ ϵ) and O (1 / μ ϵ) , respectively, where μ is the PL modulus and L is the Lipschitz constant. In the end, numerical experiments on large benchmark datasets validate well the competitiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of minimizing the sum of a large number of smooth convex functions subject to a complicated constraint set defined by a smooth convex function. Such a problem has wide applications in many areas, such as machine learning and signal processing. By utilizing variance reduction and moving balls approximation techniques, we propose a new variance reduced moving balls approximation method. Compared with existing convergence rates of moving balls approximation-type methods that require the strong convexity of the objective function, a notable advantage of the proposed method is that the linear and sublinear convergence rates can be guaranteed under the quadratic gradient growth property and convexity condition, respectively. To demonstrate its effectiveness, numerical experiments for solving the smooth regularized logistic regression problem and the Neyman-Pearson classification problem are presented. [ABSTRACT FROM AUTHOR]

As a new type of energy system, thermoelectric isotope battery system is widely used in deep space and deep sea exploration missions. Strontium titanate fuel is used as the heat source for thermoelectric isotope batteries. The radioactive isotopes of 90Sr and 90Y in the strontium titanate heat source emit β ray, which interact with substances to produce bremsstrahlung radiation. Due to the strong penetrating power of bremsstrahlung radiation, it is necessary to install a shielding layer on the surface of the isotope radioactive source strontium titanate to reduce the radiation dose of the thermoelectric isotope batteries. When the thickness of the shielding layer exceeds a certain value, the statistical error of the Monte Carlo method increases significantly. Variance reduction technique can be used to improve the accuracy of shielding calculation for thermoelectric isotope batteries. At the same time, the direct electron transport simulation takes a long time, which is much longer than the photon simulation, and the photons produced by the electrons can be processed by the thick target bremsstrahlung (TTB) model, thus improving the computational efficiency. In this paper, the variance reduction technology of cosRMC program and the application of TTB radiation model in the shielding calculation of isotope batteries were studied and verified. Based on the design scheme of SNAP-21 structure in the United States, the thermoelectric isotope battery is modeled by cosRMC program. The influence of depleted uranium on the surface dose rate was analyzed by changing the thickness of the radial and axial outer shielding layers of the thermoelectric isotope battery, and the calculated results were compared with those of MCNP program. The comparison of the two programs shows that the statistical error of dose rate of depleted uranium shield surface calculated by cosRMC and MCNP is in good agreement with ±3σ range when different thickness shielding layers were used in radial and axial directions of thermoelectric isotope batteries. In order to reduce the statistical error of the calculated results, the cell importance variance reduction was used. The results show that the statistical error of the cosRMC and MCNP programs decreases at the radial 5 cm and axial 4 cm of the battery by 52.515% and 39.338%, respectively. The dose rate relative errors calculated by cosRMC and MCNP are in good agreement within the range of ±3σ, which further verifies the accuracy of cosRMC program in the calculation of thermoelectric isotope battery shielding.

This work considers the non-convex finite-sum minimization problem. There are several algorithms for such problems, but existing methods often work poorly when the problem is badly scaled and/or ill-conditioned, and a primary goal of this work is to introduce methods that alleviate this issue. Thus, here we include a preconditioner based on Hutchinson's approach to approximating the diagonal of the Hessian and couple it with several gradient-based methods to give new 'scaled' algorithms: Scaled SARAH and Scaled L-SVRG. Theoretical complexity guarantees under smoothness assumptions are presented. We prove linear convergence when both smoothness and the PL-condition are assumed. Our adaptively scaled methods use approximate partial second-order curvature information and, therefore, can better mitigate the impact of badly scaled problems. This improved practical performance is demonstrated in the numerical experiments also presented in this work. [ABSTRACT FROM AUTHOR]

Stochastic optimization has experienced significant growth in recent decades, with the increasing prevalence of variance reduction techniques in stochastic optimization algorithms to enhance computational efficiency. In this paper, we introduce two projection-free stochastic approximation algorithms for maximizing diminishing return (DR) submodular functions over convex constraints, building upon the Stochastic Path Integrated Differential EstimatoR (SPIDER) and its variants. Firstly, we present a SPIDER Continuous Greedy (SPIDER-CG) algorithm for the monotone case that guarantees a (1 - e - 1) OPT - ε approximation after O (ε - 1) iterations and O (ε - 2) stochastic gradient computations under the mean-squared smoothness assumption. For the non-monotone case, we develop a SPIDER Frank–Wolfe (SPIDER-FW) algorithm that guarantees a 1 4 (1 - min x ∈ C ‖ x ‖ ∞) OPT - ε approximation with O (ε - 1) iterations and O (ε - 2) stochastic gradient estimates. To address the practical challenge associated with a large number of samples per iteration, we introduce a modified gradient estimator based on SPIDER, leading to a Hybrid SPIDER-FW (Hybrid SPIDER-CG) algorithm, which achieves the same approximation guarantee as SPIDER-FW (SPIDER-CG) algorithm with only O (1) samples per iteration. Numerical experiments on both simulated and real data demonstrate the efficiency of the proposed methods. [ABSTRACT FROM AUTHOR]

Using quasi-Newton methods in stochastic optimization is not a trivial task given the difficulty of extracting curvature information from the noisy gradients. Moreover, pre-conditioning noisy gradient observations tend to amplify the noise. We propose a Bayesian approach to obtain a Hessian matrix approximation for stochastic optimization that minimizes the secant equations residue while retaining the extreme eigenvalues between a specified range. Thus, the proposed approach assists stochastic gradient descent to converge to local minima without augmenting gradient noise. We propose maximizing the log posterior using the Newton-CG method. Numerical results on a stochastic quadratic function and an $ \ell _2 $ ℓ2-regularized logistic regression problem are presented. In all the cases tested, our approach improves the convergence of stochastic gradient descent, compensating for the overhead of solving the log posterior maximization. In particular, pre-conditioning the stochastic gradient with the inverse of our Hessian approximation becomes more advantageous the larger the condition number of the problem is. [ABSTRACT FROM AUTHOR]

Variational Bayes methods approximate the posterior density by a family of tractable distributions whose parameters are estimated by optimization. Variational approximation is useful when exact inference is intractable or very costly. Our article develops a flexible variational approximation based on a copula of a mixture, which is implemented by combining boosting, natural gradient, and a variance reduction method. The efficacy of the approach is illustrated by using simulated and real datasets to approximate multimodal, skewed and heavy-tailed posterior distributions, including an application to Bayesian deep feedforward neural network regression models. , including appendices and computer code for this article, are available online. [ABSTRACT FROM AUTHOR]

The StochAstic Recursive grAdient algoritHm (SARAH) algorithm is a variance reduced variant of the Stochastic Gradient Descent algorithm that needs a gradient of the objective function from time to time. In this paper, we remove the necessity of a full gradient computation. This is achieved by using a randomized reshuffling strategy and aggregating stochastic gradients obtained in each epoch. The aggregated stochastic gradients serve as an estimate of a full gradient in the SARAH algorithm. We provide a theoretical analysis of the proposed approach and conclude the paper with numerical experiments that demonstrate the efficiency of this approach. [ABSTRACT FROM AUTHOR]

To minimize the variance of Monte Carlo estimators, we develop a novel exponential embedding technique that extends the classical concept of sufficient statistics in importance sampling. Our method demonstrates bounded relative error and logarithmic efficiency when applied to normal and gamma distributions, especially in rare event scenarios. To illustrate this innovative technique, we address the problem of credit risk measurement in portfolios and present an efficient simulation algorithm to estimate the likelihood of significant portfolio losses, leveraging multi-factor models with a normal mixture copula. Finally, supported by comprehensive simulation studies, our approach offers a more effective and efficient way to simulate moderately rare events. [ABSTRACT FROM AUTHOR]

AbstractWe consider extreme value analysis in a semi-supervised setting, where we observe, next to the n data on the target variable, n + m data on one or more covariates. This is called the semi-supervised model with n labeled and m unlabeled data. By exploiting the tail dependence between the target variable and the covariates, we derive estimators for the extreme value index and extreme quantiles of the target variable in this setting and establish their asymptotic behavior. Our estimators substantially improve the univariate estimators, based on only the n target variable data, in terms of asymptotic variances whereas the asymptotic biases remain unchanged. A simulation study confirms the substantially improved behavior of both estimators. Finally the estimation method is applied to rainfall data in France. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work. [ABSTRACT FROM AUTHOR]

Sample average approximation (SAA), the standard approach to stochastic mixed-integer programming, does not provide guidance for cases with limited computational budgets. In such settings, variance reduction is critical in identifying good decisions. This paper explores two closely related ensemble methods to determine effective decisions with a probabilistic guarantee. (a) The first approach recommends a decision by coordinating aggregation in the space of decisions as well as aggregation of objective values. This combination of aggregation methods generalizes the bagging method and the "compromise decision" of stochastic linear programming. Combining these concepts, we propose a stopping rule that provides an upper bound on the probability of early termination. (b) The second approach applies efficient computational budget allocation for objective function evaluation and contributes to identifying the best solution with a predicted lower bound on the probability of correct selection. It also reduces the variance of the upper bound estimate at optimality. Furthermore, it adaptively selects the evaluation sample size. Both approaches provide approximately optimal solutions even in cases with a huge number of scenarios, especially when scenarios are generated by using oracles/simulators. Finally, we demonstrate the effectiveness of these methods via extensive computational results for "megascale" (extremely large scale) stochastic facility location problems. History: Accepted by Pascal Van Hentenryck, Area Editor for Computational Modeling: Methods & Analysis. Funding: This work was supported by The Office of Naval Research [Grant N00014-20-1-2077] and the Air Force Office of Scientific Research [Grant FA9550-20-1-0006]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information (https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2021.0324) as well as from the IJOC GitHub software repository (https://github.com/INFORMSJoC/2021.0324). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/. [ABSTRACT FROM AUTHOR]

Monte Carlo criticality simulations are widely used in nuclear safety demonstrations, as they offer an arbitrarily precise estimation of global and local tallies while making very few assumptions. However, since the inception of such numerical approaches, it is well known that bias might affect both the estimation of errors on these tallies and the tallies themselves. In particular, stochastic modeling approaches developed in the past decade have shed light on the prominent role played by spatial correlations through a phenomenon called neutron clustering. This effect is particularly of great significance when simulating loosely coupled systems (i.e., with a high dominance ratio). In order to tackle this problem, this paper proposes to recast the power iteration technique of Monte Carlo criticality codes into a variance reduction technique called Adaptive Multilevel Splitting. The central idea is that iterating over neutron generations can be seen as pushing a subpopulation of neutrons toward a generational detector (instead of a spatial detector as variance reduction techniques usually do). While both approaches allow for neutron population control, the former blindly removes or splits neutrons. In contrast, the latter optimizes the spatial, generational, and spectral attributes of neutrons when they are removed or split through an adjoint flux estimation, hence tempering both generational and spatial correlations. This is illustrated in the present paper with a simple case of a bare slab reactor in the one-speed theory on which the Adaptive Multilevel Splitting was applied and compared to variations of the Monte Carlo power iteration method used in neutron transport. Besides looking at the resulting efficiency of the methods, this work also aims to highlight the main mechanisms of the Adaptive Multilevel Splitting in criticality calculations. [ABSTRACT FROM AUTHOR]

The research presented herein outlines a comprehensive process for characterizing the major radiological source terms necessary for radiation protection and licensing activities that one would expect in a liquid-fueled molten salt reactor. This process leverages organic simulation tools in the SCALE modeling and simulation code suite to provide an "off-the-shelf" solution for shielding assessments of this reactor type. Ultimately, this source development process is applied to a representative molten salt reactor system to assess the impact of ex-core source terms on shielding in varying operating conditions. The results of the analysis determined that while the prompt core source is the major dose contributor outside the radiological shielding, specific ex-core features, such as the primary salt loop components and configuration, can have an appreciable dose impact, and thus must be accounted for. [ABSTRACT FROM AUTHOR]

Copyright of Atomic Energy Science & Technology is the property of Editorial Board of Atomic Energy Science & Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Simplices are the fundamental domain when integrating over convex polytopes. The aim of this work is to establish a novel framework of Monte Carlo integration over simplices, throughout from sampling to variance reduction. Namely, we develop a uniform sampling method on the standard simplex consisting of two independent procedures and construct theories on change of measure on each of the two independent elements in the developed sampling technique with a view towards variance reduction by importance sampling. We provide illustrative figures and numerical results to support our theoretical findings and demonstrate the strong potential of the developed framework for effective implementation and acceleration of Monte Carlo integration over simplices. [ABSTRACT FROM AUTHOR]

Blood metabolite levels are affected by numerous factors, including preanalytical factors such as collection methods and geographical sites. These perturbations have caused deleterious consequences for many metabolomics studies and represent a major challenge in the metabolomics field. It is important to understand these factors and develop models to reduce their perturbations. However, to date, the lack of suitable mathematical models for blood metabolite levels under homeostasis has hindered progress. In this study, we develop quantitative models of blood metabolite levels in healthy adults based on multisite sample cohorts that mimic the current challenge. Five cohorts of samples obtained across four geographically distinct sites were investigated, focusing on approximately 50 metabolites that were quantified using 1H NMR spectroscopy. More than one-third of the variation in these metabolite profiles is due to cross-cohort variation. A dramatic reduction in the variation of metabolite levels (90%), especially their site-to-site variation (95%), was achieved by modeling each metabolite using demographic and clinical factors and especially other metabolites, as observed in the top principal components. The results also reveal that several metabolites contribute disproportionately to such variation, which could be explained by their association with biological pathways including biosynthesis and degradation. The study demonstrates an intriguing network effect of metabolites that can be utilized to better define homeostatic metabolite levels, which may have implications for improved health monitoring. As an example of the potential utility of the approach, we show that modeling gender-related metabolic differences retains the interesting variance while reducing unwanted (site-related) variance. [ABSTRACT FROM AUTHOR]

This study investigates importance sampling under the scheme of minibatch stochastic gradient descent, under which the contributions are twofold. First, theoretically, we develop a neat tilting formula, which can be regarded as a general device for asymptotically optimal importance sampling. Second, practically, guided by the formula, we present an effective algorithm for importance sampling which accounts for the effects of minibatches and leverages the Markovian property of the gradients between iterations. Experiments conducted on artificial data confirm that our algorithm consistently delivers superior performance in terms of variance reduction. Furthermore, experiments carried out on real-world data demonstrate that our method, when paired with relatively straightforward models like multilayer perceptron and convolutional neural networks, outperforms in terms of training loss and testing error. [ABSTRACT FROM AUTHOR]

The implicit trace estimation problem asks for an approximation of the trace of a square matrix, accessed via matrix-vector products (matvecs). This paper designs new randomized algorithms, XTrace and XNysTrace, for the trace estimation problem by exploiting both variance reduction and the exchangeability principle. For a fixed budget of matvecs, numerical experiments show that the new methods can achieve errors that are orders of magnitude smaller than existing algorithms, such as the Girard--Hutchinson estimator or the Hutch++ estimator. A theoretical analysis confirms the benefits by offering a precise description of the performance of these algorithms as a function of the spectrum of the input matrix. The paper also develops an exchangeable estimator, XDiag, for approximating the diagonal of a square matrix using matvecs. [ABSTRACT FROM AUTHOR]

This article introduces an efficient stochastic method to approximate the solution to Poisson's Equation (PE). The method starts by computing sparse Transition Probability Matrices (TPMs), assembled by averaging short continuous random walks based on the transient diffusion equation with a source. The TPMs can be used within a false-transient approach to approximate the solution to PE in the steady-state limit. The method's efficiency and memory usage are comparable to the Finite-Volume method. However, the ease of implementation, stability, and versatility makes the method attractive in complex scenarios. Moreover, given that the code can be easily implemented in parallel, it displays superior efficiency and practical importance in scenarios where a large number of nodes is mandatory. [ABSTRACT FROM AUTHOR]

The fully-connected tensor network (FCTN) decomposition is an emerging method for processing and analyzing higher-order tensors. For an Nth-order tensor, the standard deterministic algorithms, such as alternating least squares (FCTN-ALS) algorithm, need to store large coefficient matrices formed by contracting N - 1 FCTN factor tensors. The memory cost of coefficient matrices grows exponentially with the size of the original tensor, which makes the algorithms memory-prohibitive for handling large-scale tensors. To enable FCTN decomposition to handle large-scale tensors effectively, we propose a stochastic gradient descent (FCTN-SGD) algorithm without sacrificing accuracy. The memory cost of FCTN-SGD algorithm grows linearly with the size of the original tensor and is significantly lower than that of the FCTN-ALS algorithm. The success of the FCTN-SGD algorithm lies in the suggested factor sampling operator, which cleverly avoids storing large coefficient matrices in the algorithm. By using the suggested operator, sampling on small factor tensors is equal to sampling on large coefficient matrices with a theoretical guarantee. Furthermore, we present an FCTN-VRSGD algorithm by introducing variance reduction into the FCTN-SGD algorithm, and theoretically prove the convergence of the FCTN-VRSGD algorithm under a mild assumption. Numerical experiments demonstrate the efficiency and accuracy of the proposed FCTN-SGD and FCTN-VRSGD algorithms, especially for real-world large-scale tensors. [ABSTRACT FROM AUTHOR]

The article summarizes a thesis which examined the theory and application of Monte Carlo methods and variance reduction techniques in structural reliability analysis. The study investigated sampling and change of measure on simplices, batching adaptive variance reduction, dynamic finite-budget allocation of stratified sampling with adaptive variance reduction by strata, adaptive stratified sampling for structural reliability analysis, and adaptive stratified directional importance sampling.

In order to better cope with large-scale high-dimensional sparse data sets in today’s era, this paper combines the advantages of BB method, mini-batch algorithm and stochastic variance reduced gradient（SVRG）, and proposes a mini-batch sparse stochastic variance reduced gradient method（MSSVRG-R2BB） with randomly improved BarzilaiBorwein step size. First, on the basis of calculating the total gradient in the outer loop of SVRG, this article adds the subgradient of L1 norm, designs a sparse approximate gradient for the inner loop of SVRG, and obtains a sparse SVRG algorithm（SSVRG）. On this basis, this paper propose to use the improved BB method of random selection to automatically calculate and update the step size in the mini-batch sparse stochastic variance reduced gradient method, which solves the step size selection problem of the mini-batch algorithm, and expands the MSSVRG-R2BB algorithm. Numerical experiments show that MSSVRG-R2BB algorithm can not only reduce the operation cost and reach the convergence upper bound faster, but also achieve the same optimization level of other advanced small-batch algorithms, and perform stably and well for different initial parameter selection. [ABSTRACT FROM AUTHOR]

Non-convex optimization, which can better capture the problem structure, has received considerable attention in the applications of machine learning, image/signal processing, statistics, etc. With faster convergence rate, there have been tremendous studies on developing stochastic variance reduced algorithms to solve these non-convex optimization problems. However, as a crucial hyper-parameter for stochastic variance reduced algorithms, that how to select an appropriate step size is less researched in solving non-convex optimization problems. To address this gap, we propose a new class of stochastic variance reduced algorithms based on hyper-gradient, which has the ability to automatically obtain the online step size. Specifically, we focus on the variance-reduced stochastic optimization algorithms, the stochastic variance reduced gradient (SVRG) algorithm, which computes a full gradient periodically. We analyze theoretically the convergence of the proposed algorithm for non-convex optimization problems. Moreover, we show that the proposed algorithm enjoys the same complexities as state-of-the-art algorithms for solving non-convex problems in terms of finding an approximate stationary point. Thorough numerical results on empirical risk minimization with non-convex loss functions validate the efficacy of our method. [ABSTRACT FROM AUTHOR]

Robust face clustering enjoys a wide range of applications for gate passes, surveillance systems and security analysis in embedded sensors. Nevertheless, existing algorithms have limitations in finding accurate clusters when data contain noise (e.g., occluded face clustering and recognition). It is known that in subspace clustering, the ℓ1- and ℓ2-norm regularizers can improve subspace preservation and connectivity, respectively, and the elastic net regularizer (i.e., the mixture of the ℓ1- and ℓ2-norms) provides a balance between the two properties. However, existing deterministic methods have high per iteration computational complexities, making them inapplicable to large-scale problems. To address this issue, this paper proposes the first accelerated stochastic variance reduction gradient (RASVRG) algorithm for robust subspace clustering. We also introduce a new momentum acceleration technique for the RASVRG algorithm. As a result of the involvement of this momentum, the RASVRG algorithm achieves both the best oracle complexity and the fastest convergence rate, and it reaches higher efficiency in practice for both strongly convex and not strongly convex models. Various experimental results show that the RASVRG algorithm outperformed existing state-of-the-art methods with elastic net and ℓ1-norm regularizers in terms of accuracy in most cases. As demonstrated on real-world face datasets with different manually added levels of pixel corruption and occlusion situations, the RASVRG algorithm achieved much better performance in terms of accuracy and robustness.