Search

The generalized likelihood ratio (GLR) method is a recently introduced gradient estimation method for handling discontinuities in a wide range of sample performances. We put the GLR methods from previous work into a single framework, simplify regularity conditions to justify the unbiasedness of GLR, and relax some of those conditions that are difficult to verify in practice. Moreover, we combine GLR with conditional Monte Carlo methods and randomized quasi-Monte Carlo methods to reduce the variance. Numerical experiments show that variance reduction could be significant in various applications.

Machine translation is one of the most classic application technologies in artificial intelligence and natural language processing. Neural machine translation models generally adopt an encoder–decoder architecture for modeling the entire translation process. However, without considering target context (e.g., decoding state) to guide the encoding, encoded source representations struggle to put great emphasis on important information for predicting some target word, yielding the weakness in generating more discriminative attentive representations across different decoding steps. Towards tackling this issue, we propose a novel encoder–refiner–decoder framework, which dynamically refines the source representations based on the generated target-side information at each decoding step. Since the refining operations are time-consuming, we propose a policy network to decide when to refine at specific decoding steps. We solve such a problem using the Gumbel-Softmax reparameterization, which makes our network differentiable and trainable through standard stochastic gradient methods. Experimental results on both Chinese–English and English–German translation tasks show that the proposed approach significantly and consistently improves translation performance over the standard encoder–decoder framework. Furthermore, when refining strategy is applied, experimental results still show a reasonable improvement over the baseline with much decrease in decoding speed. [ABSTRACT FROM AUTHOR]

We investigate a new approach to compute the gradients of artificial neural networks (ANNs), based on the so-called push-out likelihood ratio method. Unlike the widely used backpropagation (BP) method that requires continuity of the loss function and the activation function, our approach bypasses this requirement by injecting artificial noises into the signals passed along the neurons. We show how this approach has a similar computational complexity as BP, and moreover is more advantageous in terms of removing the backward recursion and eliciting transparent formulas. We also formalize the connection between BP, a pivotal technique for training ANNs, and infinitesimal perturbation analysis, a classic path-wise derivative estimation approach, so that both our new proposed methods and BP can be better understood in the context of stochastic gradient estimation. Our approach allows efficient training for ANNs with more flexibility on the loss and activation functions, and shows empirical improvements on the robustness of ANNs under adversarial attacks and corruptions of natural noises. Summary of Contribution: Stochastic gradient estimation has been studied actively in simulation for decades and becomes more important in the era of machine learning and artificial intelligence. The stochastic gradient descent is a standard technique for training the artificial neural networks (ANNs), a pivotal problem in deep learning. The most popular stochastic gradient estimation technique is the backpropagation method. We find that the backpropagation method lies in the family of infinitesimal perturbation analysis, a path-wise gradient estimation technique in simulation. Moreover, we develop a new likelihood ratio-based method, another popular family of gradient estimation technique in simulation, for training more general ANNs, and demonstrate that the new training method can improve the robustness of the ANN. [ABSTRACT FROM AUTHOR]

In this article, we aim to solve Bayesian Risk Optimization (BRO), which is a recently proposed framework that formulates simulation optimization under input uncertainty. In order to efficiently solve the BRO problem, we derive nested stochastic gradient estimators and propose corresponding stochastic approximation algorithms. We show that our gradient estimators are asymptotically unbiased and consistent, and that the algorithms converge asymptotically. We demonstrate the empirical performance of the algorithms on a two-sided market model. Our estimators are of independent interest in extending the literature of stochastic gradient estimation to the case of nested risk measures. [ABSTRACT FROM AUTHOR]

This paper proposes an online constrained extremum-seeking approach for an unknown convex function with unknown constraints within a class of uncertain dynamical systems with an available output disturbed by a stochastic noise. It is assumed that the objective function along with the constraints are available for measurement. The main contribution of the paper is the formulation of the problem using the penalty method and the development of an extremum seeking algorithm based on a modified synchronous detection method for computing a stochastic gradient descent procedure. In order to reject the undesirable uncertainties and perturbations of the dynamic plant from the beginning of the process, we employ the standard deterministic Integral Sliding Mode Control transforming the initial dynamic plant to static one. Then, we apply the gradient decedent technique. We consider time varying parameters of the suggested procedure for compensating the unknown dynamics. To validate the exposition, we perform a numerical example simulation. [ABSTRACT FROM AUTHOR]

Traditional regression assumes that the only data available are measurements of the value of the dependent variable for each combination of values for the independent variable. However, in many settings in stochastic (Monte Carlo) simulation, directly estimated derivative information is also available via techniques such as perturbation analysis or the likelihood ratio method. In this paper, we investigate potential modeling improvements that can be achieved by exploiting this additional gradient information in the regression setting. Using least squares and maximum likelihood estimation, we propose various direct gradient augmented regression (DiGAR) models that incorporate direct gradient estimators, starting with a one-dimensional independent variable and then extending to multidimensional input. For some special settings, we are able to characterize the variance of the estimated parameters in DiGAR and compare them analytically with the standard regression model. For a more typical stochastic simulation setting, we investigate the potential effectiveness of the augmented model by comparing it with standard regression in fitting a functional relationship for a simple queueing model, including both one-dimensional and four-dimensional examples. The preliminary empirical results are quite encouraging, as they indicate how DiGAR can capture trends that the standard model would miss. Even in queueing examples where there is a high correlation between the output and the gradient estimators, the basic DiGAR model that does not explicitly account for these correlations performs significantly better than the standard regression model. [ABSTRACT FROM AUTHOR]