Search

We propose a novel actor–critic algorithm with guaranteed convergence to an optimal policy for a discounted reward Markov decision process. The actor incorporates a descent direction that is motivated by the solution of a certain non-linear optimization problem. We also discuss an extension to incorporate function approximation and demonstrate the practicality of our algorithms on a network routing application. [ABSTRACT FROM AUTHOR]

Abstract: The solution of the augmented Lagrangian related system is a key ingredient of many iterative algorithms for the solution of saddle-point problems in constrained optimization with quasi-Newton methods. However, such problems are ill-conditioned when the penalty parameter tends to zero, whereas the error vanishes as . We present a new fast method based on a splitting penalty scheme to solve such problems with a judicious prediction–correction method. We prove that, due to the adapted right-hand side, the solution of the correction step only requires the approximation of operators independent of , when is taken sufficiently small. Hence, the proposed method is as cheaper as tends to zero. We apply the two-step scheme to efficiently solve the saddle-point problem with a penalty method. Indeed, that fully justifies the interest of the vector penalty-projection methods recently proposed by Angot et al. (2008) to solve the unsteady incompressible Navier–Stokes equations, for which we give the stability result and some quasi-optimal error estimates. Moreover, the numerical experiments confirm both the theoretical analysis and the efficiency of the proposed method which produces a fast splitting solution to augmented Lagrangian or penalty problems, possibly used as a suitable preconditioner to the fully coupled system. [Copyright &y& Elsevier]

Abstract: In this paper, we give new results on the Alienor method of dimension reduction. This technique is used to solve multidimensional global optimization problems of type min x∈X f(x) where f is a non convex Lipschitz function and X a compact set of defined by Lipschitz constraints. The idea is to construct an α-dense curve h in the feasible set X. The global minimum of f on X is then approximated by the global minimum of f on the curve h. That is, our problem has become a one-dimensional problem which can be solved by the Piyavskii–Shubert method. Examples of these curves and numerical implementations on several test functions are given. [Copyright &y& Elsevier]