Search

In this paper, we establish characterizations of variational $s$-convexity and tilt stability for prox-regular functions in the absence of subdifferential continuity via quadratic bundles, a kind of primal-dual generalized second-order derivatives recently introduced by Rockafellar. Deriving such characterizations in the effective pointbased form requires a certain revision of quadratic bundles investigated below. Our device is based on the notion of generalized twice differentiability and its novel characterization via classical twice differentiability of the associated Moreau envelopes combined with various limiting procedures for functions and sets.

In this paper, we investigate the concepts of generalized twice differentiability and quadratic bundles of nonsmooth functions that have been very recently proposed by Rockafellar in the framework of second-order variational analysis. These constructions, in contrast to second-order subdifferentials, are defined in primal spaces. We develop new techniques to study generalized twice differentiability for a broad class of prox-regular functions, establish their novel characterizations. Subsequently, quadratic bundles of prox-regular functions are shown to be nonempty, which provides the ground of potential applications in variational analysis and optimization.

In this paper, we introduce an inexact approach to the Boosted Difference of Convex Functions Algorithm (BDCA) for solving nonconvex and nondifferentiable problems involving the difference of two convex functions (DC functions). Specifically, when the first DC component is differentiable and the second may be nondifferentiable, BDCA utilizes the solution from the subproblem of the DC Algorithm (DCA) to define a descent direction for the objective function. A monotone linesearch is then performed to find a new point that improves the objective function relative to the subproblem solution. This approach enhances the performance of DCA. However, if the first DC component is nondifferentiable, the BDCA direction may become an ascent direction, rendering the monotone linesearch ineffective. To address this, we propose an Inexact nonmonotone Boosted Difference of Convex Algorithm (InmBDCA). This algorithm incorporates two main features of inexactness: First, the subproblem therein is solved approximately allowing us for a controlled relative error tolerance in defining the linesearch direction. Second, an inexact nonmonotone linesearch scheme is used to determine the step size for the next iteration. Under suitable assumptions, we demonstrate that InmBDCA is well-defined, with any accumulation point of the sequence generated by InmBDCA being a critical point of the problem. We also provide iteration-complexity bounds for the algorithm. Numerical experiments show that InmBDCA outperforms both the nonsmooth BDCA (nmBDCA) and the monotone version of DCA in practical scenarios., Comment: 24 pages, 2 figures

The paper is devoted to developing subdifferential theory for set-valued mappings taking values in ordered infinite-dimensional spaces. This study is motivated by applications to problems of vector and set optimization with various constraints in infinite dimensions. The main results establish new sum and chain rules for major subdifferential constructions associated with ordered set-valued mappings under appropriate qualification and sequentially normal compactness conditions.

This paper addresses a new class of generalized Bolza problems governed by nonconvex integro-differential inclusions with endpoint constraints on trajectories, where the integral terms are given in the general (with time-dependent integrands in the dynamics) Volterra form. We pursue here a threefold goal. First we construct well-posed approximations of continuous-time integro-differential systems by their discrete-time counterparts with showing that any feasible solution to the original system can be strongly approximated in the $W^{1,2}$-norm topology by piecewise-linear extensions of feasible discrete trajectories. This allows us to verify in turn the strong convergence of discrete optimal solutions to a prescribed local minimizer for the original problem. Facing intrinsic nonsmoothness of original integro-differential problem and its discrete approximations, we employ appropriate tools of generalized differentiation in variational analysis to derive necessary optimality conditions for discrete-time problems (which is our second goal) and finally accomplish our third goal to obtain necessary conditions for the original continuous-time problems by passing to the limit from discrete approximations. In this way we establish, in particular, a novel necessary optimality condition of the Volterra type, which is the crucial result for dynamic optimization of integro-differential inclusions., Comment: 28 pages

This paper introduces and develops novel coderivative-based Newton methods with Wolfe linesearch conditions to solve various classes of problems in nonsmooth optimization. We first propose a generalized regularized Newton method with Wolfe linesearch (GRNM-W) for unconstrained $C^{1,1}$ minimization problems (which are second-order nonsmooth) and establish global as well as local superlinear convergence of their iterates. To deal with convex composite minimization problems (which are first-order nonsmooth and can be constrained), we combine the proposed GRNM-W with two algorithmic frameworks: the forward-backward envelope and the augmented Lagrangian method resulting in the two new algorithms called CNFB and CNAL, respectively. Finally, we present numerical results to solve Lasso and support vector machine problems appearing in, e.g., machine learning and statistics, which demonstrate the efficiency of the proposed algorithms.

The paper addresses the study and applications of a broad class of extended-real-valued functions, known as optimal value or marginal functions, which are frequently appeared in variational analysis, parametric optimization, and a variety of applications. Functions of this type are intrinsically nonsmooth and require the usage of tools of generalized differentiation. The main results of this paper provide novel evaluations and exact calculations of regular/Fr\'echet subgradients and their singular counterparts for general classes of marginal functions via their given data. The obtained results are applied to establishing new calculus rules for such subgradients and necessary optimality conditions in bilevel programming

This paper primarily focuses on the practical applications of optimal control theory for perturbed sweeping processes within the realm of robotics dynamics. By describing these models as controlled sweeping processes with pointwise control and state constraints and by employing necessary optimality conditions for such systems, we formulate optimal control problems suitable to these models and develop numerical algorithms for their solving. Subsequently, we use the Python Dynamic Optimization library GEKKO to simulate solutions to the posed robotics problems in the case of any fixed number of robots under different initial conditions.

The paper is devoted to establishing relationships between global and local monotonicity, as well as their maximality versions, for single-valued and set-valued mappings between finite-dimensional and infinite-dimensional spaces. We first show that for single-valued operators with convex domains in locally convex topological spaces, their continuity ensures that their global monotonicity agrees with the local one around any point of the graph. This also holds for set-valued mappings defined on the real line under a certain connectedness condition. The situation is different for set-valued operators in multidimensional spaces as demonstrated by an example of locally monotone operator on the plane that is not globally monotone. Finally, we invoke coderivative criteria from variational analysis to characterize both global and local maximal monotonicity of set-valued operators in Hilbert spaces to verify the equivalence between these monotonicity properties under the closed-graph and global hypomonotonicity assumptions.

This paper proposes and develops new Newton-type methods to solve structured nonconvex and nonsmooth optimization problems with justifying their fast local and global convergence by means of advanced tools of variational analysis and generalized differentiation. The objective functions belong to a broad class of prox-regular functions with specification to constrained optimization of nonconvex structured sums. We also develop a novel line search method, which is an extension of the proximal gradient algorithm while allowing us to globalize the proposed coderivative-based Newton methods by incorporating the machinery of forward-backward envelopes. Further applications and numerical experiments are conducted for the $\ell_0$-$\ell_2$ regularized least-square model appearing in statistics and machine learning.

The paper investigates the fundamental convergence properties of Sharpness-Aware Minimization (SAM), a recently proposed gradient-based optimization method [Foret et al., 2021] that significantly improves the generalization of deep neural networks. The convergence properties, including the stationarity of accumulation points, the convergence of the sequence of gradients to the origin, the sequence of function values to the optimal value, and the sequence of iterates to the optimal solution, are established for the method. The universality of the provided convergence analysis, based on inexact gradient descent frameworks Khanh et al. [2023b], allows its extensions to efficient normalized versions of SAM such as F-SAM [Li et al., 2024], VaSSO [Li and Giannakis, 2023], RSAM [Liu et al., 2022], and to the unnormalized versions of SAM such as USAM [Andriushchenko and Flammarion, 2022]. Numerical experiments are conducted on classification tasks using deep learning models to confirm the practical aspects of our analysis., Comment: 34 pages

This paper addresses the study of novel constructions of variational analysis and generalized differentiation that are appropriate for characterizing robust stability properties of constrained set-valued mappings/multifunctions between Banach spaces important in optimization theory and its applications. Our tools of generalized differentiation revolves around the newly introduced concept of $\varepsilon$-regular normal cone to sets and associated coderivative notions for set-valued mappings. Based on these constructions, we establish several characterizations of the central stability notion known as the relative Lipschitz-like property of set-valued mappings in infinite dimensions. Applying a new version of the constrained extremal principle of variational analysis, we develop comprehensive sum and chain rules for our major constructions of conic contingent coderivatives for multifunctions between appropriate classes of Banach spaces., Comment: 26 pages. arXiv admin note: text overlap with arXiv:2212.02727

The paper is devoted to deriving novel second-order necessary and sufficient optimality conditions for local minimizers in rather general classes of nonsmooth unconstrained and constrained optimization problems in finite-dimensional spaces. The established conditions are expressed in terms of second-order subdifferentials of lower semicontinuous functions and mainly concern prox-regular objectives that cover a large territory in nonsmooth optimization and its applications. Our tools are based on the machinery of variational analysis and second-order generalized differentiation. The obtained general results are applied to problems of nonlinear programming, where the derived second-order optimality conditions are new even for problems with twice continuously differential data, being expressed there in terms of the classical Hessian matrices.

This paper addresses novel applications to practical modeling of the newly developed theory of necessary optimality conditions in controlled sweeping/Moreau processes with free time and pointwise control and state constraints. Problems of this type appear, in particular, in dynamical models dealing with unmanned surface vehicles (USVs) and nanoparticles. We formulate optimal control problems for a general class of such dynamical systems and show that the developed necessary optimality conditions for constrained free-time controlled sweeping processes lead us to designing efficient procedures to solve practical models of this class. Moreover, the paper contains numerical calculations of optimal solutions to marine USVs and nanoparticle models in specific situations. Overall, this study contributes to the advancement of optimal control theory for constrained sweeping processes and its practical applications in the fields of marine USVs and nanoparticle modeling.

The paper is devoted to the study of a new class of optimal control problems governed by discontinuous constrained differential inclusions of the sweeping type with involving the duration of the dynamic process into optimization. We develop a novel version of the method of discrete approximations of its own qualitative and numerical values with establishing its well-posedness and strong convergence to optimal solutions of the controlled sweeping process. Using advanced tools of first-order and second-order variational analysis and generalized differentiation allows us to derive new necessary conditions for optimal solutions of the discrete-time problems and then, by passing to the limit in the discretization procedure, for designated local minimizers in the original problem of sweeping optimal control. The obtained results are illustrated by a numerical example., Comment: 33 pages, 5 figures

This paper is devoted to a systematic study and characterizations of the fundamental notions of variational and strong variational convexity for lower semicontinuous functions. While these notions have been quite recently introduced by Rockafellar, the importance of them has been already recognized and documented in finite-dimensional variational analysis and optimization. Here we address general infinite-dimensional settings and derive comprehensive characterizations of both variational and strong variational convexity notions by developing novel techniques, which are essentially different from finite-dimensional counterparts. As a consequence of the obtained characterizations, we establish new quantitative and qualitative relationships between strong variational convexity and tilt stability of local minimizers in appropriate frameworks of Banach spaces.

The paper is devoted to a systematic study and characterizations of notions of local maximal monotonicity and their strong counterparts for set-valued operators that appear in variational analysis, optimization, and their applications. We obtain novel resolvent characterizations of these notions together with efficient conditions for their preservation under summation in broad infinite-dimensional settings. Further characterizations of these notions are derived by using generalized differentiation of variational analysis in the framework of Hilbert spaces.

The paper proposes and develops a novel inexact gradient method (IGD) for minimizing C1-smooth functions with Lipschitzian gradients, i.e., for problems of C1,1 optimization. We show that the sequence of gradients generated by IGD converges to zero. The convergence of iterates to stationary points is guaranteed under the Kurdyka- Lojasiewicz (KL) property of the objective function with convergence rates depending on the KL exponent. The newly developed IGD is applied to designing two novel gradient-based methods of nonsmooth convex optimization such as the inexact proximal point methods (GIPPM) and the inexact augmented Lagrangian method (GIALM) for convex programs with linear equality constraints. These two methods inherit global convergence properties from IGD and are confirmed by numerical experiments to have practical advantages over some well-known algorithms of nonsmooth convex optimization., Comment: 23 pages, 8 figures

This paper addresses the study of a new class of nonsmooth optimization problems, where the objective is represented as a difference of two generally nonconvex functions. We propose and develop a novel Newton-type algorithm to solving such problems, which is based on the coderivative generated second-order subdifferential (generalized Hessian) and employs advanced tools of variational analysis. Well-posedness properties of the proposed algorithm are derived under fairly general requirements, while constructive convergence rates are established by using additional assumptions including the Kurdyka--{\L}ojasiewicz condition. We provide applications of the main algorithm to solving a general class of nonsmooth nonconvex problems of structured optimization that encompasses, in particular, optimization problems with explicit constraints. Finally, applications and numerical experiments are given for solving practical problems that arise in biochemical models, constrained quadratic programming, etc., where advantages of our algorithms are demonstrated in comparison with some known techniques and results., Comment: 38 pages

This paper addresses the study and characterizations of variational convexity of extended-real-valued functions on Banach spaces. This notion has been recently introduced by Rockafellar, and its importance has been already realized and applied to continuous optimization problems in finite-dimensional spaces. Variational convexity in infinite-dimensional spaces, which is studied here for the first time, is significantly more involved and requires the usage of powerful tools of geometric functional analysis together with variational analysis and generalized differentiation in Banach spaces.

The paper concerns foundations of sensitivity and stability analysis in optimization and related areas, being primarily addressed constrained systems. We consider general models, which are described by multifunctions between Banach spaces and concentrate on characterizing their well-posedness properties that revolve around Lipschitz stability and metric regularity relative to sets. Invoking tools of variational analysis and generalized differentiation, we introduce new robust notions of relative contingent coderivatives. The novel machinery of variational analysis leads us to establishing complete characterizations of the relative well-posedness properties and developing basic rules of variational calculus interrelated with the obtained characterizations of well-posedness. Most of the our results valid in general infinite-dimensional settings are also new in finite dimensions., Comment: 28 pages

The paper addresses the study of a class of evolutionary quasi-variational inequalities of the parabolic type arising in the formation and growth models of granular and cohensionless materials. Such models and their mathematical descriptions are highly challenging and require powerful tools of their analysis and implementation. We formulate a space-time continuous optimal control problem for a basic model of this type, develop several regularization and approximation procedures, and establish the existence of optimal solutions for the time-continuous and space-discrete problem. Viewing a version of this problem as a controlled quasi-variational sweeping process leads us to deriving necessary optimality conditions for the fully discrete problem by using the advanced machinery of variational analysis and generalized differentiation.

The paper is devoted to the study, characterizations, and applications of variational convexity of functions, the property that has been recently introduced by Rockafellar together with its strong counterpart. First we show that these variational properties of an extended-real-valued function are equivalent to, respectively, the conventional (local) convexity and strong convexity of its Moreau envelope. Then we derive new characterizations of both variational convexity and variational strong convexity of general functions via their second-order subdifferentials (generalized Hessians), which are coderivatives of subgradient mappings. We also study relationships of these notions with local minimizers and tilt-stable local minimizers. The obtained results are used for characterizing related notions of variational and strong variational sufficiency in composite optimization with applications to nonlinear programming.

This paper proposes and develops new linesearch methods with inexact gradient information for finding stationary points of nonconvex continuously differentiable functions on finite-dimensional spaces. Some abstract convergence results for a broad class of linesearch methods are stablished. A general scheme for inexact reduced gradient (IRG) methods is proposed, where the errors in the gradient approximation automatically adapt with the magnitudes of the exact gradients. The sequences of iterations are shown to obtain stationary accumulation points when different stepsize selections are employed. Convergence results with constructive convergence rates for the developed IRG methods are established under the Kurdyka- Lojasiewicz property. The obtained results for the IRG methods are confirmed by encouraging numerical experiments, which demonstrate advantages of automatically controlled errors in IRG methods over other frequently used error selections.

The paper studies coincidence points of parameterized set-valued mappings (multifunctions), which provide an extended framework to cover several important topics in variational analysis and optimization that include the existence of solutions of parameterized generalized equations, implicit function and fixed-point theorems, optimal value functions in parametric optimization, etc. Using the advanced machinery of variational analysis and generalized differentiation that furnishes complete characterizations of well-posedness properties of multifunctions, we establish a general theorem ensuring the existence of parameter-dependent coincidence point mappings with explicit error bounds for parameterized multifunctions between infinite-dimensional spaces. The obtained major result yields a new implicit function theorem and allows us to derive efficient conditions for semicontinuity and continuity of optimal value functions associated with parametric minimization problems subject to constraints governed by parameterized generalized equations.

This paper conducts sensitivity analysis of random constraint and variational systems related to stochastic optimization and variational inequalities. We establish efficient conditions for well-posedness, in the sense of robust Lipschitzian stability and/or metric regularity, of such systems by employing and developing coderivative characterizations of well-posedness properties for random multifunctions and efficiently evaluating coderivatives of special classes of random integral set-valued mappings that naturally emerge in stochastic programming and stochastic variational inequalities., Comment: 31

This paper is mainly devoted to the study of controlled sweeping processes with polyhedral moving sets in Hilbert spaces. Based on a detailed analysis of truncated Hausdorff distances between moving polyhedra, we derive new existence and uniqueness theorems for sweeping trajectories corresponding to various classes of control functions acting in moving sets. Then we establish quantitative stability results, which provide efficient estimates on the sweeping trajectory dependence on controls and initial values. Our final topic, accomplished in finite-dimensional state spaces, is deriving new necessary optimality and suboptimality conditions for sweeping control systems with endpoint constrains by using constructive discrete approximations.