Search

The paper is devoted to developing subdifferential theory for set-valued mappings taking values in ordered infinite-dimensional spaces. This study is motivated by applications to problems of vector and set optimization with various constraints in infinite dimensions. The main results establish new sum and chain rules for major subdifferential constructions associated with ordered set-valued mappings under appropriate qualification and sequentially normal compactness conditions.

This paper addresses, for the first time in the literature, optimal control problems for dynamic systems governed by a novel class of sweeping processes with time delay. We establish well-posedness of such processes, in the sense of the existence and uniqueness of feasible trajectories corresponding to feasible controls under fairly unrestrictive assumptions. Then we construct a well-posed family of discrete approximations and find efficient conditions under the discretized time-delayed sweeping process exhibits stability with respect to strong convergence of feasible and optimal solutions. This creates a bridge between optimization of continuous-time and discrete-time sweeping control systems and justifies the effective use of discrete approximations in deriving optimality conditions and numerical techniques to solve the original time-delayed sweeping control problems via discrete approximations.

In this paper, we first provide a simple variational proof of the existence of Nash equilibrium in Hilbert spaces by using optimality conditions in convex minimization and Schauder's fixed-point theorem. Then applications of convex analysis and generalized differentiation are given to the existence of Nash equilibrium and extended versions of von Neumann's minimax theorem in locally convex topological vector spaces. Our analysis in this part combines generalized differential tools of convex analysis with elements of fixed point theory revolving around Kakutani's fixed-point theorem and related issues.

This paper addresses a new class of generalized Bolza problems governed by nonconvex integro-differential inclusions with endpoint constraints on trajectories, where the integral terms are given in the general (with time-dependent integrands in the dynamics) Volterra form. We pursue here a threefold goal. First we construct well-posed approximations of continuous-time integro-differential systems by their discrete-time counterparts with showing that any feasible solution to the original system can be strongly approximated in the $W^{1,2}$-norm topology by piecewise-linear extensions of feasible discrete trajectories. This allows us to verify in turn the strong convergence of discrete optimal solutions to a prescribed local minimizer for the original problem. Facing intrinsic nonsmoothness of original integro-differential problem and its discrete approximations, we employ appropriate tools of generalized differentiation in variational analysis to derive necessary optimality conditions for discrete-time problems (which is our second goal) and finally accomplish our third goal to obtain necessary conditions for the original continuous-time problems by passing to the limit from discrete approximations. In this way we establish, in particular, a novel necessary optimality condition of the Volterra type, which is the crucial result for dynamic optimization of integro-differential inclusions., Comment: 28 pages

This paper introduces and develops novel coderivative-based Newton methods with Wolfe linesearch conditions to solve various classes of problems in nonsmooth optimization. We first propose a generalized regularized Newton method with Wolfe linesearch (GRNM-W) for unconstrained $C^{1,1}$ minimization problems (which are second-order nonsmooth) and establish global as well as local superlinear convergence of their iterates. To deal with convex composite minimization problems (which are first-order nonsmooth and can be constrained), we combine the proposed GRNM-W with two algorithmic frameworks: the forward-backward envelope and the augmented Lagrangian method resulting in the two new algorithms called CNFB and CNAL, respectively. Finally, we present numerical results to solve Lasso and support vector machine problems appearing in, e.g., machine learning and statistics, which demonstrate the efficiency of the proposed algorithms.

This paper pursues a twofold goal. First, we introduce and study in detail a new notion of variational analysis called generalized metric subregularity, which is a far-going extension of the conventional metric subregularity conditions. Our primary focus is on examining this concept concerning first-order and second-order stationary points. We develop an extended convergence framework that enables us to derive superlinear and quadratic convergence under the generalized metric subregularity condition, broadening the widely used KL convergence analysis framework. We present verifiable sufficient conditions to ensure the proposed generalized metric subregularity condition and provide examples demonstrating that the derived convergence rates are sharp. Second, we design a new high-order regularized Newton method with momentum steps, and apply the generalized metric subregularity to establish its superlinear convergence. Quadratic convergence is obtained under additional assumptions. Specifically, when applying the proposed method to solve the (nonconvex) over-parameterized compressed sensing model, we achieve global convergence with a quadratic local convergence rate towards a global minimizer under a strict complementarity condition., Comment: 33 pages (including appendix) and 2 figures

The paper addresses the study and applications of a broad class of extended-real-valued functions, known as optimal value or marginal functions, which are frequently appeared in variational analysis, parametric optimization, and a variety of applications. Functions of this type are intrinsically nonsmooth and require the usage of tools of generalized differentiation. The main results of this paper provide novel evaluations and exact calculations of regular/Fr\'echet subgradients and their singular counterparts for general classes of marginal functions via their given data. The obtained results are applied to establishing new calculus rules for such subgradients and necessary optimality conditions in bilevel programming

This paper primarily focuses on the practical applications of optimal control theory for perturbed sweeping processes within the realm of robotics dynamics. By describing these models as controlled sweeping processes with pointwise control and state constraints and by employing necessary optimality conditions for such systems, we formulate optimal control problems suitable to these models and develop numerical algorithms for their solving. Subsequently, we use the Python Dynamic Optimization library GEKKO to simulate solutions to the posed robotics problems in the case of any fixed number of robots under different initial conditions.

The paper is devoted to establishing relationships between global and local monotonicity, as well as their maximality versions, for single-valued and set-valued mappings between finite-dimensional and infinite-dimensional spaces. We first show that for single-valued operators with convex domains in locally convex topological spaces, their continuity ensures that their global monotonicity agrees with the local one around any point of the graph. This also holds for set-valued mappings defined on the real line under a certain connectedness condition. The situation is different for set-valued operators in multidimensional spaces as demonstrated by an example of locally monotone operator on the plane that is not globally monotone. Finally, we invoke coderivative criteria from variational analysis to characterize both global and local maximal monotonicity of set-valued operators in Hilbert spaces to verify the equivalence between these monotonicity properties under the closed-graph and global hypomonotonicity assumptions.

This paper proposes and develops new Newton-type methods to solve structured nonconvex and nonsmooth optimization problems with justifying their fast local and global convergence by means of advanced tools of variational analysis and generalized differentiation. The objective functions belong to a broad class of prox-regular functions with specification to constrained optimization of nonconvex structured sums. We also develop a novel line search method, which is an extension of the proximal gradient algorithm while allowing us to globalize the proposed coderivative-based Newton methods by incorporating the machinery of forward-backward envelopes. Further applications and numerical experiments are conducted for the $\ell_0$-$\ell_2$ regularized least-square model appearing in statistics and machine learning.

The paper investigates the fundamental convergence properties of Sharpness-Aware Minimization (SAM), a recently proposed gradient-based optimization method [Foret et al., 2021] that significantly improves the generalization of deep neural networks. The convergence properties, including the stationarity of accumulation points, the convergence of the sequence of gradients to the origin, the sequence of function values to the optimal value, and the sequence of iterates to the optimal solution, are established for the method. The universality of the provided convergence analysis, based on inexact gradient descent frameworks Khanh et al. [2023b], allows its extensions to efficient normalized versions of SAM such as F-SAM [Li et al., 2024], VaSSO [Li and Giannakis, 2023], RSAM [Liu et al., 2022], and to the unnormalized versions of SAM such as USAM [Andriushchenko and Flammarion, 2022]. Numerical experiments are conducted on classification tasks using deep learning models to confirm the practical aspects of our analysis., Comment: 34 pages

This paper addresses the study of novel constructions of variational analysis and generalized differentiation that are appropriate for characterizing robust stability properties of constrained set-valued mappings/multifunctions between Banach spaces important in optimization theory and its applications. Our tools of generalized differentiation revolves around the newly introduced concept of $\varepsilon$-regular normal cone to sets and associated coderivative notions for set-valued mappings. Based on these constructions, we establish several characterizations of the central stability notion known as the relative Lipschitz-like property of set-valued mappings in infinite dimensions. Applying a new version of the constrained extremal principle of variational analysis, we develop comprehensive sum and chain rules for our major constructions of conic contingent coderivatives for multifunctions between appropriate classes of Banach spaces., Comment: 26 pages. arXiv admin note: text overlap with arXiv:2212.02727

The paper is devoted to deriving novel second-order necessary and sufficient optimality conditions for local minimizers in rather general classes of nonsmooth unconstrained and constrained optimization problems in finite-dimensional spaces. The established conditions are expressed in terms of second-order subdifferentials of lower semicontinuous functions and mainly concern prox-regular objectives that cover a large territory in nonsmooth optimization and its applications. Our tools are based on the machinery of variational analysis and second-order generalized differentiation. The obtained general results are applied to problems of nonlinear programming, where the derived second-order optimality conditions are new even for problems with twice continuously differential data, being expressed there in terms of the classical Hessian matrices.

This paper addresses novel applications to practical modeling of the newly developed theory of necessary optimality conditions in controlled sweeping/Moreau processes with free time and pointwise control and state constraints. Problems of this type appear, in particular, in dynamical models dealing with unmanned surface vehicles (USVs) and nanoparticles. We formulate optimal control problems for a general class of such dynamical systems and show that the developed necessary optimality conditions for constrained free-time controlled sweeping processes lead us to designing efficient procedures to solve practical models of this class. Moreover, the paper contains numerical calculations of optimal solutions to marine USVs and nanoparticle models in specific situations. Overall, this study contributes to the advancement of optimal control theory for constrained sweeping processes and its practical applications in the fields of marine USVs and nanoparticle modeling.

The paper is devoted to the study of a new class of optimal control problems governed by discontinuous constrained differential inclusions of the sweeping type with involving the duration of the dynamic process into optimization. We develop a novel version of the method of discrete approximations of its own qualitative and numerical values with establishing its well-posedness and strong convergence to optimal solutions of the controlled sweeping process. Using advanced tools of first-order and second-order variational analysis and generalized differentiation allows us to derive new necessary conditions for optimal solutions of the discrete-time problems and then, by passing to the limit in the discretization procedure, for designated local minimizers in the original problem of sweeping optimal control. The obtained results are illustrated by a numerical example., Comment: 33 pages, 5 figures

This paper is devoted to a systematic study and characterizations of the fundamental notions of variational and strong variational convexity for lower semicontinuous functions. While these notions have been quite recently introduced by Rockafellar, the importance of them has been already recognized and documented in finite-dimensional variational analysis and optimization. Here we address general infinite-dimensional settings and derive comprehensive characterizations of both variational and strong variational convexity notions by developing novel techniques, which are essentially different from finite-dimensional counterparts. As a consequence of the obtained characterizations, we establish new quantitative and qualitative relationships between strong variational convexity and tilt stability of local minimizers in appropriate frameworks of Banach spaces.

The paper is devoted to a systematic study and characterizations of notions of local maximal monotonicity and their strong counterparts for set-valued operators that appear in variational analysis, optimization, and their applications. We obtain novel resolvent characterizations of these notions together with efficient conditions for their preservation under summation in broad infinite-dimensional settings. Further characterizations of these notions are derived by using generalized differentiation of variational analysis in the framework of Hilbert spaces.

This paper focuses on investigating generalized relative interior notions for sets in locally convex topological vector spaces with particular attentions to graphs of set-valued mappings and epigraphs of extended-real-valued functions. We introduce, study, and utilize a novel notion of quasi-near convexity of sets that is an infinite-dimensional extension of the widely acknowledged notion of near convexity. Quasi-near convexity is associated with the quasi-relative interior of sets, which is investigated in the paper together with other generalized relative interior notions for sets, not necessarily convex. In this way, we obtain new results on generalized relative interiors for graphs of set-valued mappings in convexity and generalized convexity settings.

This paper proposes and develops inexact proximal methods for finding stationary points of the sum of a smooth function and a nonsmooth weakly convex one, where an error is present in the calculation of the proximal mapping of the nonsmooth term. A general framework for finding zeros of a continuous mapping is derived from our previous paper on this subject to establish convergence properties of the inexact proximal point method when the smooth term is vanished and of the inexact proximal gradient method when the smooth term satisfies a descent condition. The inexact proximal point method achieves global convergence with constructive convergence rates when the Moreau envelope of the objective function satisfies the Kurdyka-Lojasiewicz (KL) property. Meanwhile, when the smooth term is twice continuously differentiable with a Lipschitz continuous gradient and a differentiable approximation of the objective function satisfies the KL property, the inexact proximal gradient method achieves the global convergence of iterates with constructive convergence rates., Comment: 26 pages, 3 tables

The paper proposes and develops a novel inexact gradient method (IGD) for minimizing C1-smooth functions with Lipschitzian gradients, i.e., for problems of C1,1 optimization. We show that the sequence of gradients generated by IGD converges to zero. The convergence of iterates to stationary points is guaranteed under the Kurdyka- Lojasiewicz (KL) property of the objective function with convergence rates depending on the KL exponent. The newly developed IGD is applied to designing two novel gradient-based methods of nonsmooth convex optimization such as the inexact proximal point methods (GIPPM) and the inexact augmented Lagrangian method (GIALM) for convex programs with linear equality constraints. These two methods inherit global convergence properties from IGD and are confirmed by numerical experiments to have practical advantages over some well-known algorithms of nonsmooth convex optimization., Comment: 23 pages, 8 figures

This paper addresses the study of a new class of nonsmooth optimization problems, where the objective is represented as a difference of two generally nonconvex functions. We propose and develop a novel Newton-type algorithm to solving such problems, which is based on the coderivative generated second-order subdifferential (generalized Hessian) and employs advanced tools of variational analysis. Well-posedness properties of the proposed algorithm are derived under fairly general requirements, while constructive convergence rates are established by using additional assumptions including the Kurdyka--{\L}ojasiewicz condition. We provide applications of the main algorithm to solving a general class of nonsmooth nonconvex problems of structured optimization that encompasses, in particular, optimization problems with explicit constraints. Finally, applications and numerical experiments are given for solving practical problems that arise in biochemical models, constrained quadratic programming, etc., where advantages of our algorithms are demonstrated in comparison with some known techniques and results., Comment: 38 pages

This paper addresses the study and characterizations of variational convexity of extended-real-valued functions on Banach spaces. This notion has been recently introduced by Rockafellar, and its importance has been already realized and applied to continuous optimization problems in finite-dimensional spaces. Variational convexity in infinite-dimensional spaces, which is studied here for the first time, is significantly more involved and requires the usage of powerful tools of geometric functional analysis together with variational analysis and generalized differentiation in Banach spaces.

The paper concerns foundations of sensitivity and stability analysis in optimization and related areas, being primarily addressed constrained systems. We consider general models, which are described by multifunctions between Banach spaces and concentrate on characterizing their well-posedness properties that revolve around Lipschitz stability and metric regularity relative to sets. Invoking tools of variational analysis and generalized differentiation, we introduce new robust notions of relative contingent coderivatives. The novel machinery of variational analysis leads us to establishing complete characterizations of the relative well-posedness properties and developing basic rules of variational calculus interrelated with the obtained characterizations of well-posedness. Most of the our results valid in general infinite-dimensional settings are also new in finite dimensions., Comment: 28 pages

The paper addresses the study of a class of evolutionary quasi-variational inequalities of the parabolic type arising in the formation and growth models of granular and cohensionless materials. Such models and their mathematical descriptions are highly challenging and require powerful tools of their analysis and implementation. We formulate a space-time continuous optimal control problem for a basic model of this type, develop several regularization and approximation procedures, and establish the existence of optimal solutions for the time-continuous and space-discrete problem. Viewing a version of this problem as a controlled quasi-variational sweeping process leads us to deriving necessary optimality conditions for the fully discrete problem by using the advanced machinery of variational analysis and generalized differentiation.

The paper is devoted to the study, characterizations, and applications of variational convexity of functions, the property that has been recently introduced by Rockafellar together with its strong counterpart. First we show that these variational properties of an extended-real-valued function are equivalent to, respectively, the conventional (local) convexity and strong convexity of its Moreau envelope. Then we derive new characterizations of both variational convexity and variational strong convexity of general functions via their second-order subdifferentials (generalized Hessians), which are coderivatives of subgradient mappings. We also study relationships of these notions with local minimizers and tilt-stable local minimizers. The obtained results are used for characterizing related notions of variational and strong variational sufficiency in composite optimization with applications to nonlinear programming.

This paper proposes and develops new linesearch methods with inexact gradient information for finding stationary points of nonconvex continuously differentiable functions on finite-dimensional spaces. Some abstract convergence results for a broad class of linesearch methods are stablished. A general scheme for inexact reduced gradient (IRG) methods is proposed, where the errors in the gradient approximation automatically adapt with the magnitudes of the exact gradients. The sequences of iterations are shown to obtain stationary accumulation points when different stepsize selections are employed. Convergence results with constructive convergence rates for the developed IRG methods are established under the Kurdyka- Lojasiewicz property. The obtained results for the IRG methods are confirmed by encouraging numerical experiments, which demonstrate advantages of automatically controlled errors in IRG methods over other frequently used error selections.

The paper studies coincidence points of parameterized set-valued mappings (multifunctions), which provide an extended framework to cover several important topics in variational analysis and optimization that include the existence of solutions of parameterized generalized equations, implicit function and fixed-point theorems, optimal value functions in parametric optimization, etc. Using the advanced machinery of variational analysis and generalized differentiation that furnishes complete characterizations of well-posedness properties of multifunctions, we establish a general theorem ensuring the existence of parameter-dependent coincidence point mappings with explicit error bounds for parameterized multifunctions between infinite-dimensional spaces. The obtained major result yields a new implicit function theorem and allows us to derive efficient conditions for semicontinuity and continuity of optimal value functions associated with parametric minimization problems subject to constraints governed by parameterized generalized equations.

In this paper we revisit a theorem by Rockafellar on representing the relative interior of the graph of a convex set-valued mapping in terms of the relative interior of its domain and function values. Then we apply this theorem to provide a simple way to prove many calculus rules of generalized differentiation of set-valued mappings and nonsmooth functions in finite dimensions. These results improve upon those in [14] by replacing the relative interior qualifications on graphs with qualifications on domains and/or ranges.