Search

Recently, several convergence rate results for Douglas-Rachford splitting and the alternating direction method of multipliers (ADMM) have been presented in the literature. In this paper, we show global linear convergence rate bounds for Douglas-Rachford splitting and ADMM under strong convexity and smoothness assumptions. We further show that the rate bounds are tight for the class of problems under consideration for all feasible algorithm parameters. For problems that satisfy the assumptions, we show how to select step-size and metric for the algorithm that optimize the derived convergence rate bounds. For problems with a similar structure that do not satisfy the assumptions, we present heuristic step-size and metric selection methods. [ABSTRACT FROM PUBLISHER]

Reducing address arithmetic operations by optimization of address offset assignment greatly improves the performance of digital signal processor (DSP) applications. However, minimizing address operations alone may not directly reduce code size and schedule length for DSPs with multiple functional units. Little research work has been conducted on loop optimization with address offset assignment problem for architectures with multiple functional units. In this paper, we combine loop scheduling, array interleaving, and address assignment to minimize the schedule length and the number of address operations for loops on DSP architectures with multiple functional units. Array interleaving is applied to optimize address assignment for arrays in loop scheduling process. An algorithm, Address Operation Reduction Rotation Scheduling (AORRS), is proposed. The algorithm minimizes both schedule length and the number of address operations. with to list scheduling, AORRS shows an average reduction of 38.4% in schedule length and an average reduction of 31.7% in the number of address operations. Compared with rotation scheduling, AORRS shows an average reduction of 15.9% in schedule length and 33.6% in the number of address operations. [ABSTRACT FROM AUTHOR]

In model predictive control (MPC), the control input is computed by solving a constrained finite-time optimal control (CFTOC) problem at each sample in the control loop. The main computational effort when solving the CFTOC problem using an active-set (AS) method is often spent on computing the search directions, which in MPC corresponds to solving unconstrained finite-time optimal control (UFTOC) problems. This is commonly performed using Riccati recursions or generic sparsity exploiting algorithms. In this paper, the focus is efficient search direction computations for AS type methods. The system of equations to be solved at each AS iteration is changed only by a low-rank modification of the previous one, and exploiting this structured change is important for the performance of AS-type solvers. In this paper, theory for how to exploit these low-rank changes by modifying the Riccati factorization between AS iterations in a structured way is presented. A numerical evaluation of the proposed algorithm shows that the computation time can be significantly reduced by modifying, instead of re-computing, the Riccati factorization. This speedup can be important for AS-type solvers used for linear, nonlinear, and hybrid MPC. [ABSTRACT FROM PUBLISHER]

Finding the optimal solution to the constrained l0 -norm minimization problems in the recovery of compressive sensed signals is an NP-hard problem and it usually requires intractable combinatorial searching operations for getting the global optimal solution, unless using other objective functions (e.g., the l1 norm or lp norm) for approximate solutions or using greedy search methods for locally optimal solutions (e.g., the orthogonal matching pursuit type algorithms). In this paper, a neurodynamic optimization method is proposed to solve the l0 -norm minimization problems for obtaining the global optimum using a recurrent neural network (RNN) model. For the RNN model, a group of modified Gaussian functions are constructed and their sum is taken as the objective function for approximating the l0 norm and for optimization. The constructed objective function sets up a convexity condition under which the neurodynamic system is guaranteed to obtain the globally convergent optimal solution. An adaptive adjustment scheme is developed for improving the performance of the optimization algorithm further. Extensive experiments are conducted to test the proposed approach in this paper and the output results validate the effectiveness of the new method. [ABSTRACT FROM PUBLISHER]

This paper proposes distributed algorithms to solve robust convex optimization (RCO) when the constraints are affected by nonlinear uncertainty. We adopt a scenario approach by randomly sampling the uncertainty set. To facilitate the computational task, instead of using a single centralized processor to obtain a “global solution” of the scenario problem (SP), we resort to multiple interconnected processors that are distributed among different nodes of a network to simultaneously solve the SP. Then, we propose a primal-dual subgradient algorithm and a random projection algorithm to distributedly solve the SP over undirected and directed graphs, respectively. Both algorithms are given in an explicit recursive form with simple iterations, which are especially suited for processors with limited computational capability. We show that, if the underlying graph is strongly connected, each node asymptotically computes a common optimal solution to the SP with a convergence rate $O(1/(\sum _{t=1}^k\zeta ^t))$ , where $\lbrace \zeta ^t\rbrace$ is a sequence of appropriately decreasing stepsizes. That is, the RCO is effectively solved in a distributed way. The relations with the existing literature on robust convex programs are thoroughly discussed and an example of robust system identification is included to validate the effectiveness of our distributed algorithms. [ABSTRACT FROM AUTHOR]

A reduction of a source distribution is a collection of smaller sized distributions that are collectively equivalent to the source distribution with respect to the property of decomposability. That is, an arbitrary language is decomposable with respect to the source distribution if and only if it is decomposable with respect to each smaller sized distribution (in the reduction). The notion of reduction of distributions has previously been proposed to improve the complexity of decomposability verification. In this paper, we address the problem of generating (optimal) reductions of distributions automatically. A (partial) solution to this problem is provided, which consists of an incremental algorithm for the production of candidate reductions and a reduction validation procedure. In the incremental production stage, backtracking is applied whenever a candidate reduction that cannot be validated is produced. A strengthened substitution-based proof technique is used for reduction validation, while a fixed template of candidate counter examples is used for reduction refutation; put together, they constitute our (partial) solution to the reduction verification problem. In addition, we show that a recursive approach for the generation of (small) reductions is easily supported. [ABSTRACT FROM AUTHOR]

This paper is concerned with a general class of distributed constrained optimization problems over a multiagent network, where the global objective function is represented by the sum of all local objective functions. Each agent in the network only knows its own local objective function, and is restricted to a global nonempty closed convex set. We discuss the scenario where the communication of the whole multiagent network is expressed as a sequence of time-varying general unbalanced directed graphs. The directed graphs are required to be uniformly jointly strongly connected and the weight matrices are only row-stochastic. To collaboratively deal with the optimization problems, existing distributed methods mostly require the communication graph to be fixed or balanced, which is impractical and hardly inevitable. In contrast, we propose a new distributed projection subgradient algorithm which is applicable to the time-varying general unbalanced directed graphs and does not need each agent to know its in-neighbors’ out-degree. When the objective functions are convex and Lipschitz continuous, it is proved that the proposed algorithm exactly converges to the optimal solution. Simulation results on a numerical experiment are shown to substantiate feasibility of the proposed algorithm and correctness of the theoretical findings. [ABSTRACT FROM AUTHOR]

This paper presents a discrete-time distributed and noncooperative routing algorithm, which is proved, via Lyapunov arguments, to asymptotically converge to a specific equilibrium condition among the traffic flows over the network paths, known as Wardrop equilibrium. This convergence result improves the discrete-time algorithms in the literature, which achieve approximate convergence to the Wardrop equilibrium. Numerical simulations show the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

This paper discusses the computation of matrix chain products of the form M1 X M2 X ... X Mn where Mi's are matrices. The order in which the matrices are computed affects the number of operations. A sufficient condition about the association of the matrices in the optimal order is presented. An O( n) algorithm to find an order of computation which takes less than 25 percent longer than the optimal time Topt is also presented. In most cases, the algorithm yields the optimal order or an order which takes only a few percent longer than Topt (less than I percent on the average). [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of distributed control for network systems aiming to achieve optimal steady-state performance. Motivated by recent research on reengineering cyber-physical systems, such as power systems and the Internet, we propose a two-step control retrofit procedure. First, we reformulate the dynamical system as an optimization algorithm to solve a certain optimization problem. Second, we combine a predefined steady-state optimization problem and the reformulated problem to systematically (re)design the control. As a result, the system automatically tracks the optimal solution of the predefined steady-state optimization problem and the control scheme can be implemented in a distributed and closed-loop manner. In order to investigate how general this framework is, we establish necessary and sufficient conditions under which a linear dynamical system can be viewed as an optimization algorithm. These conditions are characterized using properties of system matrices and related linear matrix inequalities. A practical example of frequency control in power systems shows the effectiveness of the proposed framework. [ABSTRACT FROM PUBLISHER]

Several sparsity-constrained algorithms, such as orthogonal matching pursuit (OMP) or the Frank–Wolfe (FW) algorithm, with sparsity constraints work by iteratively selecting a novel atom to add to the current nonzero set of variables. This selection step is usually performed by computing the gradient and then by looking for the gradient component with maximal absolute entry. This step can be computationally expensive especially for large-scale and high-dimensional data. In this paper, we aim at accelerating these sparsity-constrained optimization algorithms by exploiting the key observation that, for these algorithms to work, one only needs the coordinate of the gradient’s top entry. Hence, we introduce algorithms based on greedy methods and randomization approaches that aim at cheaply estimating the gradient and its top entry. Another of our contribution is to cast the problem of finding the best gradient entry as a best-arm identification in a multiarmed bandit problem. Owing to this novel insight, we are able to provide a bandit-based algorithm that directly estimates the top entry in a very efficient way. Theoretical observations stating that the resulting inexact FW or OMP algorithms act, with high probability, similar to their exact versions are also given. We have carried out several experiments showing that the greedy deterministic and the bandit approaches we propose can achieve an acceleration of an order of magnitude while being as efficient as the exact gradient when used in algorithms, such as OMP, FW, or CoSaMP. [ABSTRACT FROM PUBLISHER]

The recent literature indicates that preserving global pairwise sample similarity is of great importance for feature selection and that many existing selection criteria essentially work in this way. In this paper, we argue that besides global pairwise sample similarity, the local geometric structure of data is also critical and that these two factors play different roles in different learning scenarios. In order to show this, we propose a global and local structure preservation framework for feature selection (GLSPFS) which integrates both global pairwise sample similarity and local geometric data structure to conduct feature selection. To demonstrate the generality of our framework, we employ methods that are well known in the literature to model the local geometric data structure and develop three specific GLSPFS-based feature selection algorithms. Also, we develop an efficient optimization algorithm with proven global convergence to solve the resulting feature selection problem. A comprehensive experimental study is then conducted in order to compare our feature selection algorithms with many state-of-the-art ones in supervised, unsupervised, and semisupervised learning scenarios. The result indicates that: 1) our framework consistently achieves statistically significant improvement in selection performance when compared with the currently used algorithms; 2) in supervised and semisupervised learning scenarios, preserving global pairwise similarity is more important than preserving local geometric data structure; 3) in the unsupervised scenario, preserving local geometric data structure becomes clearly more important; and 4) the best feature selection performance is always obtained when the two factors are appropriately integrated. In summary, this paper not only validates the advantages of the proposed GLSPFS framework but also gains more insight into the information to be preserved in different feature selection tasks. [ABSTRACT FROM AUTHOR]

• In this paper, proposing a novel design of DE for Knapsack problems is reported. • A dual representation of solutions in continuous and binary is introduced. • New efficient fitness evaluation and repairing method is proposed. • Results demonstrated the efficiency of Novel DE in solving Knapsack problems. • Novel DE defined new solutions, better than best-known ones for 5 large instances. The capability of the conventional differential evolution algorithm to solve optimization problems in continuous spaces has been well demonstrated and documented in the literature. However, differential evolution has been commonly considered inapplicable for several binary/permutation-based real-world problems because of its arithmetic reproduction operator. Moreover, many limitations of the standard differential evolution algorithm, such as slow convergence and becoming trapped in local optima, have been defined. In this paper, a novel technique which makes a simple differential evolution algorithm suitable and very effective for solving binary-based problems, such as binary knapsack ones, is proposed. It incorporates new components, such as representations of solutions, a mapping method and a diversity technique. Also, a new efficient fitness evaluation approach for calculating and, at the same time, repairing knapsack candidate solutions, is introduced. To assess the performance of this new algorithm, four datasets with a total of 44 instances of binary knapsack problems are considered. Its performance and those of 22 state-of-the-art algorithms are compared, with the experimental results demonstrating its superiority in terms of both the quality of solutions and computational times. It is also capable of finding new solutions which are better than the current best ones for five large knapsack problems. [ABSTRACT FROM AUTHOR]

This note presents further results based on the recent paper [J. Liang, H. Chen, and J. Lam, “An improved criterion for controllability of Boolean control networks,” IEEE Trans. Autom. Control, vol. 62, no. 11, pp. 6012–6018, Nov. 2017]. After some optimizations, the conventional method can be more efficient than the method used in the above paper. We also propose an improved method via combining the well known Tarjan's algorithm and depth-first search technique for the controllability analysis of Boolean control networks (BCNs). As a result, the computational complexity will not exceed $O(N^2)$ with $N=2^n$ , where $n$ is the number of state-variables in a BCN. [ABSTRACT FROM AUTHOR]

Nonbinary low-density parity-check (NB-LDPC) codes constructed over GF(q) (q>2) can achieve higher coding gain than binary LDPC codes when the code length is moderate. A complete partial-parallel decoder architecture based on the Min-max algorithm is proposed for quasi-cyclic NB-LDPC codes in this paper. A novel scheme and corresponding architecture are developed to implement the elementary step of the check node processing. In our design, layered decoding is applied and only nm

This paper presents a novel method to compute the distance to instability for linear time-invariant delay systems with multiple delays, subjected to combined uncertainties on the system matrices and the delay parameters. The method is able to fully exploit structure and possible interdependencies between the uncertainties affecting the coefficient matrices, the property that realistic perturbations are real valued, and the structure of the delay equation. The resulting algorithm relies on a characterization in terms of the pseudospectral abscissa and on techniques from optimization over manifolds of matrices. [ABSTRACT FROM AUTHOR]

In this paper, we study the problem of achieving average consensus over a random time-varying sequence of directed graphs by extending the class of so-called push-sum algorithms to such random scenarios. Provided that an ergodicity notion, which we term the directed infinite flow property, holds and the auxiliary states of agents are uniformly bounded away from zero infinitely often, we prove the almost sure convergence of the evolutions of this class of algorithms to the average of initial states. Moreover, for a random sequence of graphs generated using a so-called time-varying $B$ -irreducible probability matrix, we establish convergence rates for the proposed push-sum algorithm. [ABSTRACT FROM AUTHOR]

This paper is concerned with the design of observers and dynamic output-feedback controllers for positive linear systems with interval uncertainties. The continuous-time case and the discrete-time case are both treated in a unified linear matrix inequality (LMI) framework. Necessary and sufficient conditions for the existence of positive observers with general structure are established, and the desired observer matrices can be constructed easily through the solutions of LMIs. An optimization algorithm to the error dynamics is also given. Furthermore, the problem of positive stabilization by dynamic output-feedback controllers is investigated. It is revealed that an unstable positive system cannot be positively stabilized by a certain dynamic output-feedback controller without taking the positivity of the error signals into account. When the positivity of the error signals is considered, an LMI-based synthesis approach is provided to design the stabilizing controllers. Unlike other conditions which may require structural decomposition of positive matrices, all proposed conditions in this paper are expressed in terms of the system matrices, and can be verified easily by effective algorithms. Two illustrative examples are provided to show the effectiveness and applicability of the theoretical results. [ABSTRACT FROM AUTHOR]

The strong γk-γcl H∞ stabilization problem is addressed in this paper. Based on the bounded real lemma, an improved sufficient condition on strong H∞, stabilization as well as its dual form is derived, which extends the ARE constraint in the former results into an inequality constraint. Two path-following algorithms are proposed to solve the associated optimization problems with BMI constraints, which lead to full order controllers. A numerical example is presented to demonstrate the effectiveness of the result. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new approach to the theory of finite multichain Markov decision processes (MDPs) with different performance optimization criteria. We first propose the concept of nth-order bias; then, using the average reward and bias difference formulas derived in this paper, we develop an optimization theory for finite MDPs that covers a complete spectrum from average optimality, bias optimality, to all high-order bias optimality, in a unified way. The approach is simple, direct, natural, and intuitive; it depends neither on Laurent series expansion nor on discounted MDPs. We also propose one-phase policy iteration algorithms for bias and high-order bias optimal policies, which are more efficient than the two-phase algorithms in the literature. Furthermore, we derive high-order bias optimality equations. This research is a part of our effort in developing sensitivity-based learning and optimization theory. [ABSTRACT FROM AUTHOR]

In this paper, a novel optimization technique is proposed to optimize filter coefficients of linear phase finite-impulse response (FIR) filter to share common subexpressions within and among coefficients. Existing approaches of common subexpression elimination optimize digital filters in two stages: first, an FIR filter is designed in a discrete space such as finite wordlength space or signed power-of-two (SPT) space to meet a given specification; in the second stage, an optimization algorithm is applied on the discrete coefficients to find and eliminate the common subexpressions. Such a two-stage optimization technique suffers from the problem that the search space in the second stage is limited by the finite wordlength or SPT coefficients obtained in the first stage optimization. The new proposed algorithm overcomes this problem by optimizing the filter coefficients directly in subexpression space for a given specification. Numerical examples of benchmark filters show that the required number of adders obtained using the proposed algorithm is much less than those obtained using two-stage optimization approaches. [ABSTRACT FROM AUTHOR]

Designing optimal Perfect reconstruction (PR) and near PR (NPR) cosine-modulated filter banks is essentially a constrained nonlinear minimization problem. This paper proposes two seconds order cone programming based algorithms for designing NPR and practically PR cosine modulated filter banks with improved performance relative to several established design methods. In essence, the design of a cosine-modulated filter bank is a nonconvex constrained optimization problem. The focus of this paper is primarily on developing two new optimization algorithms that are tailored to the time-domain formulation set forth in so as to obtain improved designs relative to several existing methods.

In evolutionary multiobjective optimization, effectiveness refers to how an evolutionary algorithm performs in terms of converging its solutions into the Pareto front and also diversifying them over the front. This is not an easy job, particularly for optimization problems with more than three objectives, dubbed many-objective optimization problems. In such problems, classic Pareto-based algorithms fail to provide sufficient selection pressure towards the Pareto front, whilst recently developed algorithms, such as decomposition-based ones, may struggle to maintain a set of well-distributed solutions on certain problems (e.g., those with irregular Pareto fronts). Another issue in some many-objective optimizers is rapidly increasing computational requirement with the number of objectives, such as hypervolume-based algorithms and shift-based density estimation (SDE) methods. In this paper, we aim to address this problem and develop an effective and efficient evolutionary algorithm (E3A) that can handle various many-objective problems. In E3A, inspired by SDE, a novel population maintenance method is proposed to select high-quality solutions in the environmental selection procedure. We conduct extensive experiments and show that E3A performs better than 11 state-of-the-art many-objective evolutionary algorithms in quickly finding a set of well-converged and well-diversified solutions. [ABSTRACT FROM AUTHOR]

Feature selection and feature transformation, the two main ways to reduce dimensionality, are often presented separately. In this paper, a feature selection method is proposed by combining the popular transformation-based dimensionality reduction method linear discriminant analysis (LDA) and sparsity regularization. We impose row sparsity on the transformation matrix of LDA through \ell 2,1 -norm regularization to achieve feature selection, and the resultant formulation optimizes for selecting the most discriminative features and removing the redundant ones simultaneously. The formulation is extended to the \ell 2,p -norm regularized case, which is more likely to offer better sparsity when $0

This paper considers a class of stochastic source-seeking problems to drive a mobile robot to the minimizer of a source signal. Our approach is first analyzed in an obstacle-free scenario, where measurements of the signal at the robot location and information of a contact sensor are required. We extend our results to environments with obstacles under mild assumptions on the step size. Our approach builds on the simultaneous perturbation stochastic approximation idea to obtain information of the signal field. We prove the practical convergence of the algorithms to a ball whose size depends on the step size that contains the location of the source. The novelty relies in that we consider nondifferentiable convex functions, a fixed step size, and the environment may contain obstacles. Our proof methods employ nonsmooth Lyapunov function theory, tools from convex analysis, and stochastic difference inclusions. Finally, we illustrate the applicability of the proposed algorithms in a two-dimensional scenarios. [ABSTRACT FROM AUTHOR]

The distributed convex optimization problem is studied in this paper for any fixed and connected network with general constraints. To solve such an optimization problem, a new type of continuous-time distributed subgradient optimization algorithm is proposed based on the Karuch–Kuhn–Tucker condition. By using tools from nonsmooth analysis and set-valued function theory, it is proved that the distributed convex optimization problem is solved on a network of agents equipped with the designed algorithm. For the case that the objective function is convex but not strictly convex, it is proved that the states of the agents associated with optimal variables could converge to an optimal solution of the optimization problem. For the case that the objective function is strictly convex, it is further shown that the states of agents associated with optimal variables could converge to the unique optimal solution. Finally, some simulations are performed to illustrate the theoretical analysis. [ABSTRACT FROM AUTHOR]

This paper considers the problem of a leader that seeks to optimally influence the opinions of agents in a directed network through connecting with a limited number of the agents (“direct followers”), possibly in the presence of a fixed competing leader. The settings involving a single leader and two competing leaders are unified into a general combinatoric optimization problem, for which two heuristic approaches are developed. The first approach is based on a convex relaxation scheme, possibly in combination with the $\ell _1$ -norm regularization technique, and the second is based on a greedy selection strategy. The main technical novelties of this work are in the establishment of supermodularity of the objective function and convexity of its continuous relaxation. The greedy approach is guaranteed to have a lower bound on the approximation ratio sharper than $(1-1/e)$ , while the convex approach can benefit from efficient (customized) numerical solvers to have practically comparable solutions possibly with faster computation times. The two approaches can be combined to provide improved results. In numerical examples, the approximation ratio can be made to reach ${\text{90}\%}$ or higher depending on the number of direct followers. [ABSTRACT FROM AUTHOR]

Single-layer feedforward networks (SLFNs) have been proven to be a universal approximator when all the parameters are allowed to be adjustable. It is widely used in classification and regression problems. The SLFN learning involves two tasks: determining network size and training the parameters. Most current algorithms could not be satisfactory to both sides. Some algorithms focused on construction and only tuned part of the parameters, which may not be able to achieve a compact network. Other gradient-based optimization algorithms focused on parameters tuning while the network size has to be preset by the user. Therefore, trial-and-error approach has to be used to search the optimal network size. Because results of each trial cannot be reused in another trial, it costs much computation. In this paper, a hybrid constructive (HC)algorithm is proposed for SLFN learning, which can train all the parameters and determine the network size simultaneously. At first, by combining Levenberg–Marquardt algorithm and least-square method, a hybrid algorithm is presented for training SLFN with fixed network size. Then,with the hybrid algorithm, an incremental constructive scheme is proposed. A new randomly initialized neuron is added each time when the training entrapped into local minima. Because the training continued on previous results after adding new neurons, the proposed HC algorithm works efficiently. Several practical problems were given for comparison with other popular algorithms. The experimental results demonstrated that the HC algorithm worked more efficiently than those optimization methods with trial and error, and could achieve much more compact SLFN than those construction algorithms. [ABSTRACT FROM PUBLISHER]

This paper proposes a distributed model predictive control (MPC) approach for a family of discrete-time linear systems with local (uncoupled) and global (coupled) constraints. The proposed approach is based on the dual problem of an overall MPC optimization problem involving all systems, which is then solved distributively using a modified distributed Nesterov-accelerated-gradient algorithm. To further reduce the computational requirement, this approach allows for early termination of the distributed gradient algorithm. This is made possible via a consensus algorithm that determines the satisfaction of the termination condition and by appropriate tightening of the coupled constraints. Under reasonable assumptions, the approach is able to produce a suboptimal solution as long as the network of the systems is connected while ensuring recursive feasibility and exponential stability of the closed-loop system. The performance of the proposed approach is demonstrated by a numerical example. [ABSTRACT FROM AUTHOR]

Several optimization-based control design techniques can be cast in the form of parametric optimization problems. The multiparametric quadratic programming (mpQP) represents a popular class often related to the control of constrained linear systems. The complete solution to mpQP takes the form of explicit feedback functions with a piecewise affine structure, valid in polyhedral partitions of the feasible parameter space known as critical regions. The recently proposed combinatorial approach for solving mpQP has shown better efficiency than geometric approaches in finding the complete solution to problems with high dimensions of the parameter vectors. The drawback of this method, on the other hand, is that it tends to become very slow as the number of constraints increases in the problem. This paper presents an alternative method for enumerating all optimal active sets in an mpQP based on theoretical properties of adjacent critical regions and their corresponding optimal active sets. Consequently, it results in excluding a noticeable number of feasible but not optimal candidate active sets from investigation. Therefore, the number of linear programs that should be solved decreases noticeably and the algorithm becomes faster. Simulation results confirm the reliability of the suggested method in finding the complete solution to the mpQPs while decreasing the computational time compared favorably with the best alternative approaches. [ABSTRACT FROM AUTHOR]

This paper investigates the performance of 35 dynamic memory allocation algorithms when used to service simulation programs as represented by 18 test cases. Algorithm performance was measured in terms of processing time, memory usage, and external memory fragmentation. Algorithms maintaining separate free space lists for each size of memory block used tended to perform quite well compared with other algorithms. Simple algorithms operating on memory ordered lists (without any free list) performed surprisingly well. Algorithms employing power-of-two block sizes had favorable processing requirements hut generally unfavorable memory usage. Algorithms employing LIFO, FIFO, or memory ordered free lists generally performed poorly compared with others. [ABSTRACT FROM AUTHOR]

Nonnegative matrix factorization (NMF) has recently become a very popular unsupervised learning method because of its representational properties of factors and simple multiplicative update algorithms for solving the NMF. However, for the common NMF approach of minimizing the Euclidean distance between approximate and true values, the convergence of multiplicative update algorithms has not been well resolved. This paper first discusses the convergence of existing multiplicative update algorithms. We then propose a new multiplicative update algorithm for minimizing the Euclidean distance between approximate and true values. Based on the optimization principle and the auxiliary function method, we prove that our new algorithm not only converges to a stationary point, but also does faster than existing ones. To verify our theoretical results, the experiments on three data sets have been conducted by comparing our proposed algorithm with other existing methods. [ABSTRACT FROM AUTHOR]

Distributed optimization algorithms have the advantages of privacy protection and parallel computing. However, the distributed nature of these algorithms makes the system vulnerable to external attacks. This paper presents two penalty function based resilient algorithms for constrained distributed optimization under static and dynamic attacks. The objective function of the optimization problem is extended to nonsmooth ones and the convergence of the proposed algorithms in this case are proved under some mild conditions. Simulation experiments are performed and compared with some existing resilient primal-dual optimization algorithms using median-based mean estimator. For static attack, the proposed algorithm has better performance and faster convergence rate in the simulation experiments. For dynamic attack, the proposed algorithm has better performance and robustness in the simulation experiments, which illustrate that the proposed algorithms are more effective. [ABSTRACT FROM AUTHOR]

Data clustering is one of the most common and challenging problems in the machine learning domain. It requires an efficient method to be addressed. This paper proposed a new version of the Flow Direction Algorithm (FDA) to solve various optimization problems. The proposed method is called FDAOA, which enhanced the performance of the original Flow Direction Algorithm by the arithmetic operators that have been used in the Arithmetic Optimization Algorithm (AOA). The main aim of the proposed FDAOA is to avoid the recognized weaknesses in the original methods; stuck in the local area, premature convergence, and weak equilibrium between the exploration and exploitation search mechanisms. The proposed method is tested on two sets of various problems to validate its performance. In the first set, twenty-three benchmark functions are used, which belong to three categories; seven unimodal functions, six multimodal functions, and ten fixed dimension functions. In the second set, eight common data clustering problems are used to prove the ability of the proposed FDAOA to deal with real-world optimization problems. The results of the proposed method are compared with other well-established methods, and the proposed FDAOA achieved promising output compared to the other methods on various tested problems. The proposed method got the optimal clustering solutions almost in all the tested data clustering problems with clear significant improvements against the other comparative methods. [ABSTRACT FROM AUTHOR]

In conventional sparse Space-Time Adaptive Processing (STAP) algorithms, the l 0 -norm is often used instead of the l 1 -norm to relax convexity constraints. The approach presented in this paper can be mathematically reformulated as a linear problem, which can be efficiently solved using various convex optimization techniques. While this method effectively circumvents the NP-Hard complexity associated with the l 0 -norm, it does come with certain potential challenges. For example, the observation matrix must meet the criteria of the Restricted Isometry Property (RIP) and other stringent conditions. In this research, a novel STAP algorithm is introduced, which is based on a non-convex penalty function. This innovative algorithm substitutes the traditional l 0 -norm with a custom-designed non-convex penalty function and is applied to the Direct Filter Processor (DFP), solved using the recursive least squares (RLS) algorithm. The resulting algorithm, known as g p -RLS, is demonstrated through Monte Carlo simulations to outperform other l 1 -based and reduced-rank STAP algorithms in terms of faster convergence, improved signal-to-clutter-plus-noise ratio (SCNR), and enhanced target detection performance. [ABSTRACT FROM AUTHOR]

• A supervised multi-view model is proposed to realize a block-based feature selection. • The proposed model is composed of all sharing sub-models in each class. • The sparse regularizer can enhance the sparsity of blocks from features and views. • The proposed algorithm can realize the block separation and independent solution. • Numerical experiments show the effectiveness of our method on large-scale datasets. With the increasing scale of obtained multi-view data, how to deal with large-scale multi-view data quickly and efficiently is a significant problem. In this paper, a novel supervised multi-view feature selection method based on locally sparse regularization and block computing is proposed to solve the problem. Specifically, the multi-view dataset is firstly divided into sub-blocks according to classes and views. Then with the aid of the Alternating Direction Method of Multipliers (ADMM), a sharing sub-model is proposed to perform feature selection on each class by integrating each view's locally sparse regularizers and shared loss that makes all views share a common penalty and regresses samples to their labels. Finally, all the sharing sub-models are fused to form the final general additive feature selection model, in which each sub-block adjusts its corresponding variables to perform block-based feature selection. In the optimization process, the proposed model can be decomposed into multiple separate subproblems, and an efficient optimization algorithm is proposed to solve them quickly. The comparison experiments with several state-of-the-art feature selection methods show that the proposed method is superior in classification accuracy and training speed. [ABSTRACT FROM AUTHOR]

This paper deals with a method for the approximation of a spectral density function among the solutions of a generalized moment problem à la Byrnes/Georgiou/Lindquist. The approximation is pursued with respect to the Kullback–Leibler pseudo-distance, which gives rise to a convex optimization problem. After developing the variational analysis, we discuss the properties of an efficient algorithm for the solution of the corresponding dual problem, based on the iteration of a nonlinear map in a bounded subset of the dual space. Our main result is the proof of local convergence of the latter, established as a consequence of the central manifold theorem. Supported by numerical evidence, we conjecture that, in the mentioned bounded set, the convergence is actually global. [ABSTRACT FROM AUTHOR]

One of the fundamental problems in the area of large-scale optimization is to study locality features of spatially distributed optimization problems in which the variables are coupled in the cost function as well as constraints. Such problems can motivate the development of fast and well-conditioned distributed algorithms. In this paper, we study spatial locality features of large-scale multi-parametric quadratic programming (MPQP) problems with linear inequality constraints. Our main application focus is receding horizon control of spatially distributed linear systems with input and state constraints. We propose a new approach for analysis of large-scale MPQP problems by blending tools from duality theory with operator theory. The class of spatially decaying matrices is introduced to capture couplings between optimization variables in the cost function and the constraints. We show that the optimal solution of a convex MPQP is piecewise affine- represented as convolution sums. More importantly, we prove that the kernel of each convolution sum decays in the spatial domain at a rate proportional to the inverse of the corresponding coupling function of the optimization problem. [ABSTRACT FROM AUTHOR]

Higher order sliding mode (HOSM) control design is considered for systems with a known permanent relative degree. In this paper, we introduce the Robust Fuller's Problem that is a robust generalization of the Fuller's problem, a standard optimal control problem for a chain of integrators with bounded control. By solving the Robust Fuller's Problem it is possible to obtain feed. back laws that are HOSM algorithms of generic order and, in addition, provide optimal finite-time reaching of the sliding manifold. A common difficulty in the use of existing HOSM algorithms is the tuning of design parameters: our methodology proves useful for the tuning of HOSM controller parameters in order to assure desired performances and prevent instabilities. The convergence and stability properties of the proposed family of controllers are theoretically analyzed. Simulation evidence demonstrates their effectiveness. [ABSTRACT FROM AUTHOR]

Adaptive infinite impulse response (IIR) filters provide significant advantages over equivalent finite impulse response (FIR) implementations because they are able to more accurately model physical plants that have pole-zero structures. Additionally, they are typically capable of meeting performance specifications using fewer filter parameters. This savings in parameters, which can be as much as 5-10 times, leads to the use of fewer multiplier blocks and therefore, lower power consumption. Despite these advantages, adaptive hR filters have not found widespread use because the associated mean squared error (MSE) cost function is multimodal and therefore, significantly difficult to minimize. Additionally, the filter can become unstable during adaptation. These two properties pose several problems for adaptive algorithms, causing them to be sensitive to initial conditions, produce biased solutions, unstable filter configurations or converge to local minima. These problems prevent the widespread use of adaptive hR filters in practice and if such filter structures are to become more practical, new, innovative solutions are required. This paper proposes a new algorithm for minimizing the MSE cost function of adaptive hR filters aimed at addressing some of the aforementioned issues. We adopt the approach of using a Branch-and-Bound algorithm, which is an exhaustive search method, and employ interval arithmetic for all computations. Simulation results show that the resulting algorithm is viable and competitive and, when compared with a number of existing state-of-the-art algorithms, outperforms them in terms of the MSE of the final point. [ABSTRACT FROM AUTHOR]

Optimal power flow (OPF) problem has already been attempted as a static optimization problem, by adopting conventional gradient-based methods and more recently, nonconventional ones, such as evolutionary algorithms. However, as the loads, generation capacities and network connections in a power system are always in a changing status, these static-oriented methods are of limited use for this issue. This paper presents a new algorithm, dynamic bacterial foraging algorithm (DBFA), for solving an OPF problem in a dynamic environment in which system loads are changing. DBFA is based on the recently proposed BFA which mimics the basic foraging behavior of E. coli bacteria. A selection scheme for bacteria's reproduction is employed in DBFA, which explores the self-adaptability of each bacterium in the group searching activities. DBFA has been evaluated, for optimizing the power system fuel cost with the OPF embedded, on the standard IEEE 30-bus and 118-bus test systems, respectively, with a range of load changes which occurred in different probabilities. The simulation results show that DBFA can more rapidly adapt to load changes, and more closely trace the global optimum of the system fuel cost, in comparison with BFA and particle swarm optimizer. [ABSTRACT FROM AUTHOR]

For the control of discrete event systems, the notion of directed control refines that of supervisory control. A directed controller is one that selects at most one controllable event to be enabled at any state (without disabling any uncontrollable event), which is in fact how a discrete event control is implemented. In contrast, a supervisory controller computes a maximal allowable set of controllable events at each state, leaving undecided exactly which controllable event should be enabled. In previous works, we developed a framework for the computation of optimal directed controllers and a polynomial synthesis algorithm for acydlic plants. In this paper, we present a novel synthesis approach for general plants, i.e., plants with or without cycles, thus providing a complete solution to the optimal directed control problem. The complexity of the approach remains polynomial in the size of plant. [ABSTRACT FROM AUTHOR]

This paper presents a constructive method for (sub)optimal finite-impulse response (FIR) approximation of infinite-impulse response (IIR) models. The method minimizes the Hankel norm of the approximation error by using an explicit solution to the norm-preserved dilation problem. It has advantages over the existing methods in that it is a unified method for both single-input single-output and multiple-input multiple-output systems which allows direct tradeoff between .the least-squares and Chebyshev error criteria by using a single tuning parameter, and that the approximation algorithm is constructive and only involves algebraic manipulations rather than iteration and convex optimization procedures. The lower and upper bounds on the l² and Chebyshev norms of the approximation error are derived and are related to the tuning parameter. The problem of approximating noncausal IIR models by causal FIR models is also formulated and solved. The effectiveness and properties of the proposed algorithms are demonstrated by examples. [ABSTRACT FROM AUTHOR]

In this paper, we present robust stability results for constrained discrete-time nonlinear systems using a finite-horizon model predictive control (MPC) algorithm for which we do not require the terminal cost to have any particular properties. We introduce a definition that attempts to characterize the robustness properties of the MPC optimization problem. We assume the systems under consideration satisfy this definition (for which we give sufficient conditions) and make two further assumptions. These are that the value function is bounded by a K∞ function of a state measure (related to the distance from the state to some target set) and that this measure is detectable from the stage cost used in the MPC algorithm. We show that these assumptions lead to stability that is robust to sufficiently small disturbances. While in general the results are semiglobal and practical, when the detectability and upper bound assumptions are satisfied with linear K∞ functions, the stability and robustness are either semiglobal or global (with respect to the feasible set). We discuss algorithms employing terminal inequality constraints and also provide a specific example of an algorithm that employs a terminal equality constraint. [ABSTRACT FROM AUTHOR]

This paper describes an efficient algorithm for designing lattice wave digital (LWD) filters (parallel connections of two all-pass filters) with short-coefficient wordlength. The coefficient optimization is performed using the following three steps. First, an initial infinite-precision filter is designed such that it exceeds the given criteria in order to provide some tolerance for co-efficient quantization. Second, a nonlinear optimization algorithm is used for determining a parameter space of the infinite-precision coefficients including the feasible space where the filter meets the given criteria. The third step involves finding the filter parameters in this space so that the resulting filter meets the given criteria with the simplest coefficient representation forms. The proposed algorithm guarantees that the optimum finite-precision solution can be found for both the fixed-point binary and multiplierless coefficient representation forms. In addition, this algorithm is applicable for producing the desired finite-precision solutions for both conventional and approximately linear-phase LWD filters. Comparisons with some other existing quantization schemes show that the proposed algorithm gives the best finite-precision solutions in all examples taken from the literature. [ABSTRACT FROM AUTHOR]

In this paper, the problem of maximizing power-system loadability with multiple complementarity constraints representing generator limits is examined. The structure of the loadability surface is investigated, the various type of limits are classified and a general algorithm to optimize settings of control variables, in order to maximize loadability margin is developed. The importance of nonsmooth corner points (CPs) of the loadability surface that are due to the simultaneous occurrence of several loadability conditions is discussed. At such limits, maximization, of loadability margin is performed based on techniques provided by nonlinear optimization theory. For this purpose an algorithm to identify multiple binding constraints on a CP is developed. Illustrative examples on small, but realistic systems are included. [ABSTRACT FROM AUTHOR]

This paper proposes two new methods for optimizing hardware resources in finite wordlength implementation of multiple-output (MO) linear time-invariant (LTI) systems. The hardware complexity is measured by the exact internal wordlength used for each intermediate data. The first method relaxes the wordlength from integer to real-value and formulates the design problem as a geometric programming, from which an optimal solution of the relaxed problem can be determined. The second method is based on a discrete optimization method called the marginal analysis method, and it yields the desired wordlengths in integer values. By combining these two methods, a hybrid method is also proposed, which is found to be very effective for large scale MO LTI systems. To illustrate the effectiveness of the proposed methods, wordlength optimization problems of two-channel structural perfect reconstruction filter banks and multiplier-less fast Fourier transforms are studied in detail. Design results show that the proposed algorithms offer better results and a lower design complexity than conventional methods. [ABSTRACT FROM AUTHOR]

Hybrid systems are known to be potential sources of electromagnetic interference due to their principle of recurrently switching voltages and currents. In this paper, we demonstrate different periodic modulation techniques and propose corresponding electromagnetic compatibility optimization methods. A systematic way to optimize finite length modulation sequences is proposed. We prepare mathematical expressions of performance indexes as a base for optimization algorithms. The performance of the different waveform generation methods is veryfied using a dc-dc converter example. It points out that the optimized waveforms are a powerful tool to shape the power spectrum. [ABSTRACT FROM AUTHOR]

In this paper, we use the multidimensional multiscale parser (MMP) algorithm, a recently developed universal lossy compression method, to compress data from electrocardiogram (ECG) signals. The MMP is based on approximate multiscale pattern matching, encoding segments of an input signal using expanded and contracted versions of patterns stored in a dictionary. The dictionary is updated using concatenated and displaced versions of previously encoded segments, therefore MMP builds its own dictionary while the input data is being encoded. The MMP can be easily adapted to compress signals of any number of dimensions, and has been successfully applied to compress two-dimensional (2-D) image data. The quasi-periodic nature of ECG signals makes them suitable for compression using recurrent patterns, like MMP does. However, in order for MMP to be able to efficiently compress ECG signals, several adaptations had to be performed, such as the use of a continuity criterion among segments and the adoption of a prune-join strategy for segmentation. The rate-distortion performance achieved was very good. We show simulation results were MMP performs as well as some of the best encoders in the literature, although at the expense of a high computational complexity. [ABSTRACT FROM AUTHOR]

Particle swarm optimization (PSO) is a population-based random-search optimization technique that has become an increasingly important branch of swarm intelligence studies. Population diversity is an effective measurement that shows the distribution of particles in a search space. In this paper, we propose a diversity-based PSO algorithm that is combined with multiple techniques, and population diversity is used as a search-strategy selection criterion for particles. When diversity is high, we suggest implementing a search strategy that has a strong exploration ability. When diversity is low, we suggest implementing a search strategy that has a strong exploitation ability. In addition to our diversity-based method, we introduce a gradient-based local-search technique, multi-crossover operation, and disturbance strategy to help improve the performance of the proposed algorithm. In the experiments, we compare the proposed algorithm with 10 advanced PSO variants based on 40 widely used benchmark functions, including the CEC2017 benchmark. The results indicate that the proposed algorithm yields a better solution accuracy and convergence speed than those of other PSO variants. [ABSTRACT FROM AUTHOR]