Search

The computation-intensive circuit simulation makes the analog circuit sizing challenging for large-scale/complicated analog/RF circuits. A Bayesian optimization approach has been proposed recently for the optimization problems involving the evaluations of black-box functions with high computational cost in either objective functions or constraints. In this paper, we propose a weighted expected improvement-based Bayesian optimization approach for automated analog circuit sizing. Gaussian processes (GP) are used as the online surrogate models for circuit performances. Expected improvement is selected as the acquisition function to balance the exploration and exploitation during the optimization procedure. The expected improvement is weighted by the probability of satisfying the constraints. In this paper, we propose a complete Bayesian optimization framework for the optimization of analog circuits with constraints for the first time. The existing GP model-based optimization methods for analog circuits take the GP models as either offline models or as assistance for the evolutionary algorithms. We also extend the Bayesian optimization algorithm to handle multi-objective optimization problems. Compared with the state-of-the-art approaches listed in this paper, the proposed Bayesian optimization method achieves better optimization results with significantly less number of simulations. [ABSTRACT FROM PUBLISHER]

The main focus of this paper is the presentation of reliable methods for the determination of the optimum coloring of a graph, commonly known in the literature as vertex coloring problem. It has been shown that networks of capacitively coupled oscillators can be used to solve vertex coloring problems. In this paper we address the negative impact of an unbalanced number of couplings for the oscillators on the performance of the network and compensate for this non-uniform coupling structure by an adjustment in the network itself. The negative effect of the memristor device-to-device variability of the NbOx memristor on the array functionality will be investigated and reduced via an adaptation of the memristor operating point. The main improvement in network performance is achieved by setting up a control procedure allowing the network to bypass the local solutions and converge to the global one. Two strategies inspired by global optimization algorithms will be proposed to allow the network to overcome sub-optimal solutions, and find the solution corresponding to the absolute minimum of a performance measure function of the vertex coloring problem. [ABSTRACT FROM AUTHOR]

To better understand the correlation between network topological features and the robustness of network controllability in a general setting, this paper suggests a practical approach to searching for optimal network topologies with given numbers of nodes and edges. Since theoretical analysis seems impossible at least in the present time, exhaustive search based on optimization techniques is employed, firstly for a group of small-sized networks that are realistically workable, where exhaustive means 1) all possible network structures with the given numbers of nodes and edges are computed and compared, and 2) all possible node-removal sequences are considered. A main contribution of this paper is the observation of an empirical necessary condition (ENC) from the results of exhaustive search, which shrinks the search space to quickly find an optimal solution. ENC shows that the maximum and minimum in- and out-degrees of an optimal network structure should be almost identical, or within a very narrow range, i.e., the network should be extremely homogeneous. Edge rectification towards the satisfaction of the ENC is then designed and evaluated. Simulation results on large-sized synthetic and real-world networks verify the effectiveness of both the observed ENC and the edge rectification scheme. As more operations of edge rectification are performed, the network is getting closer to exactly satisfying the ENC, and consequently the robustness of the network controllability is enhanced towards optimum. [ABSTRACT FROM AUTHOR]

This paper develops and validates a comprehensive and universally applicable computational concept for solving nonlinear differential equations (NDEs) through a neurocomputing concept based on cellular neural networks (CNNs). High-precision, stability, convergence, and lowest-possible memory requirements are ensured by the CNN processor architecture. A significant challenge solved in this paper is that all these cited computing features are ensured in all system-states (regular or chaotic ones) and in all bifurcation conditions that may be experienced by NDEs.One particular quintessence of this paper is to develop and demonstrate a solver concept that shows and ensures that CNN processors (realized either in hardware or in software) are universal solvers of NDE models. The solving logic or algorithm of given NDEs (possible examples are: Duffing, Mathieu, Van der Pol, Jerk, Chua, Rössler, Lorenz, Burgers, and the transport equations) through a CNN processor system is provided by a set of templates that are computed by our comprehensive templates calculation technique that we call nonlinear adaptive optimization. This paper is therefore a significant contribution and represents a cutting-edge real-time computational engineering approach, especially while considering the various scientific and engineering applications of this ultrafast, energy-and-memory-efficient, and high-precise NDE solver concept. For illustration purposes, three NDE models are demonstratively solved, and related CNN templates are derived and used: the periodically excited Duffing equation, the Mathieu equation, and the transport equation. [ABSTRACT FROM PUBLISHER]

In this paper, we present a unified framework for solving a general class of problems arising in the context of set-membership estimation/identification theory. More precisely, the paper aims at providing an original approach for the computation of optimal conditional and robust projection estimates in a nonlinear estimation setting, where the operator relating the data and the parameter to be estimated is assumed to be a generic multivariate polynomial function, and the uncertainties affecting the data are assumed to belong to semialgebraic sets. By noticing that the computation of both the conditional and the robust projection optimal estimators requires the solution to min-max optimization problems that share the same structure, we propose a unified two-stage approach based on semidefinite-relaxation techniques for solving such estimation problems. The key idea of the proposed procedure is to recognize that the optimal functional of the inner optimization problems can be approximated to any desired precision by a multivariate polynomial function by suitably exploiting recently proposed results in the field of parametric optimization. Two simulation examples are reported to show the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

Checking validity of a model is a crucial step in the process of system identification. This is especially true when dealing with switched affine systems since, in this case, the problem of system identification from noisy data is known to be generically NP-Hard and can only be solved in practice by using heuristics and relaxations. Therefore, before the identified models can be used for instance for controller design, they should be systematically validated against additional experimental data. In this paper we address the problem of model (in)validation for multi-input multi-output switched affine systems in output error form with unknown switches. As a first step, we prove that necessary and sufficient invalidation certificates can be obtained by solving a sequence of convex optimization problems. In principle, these problems involve increasingly large matrices. However, as we show in the paper by exploiting recent results from semialgebraic geometry, the proposed algorithm is guaranteed to stop after a finite number of steps that can be be explicitly computed from the a priori information. In addition, this algorithm exploits the sparse structure of the underlying optimization problem to substantially reduce the computational burden. The effectiveness of the proposed method is illustrated using both academic examples and a non-trivial problem arising in computer vision: activity monitoring. [ABSTRACT FROM AUTHOR]

The aim of this paper is to approximate a Finite-State Markov (FSM) process by another process defined on a lower dimensional state space, called the approximating process, with respect to a total variation distance fidelity criterion. The approximation problem is formulated as an optimization problem using two different approaches. The first approach is based on approximating the transition probability matrix of the FSM process by a lower-dimensional transition probability matrix, resulting in an approximating process which is a Finite-State Hidden Markov (FSHM) process. The second approach is based on approximating the invariant probability vector of the original FSM process by another invariant probability vector defined on a lower-dimensional state space. Going a step further, a method is proposed based on optimizing a Kullback-Leibler divergence to approximate the FSHM processes by FSM processes. The solutions of these optimization problems are described by optimal partition functions which aggregate the states of the FSM process via a corresponding water-filling solution, resulting in lower-dimensional approximating processes which are FSHM or FSM processes. Throughout the paper, the theoretical results are justified by illustrative examples that demonstrate our proposed methodology. [ABSTRACT FROM AUTHOR]

This paper is concerned with the optimal static distributed control problem for linear discrete-time deterministic and stochastic systems. The objective is to design a stabilizing static distributed controller whose performance is close to that of the optimal centralized controller. To this end, we first consider deterministic systems, where the initial state is either given or belongs to a known bounded region. Given an arbitrary centralized controller, we derive a condition under which there exists a distributed controller that generates input and state trajectories close to their counterparts in the centralized closed-loop system. This condition for the design of a distributed controller is translated into an optimization problem, where the optimal objective value of this problem quantifies the closeness of the designed distributed and given centralized control systems. The results are then extended to stochastic systems that are subject to input disturbance and measurement noise. The proposed optimization problem has a closed-form solution (explicit formula) and can be efficiently solved for large-scale systems. The mathematical framework developed in this paper is utilized to design a near-globally optimal distributed controller based on the optimal centralized controller, and strong theoretical lower bounds on the global optimality guarantee of the obtained distributed controller are derived. We show that if the optimal objective value of the proposed convex program is sufficiently small, the designed controller is stabilizing and nearly globally optimal. To illustrate the results, case studies on aircraft formation and frequency control of power systems are offered. [ABSTRACT FROM AUTHOR]

In this paper, we present a framework for analytically estimating the output quality of common digital signal processing (DSP) blocks that utilize approximate adders. The framework is based on considering the error of approximate adders as an additive noise (approximation noise) that disturbs the output of the DSP block in question. A signal processing theoretical modeling approach for describing the power of the approximation noise which is the integral of error spectral density over the bandwidth, is developed. The output qualities of DSP blocks, such as finite impulse response filter, discrete cosine transform, and fast Fourier transform, which utilize approximate adders, are thus estimated. The accuracy of the proposed framework is evaluated by comparing mathematical model predictions to simulation results by using the signal-to-noise ratio (SNR) metric. The inaccuracy of the SNRs predicted by the framework was, on average, less than 2.5dB compared with that obtained from simulations. Therefore, a mathematical optimization approach based on Lagrange Multipliers for optimizing design parameters is also presented. The optimization is realized by choosing a proper configuration of the target block, such as determining the data width of the inexact computation part for each approximate adder in the design. [ABSTRACT FROM AUTHOR]

In this paper, a built-in self-test and in situ analog circuit optimization platform is proposed and characterized. By integrating a fully digital optimization engine and self-test circuits along with circuits-under-optimization (CUO), this platform can automatically find an operating point that makes a good balance among multiple competing characteristic goals. Therefore, the arbitrary linear time-invariant CUO can be optimized even when there are large process-voltage-temperature variations and aging effects of devices. This platform is analyzed to determine the required accuracy of its building blocks in terms of noise and linearity. The feasibility of this platform is proved by a case study utilizing a Tow-Thomas bandpass biquad. [ABSTRACT FROM AUTHOR]

This paper presents a human-inspired control approach to bipedal robotic walking: utilizing human data and output functions that appear to be intrinsic to human walking in order to formally design controllers that provably result in stable robotic walking. Beginning with human walking data, outputs—or functions of the kinematics—are determined that result in a low-dimensional representation of human locomotion. These same outputs can be considered on a robot, and human-inspired control is used to drive the outputs of the robot to the outputs of the human. The main results of this paper are that, in the case of both under and full actuation, the parameters of this controller can be determined through a human-inspired optimization problem that provides the best fit of the human data while simultaneously provably guaranteeing stable robotic walking for which the initial condition can be computed in closed form. These formal results are demonstrated in simulation by considering two bipedal robots—an underactuated 2-D bipedal robot, AMBER, and fully actuated 3-D bipedal robot, NAO—for which stable robotic walking is automatically obtained using only human data. Moreover, in both cases, these simulated walking gaits are realized experimentally to obtain human-inspired bipedal walking on the actual robots. [ABSTRACT FROM PUBLISHER]

The paper proposes an identification procedure for autoregressive Gaussian stationary stochastic processes under the assumption that the manifest (or observed) variables are nearly independent when conditioned on a limited number of latent (or hidden) variables. The method exploits the sparse plus low-rank decomposition of the inverse of the manifest spectral density and the efficient convex relaxations recently proposed for such decompositions. [ABSTRACT FROM PUBLISHER]

In this paper, we consider the l\infty gain characterizations of linear switched systems (LSS) and present various relevant results on their exact computation and optimization. Depending on the role of the switching sequence, we study two broad cases: first, when the switching sequence attempts to maximize, and second, when it attempts to minimize the l\infty gain. The first, named as worst-case throughout the paper, can be related to robustness of the system to uncontrolled switching; the second relates to situations when the switching can be part to the overall decision making. Although, in general, the exact computation of l\infty gains is difficult, we provide specific classes, the input-output switching systems, for which it is shown that linear programming can be used to obtain the worst-case l\infty gain. This is a sufficiently rich class of systems as any stable LSS can be approximated by one. Certain applications to robust control design are provided where we show that a switched compensation independently of the plant has no advantage over a linear time invariant (LTI) compensation, and further, if the plant is strictly causal, even a switched compensation which has a matched switching with the plant does not provide a better performance over an LTI compensation. Also, we present a new necessary and sufficient condition to check the stability of LSS in form of a model matching problem. On the other hand, if one is interested in minimizing the l\infty gain over the switching sequences, we show that, for finite impulse response (FIR) switching systems the minimizing switching sequence can be chosen to be periodic. For input-only or output-only switching an exact, readily computable, characterization of the minimal l\infty gain is provided, and it is shown that the minimizing switching sequence is constant, which, as also shown, is not true for input-output switching. [ABSTRACT FROM AUTHOR]

The first part of this note identifies and analyzes two weak points of the commented paper refid="ref1"/. i) Example 1 reports an incorrect result for which we provide a complete explanation and correction. ii) The practical use of Theorem 1 reveals major drawbacks such as a severe limitation in the applicability (due to the restrictive conditions requested by the hypothesis) and a relatively low accuracy of the results (i.e., rough values for the right outer bounds of the eigenvalue ranges of interval matrices). The second part of the note presents three methods that circumvent the use of Theorem 1 refid="ref1"/ and provide the following information about the eigenvalue range of an interval matrix: \bullet a right outer bound—relying on an inequality proved by J. Rohn (see References); \bullet the right end point—by solving a constrained global maximization problem; \bullet a right outer bound (for arbitrary interval matrices) and the right end point (for some classes of interval matrices)—by calculating the spectral abscissa of a constant matrix that majorizes the considered interval matrix. For illustration by numerical examples, throughout the note we use three interval matrices related to Example 1 from the commented paper. [ABSTRACT FROM AUTHOR]

This paper presents an ultra-broadband 5-bit switched-type attenuator (STA). Three attenuation topologies are employed for the design of the attenuation cells, including the T-type, simplified T-type and Π-type topologies. For each attenuation topology, the optimal values of the series and shunt resistors are derived to achieve accurate amplitude tuning and ensure good impedance matching. The capacitive compensation technique is adopted by both the T-type and Π-type topologies to enhance the high-frequency performance. Based on the proposed optimization techniques, a systematic design methodology is developed for the STA. The 5-bit STA is implemented in 65-nm CMOS technology and occupies a core chip area of only 0.036 mm2. It exhibits ultra-broadband operation with 15.5-dB amplitude tuning range and 0.5-dB tuning step. The insertion loss of the reference state is 1.5 – 5.9 dB from DC to 50 GHz. The return loss is better than 12 dB for all the 32 states. The RMS amplitude error and phase error are less than 0.25 dB and 3.5° over the DC – 50 GHz band. [ABSTRACT FROM AUTHOR]

This paper considers the problem of selecting the top-m designs using simulation with input uncertainty. The performance of each design is measured by its worst case performance. The objective of this paper is to maximize the probability of correctly selecting the top-m designs given a fixed simulation budget. Due to the complexity of probability of correct selection (PCS), we develop a lower bound for the PCS and derive an asymptotically optimal budget allocation rule. Useful insights on characterizing the efficient budget allocation rule with input uncertainty are provided. Meanwhile, a sequential simulation procedure is suggested to implement the allocation rule. A series of numerical experiments indicate that the proposed simulation budget allocation rule can outperform all existing selection rules. [ABSTRACT FROM AUTHOR]

In model predictive control (MPC), the control input is computed by solving a constrained finite-time optimal control (CFTOC) problem at each sample in the control loop. The main computational effort when solving the CFTOC problem using an active-set (AS) method is often spent on computing the search directions, which in MPC corresponds to solving unconstrained finite-time optimal control (UFTOC) problems. This is commonly performed using Riccati recursions or generic sparsity exploiting algorithms. In this paper, the focus is efficient search direction computations for AS type methods. The system of equations to be solved at each AS iteration is changed only by a low-rank modification of the previous one, and exploiting this structured change is important for the performance of AS-type solvers. In this paper, theory for how to exploit these low-rank changes by modifying the Riccati factorization between AS iterations in a structured way is presented. A numerical evaluation of the proposed algorithm shows that the computation time can be significantly reduced by modifying, instead of re-computing, the Riccati factorization. This speedup can be important for AS-type solvers used for linear, nonlinear, and hybrid MPC. [ABSTRACT FROM PUBLISHER]

Finding the optimal solution to the constrained l0 -norm minimization problems in the recovery of compressive sensed signals is an NP-hard problem and it usually requires intractable combinatorial searching operations for getting the global optimal solution, unless using other objective functions (e.g., the l1 norm or lp norm) for approximate solutions or using greedy search methods for locally optimal solutions (e.g., the orthogonal matching pursuit type algorithms). In this paper, a neurodynamic optimization method is proposed to solve the l0 -norm minimization problems for obtaining the global optimum using a recurrent neural network (RNN) model. For the RNN model, a group of modified Gaussian functions are constructed and their sum is taken as the objective function for approximating the l0 norm and for optimization. The constructed objective function sets up a convexity condition under which the neurodynamic system is guaranteed to obtain the globally convergent optimal solution. An adaptive adjustment scheme is developed for improving the performance of the optimization algorithm further. Extensive experiments are conducted to test the proposed approach in this paper and the output results validate the effectiveness of the new method. [ABSTRACT FROM PUBLISHER]

This paper presents a loop gain optimization technique for integer-N digital phase-locked loops with a time-to-digital converter. Due to noise filtering properties, a phase-locked loop has an optimal loop gain which gives rise to the best jitter performance, taking into account external and internal noise sources. By using the loop gain optimization technique, the digital phase-locked loops can automatically attain this loop gain in background to minimize the jitter. Theoretical analysis is presented. The stability issue and the impact of loop latency are also discussed. Finally, the analysis is compared to behavioral simulations with good agreement. [ABSTRACT FROM AUTHOR]

This paper deals with the joint optimization problem of high-order error-feedback and realization for minimizing roundoff noise at the output of a closed-loop system with a full-order observer feedback system subject to l2-scaling constraints. It is shown that the problem can be converted into an unconstrained optimization problem by using linear-algebraic techniques. The unconstrained optimization problem at hand is then solved iteratively by employing an efficient quasi-Newton algorithm with closed-form formulas for key gradient evaluation. A case study is presented to demonstrate the validity and effectiveness of the proposed technique. [ABSTRACT FROM AUTHOR]

This paper proposes a quaternion-valued one-layer recurrent neural network approach to resolve constrained convex function optimization problems with quaternion variables. Leveraging the novel generalized Hamilton-real (GHR) calculus, the quaternion gradient-based optimization techniques are proposed to derive the optimization algorithms in the quaternion field directly rather than the methods of decomposing the optimization problems into the complex domain or the real domain. Via chain rules and Lyapunov theorem, the rigorous analysis shows that the deliberately designed quaternion-valued one-layer recurrent neural network stabilizes the system dynamics while the states reach the feasible region in finite time and converges to the optimal solution of the considered constrained convex optimization problems finally. Numerical simulations verify the theoretical results. [ABSTRACT FROM AUTHOR]

Many machine learning problems involve learning a low-rank positive semidefinite matrix. However, existing solvers for this low-rank semidefinite program (SDP) are often expensive. In this paper, by factorizing the target matrix as a product of two matrices and using a Courant penalty to penalize for their difference, we reformulate the SDP as a biconvex optimization problem. This allows the use of multiconvex optimization techniques to define simple surrogates, which can be minimized easily by block coordinate descent. Moreover, while traditionally this biconvex problem approaches the original problem only when the penalty parameter is infinite, we show that the two problems are equivalent when the penalty parameter is sufficiently large. Experiments on a number of SDP applications in machine learning show that the proposed algorithm is as accurate as other state-of-the-art algorithms, but is much faster, especially on large data sets. [ABSTRACT FROM AUTHOR]

Human preferences are usually measured using ordinal variables. A system whose goal is to estimate the preferences of humans and their underlying decision mechanisms requires to learn the ordering of any given sample set. We consider the solution of this ordinal regression problem using a support vector machine algorithm. Specifically, the goal is to learn a set of classifiers with common direction vectors and different biases correctly separating the ordered classes. Current algorithms are either required to solve a quadratic optimization problem, which is computationally expensive, or based on maximizing the minimum margin (i.e., a fixed-margin strategy) between a set of hyperplanes, which biases the solution to the closest margin. Another drawback of these strategies is that they are limited to order the classes using a single ranking variable (e.g., perceived length). In this paper, we define a multiple ordinal regression algorithm based on maximizing the sum of the margins between every consecutive class with respect to one or more rankings (e.g., perceived length and weight). We provide derivations of an efficient, easy-to-implement iterative solution using a sequential minimal optimization procedure. We demonstrate the accuracy of our solutions in several data sets. In addition, we provide a key application of our algorithms in estimating human subjects’ ordinal classification of attribute associations to object categories. We show that these ordinal associations perform better than the binary one typically employed in the literature. [ABSTRACT FROM AUTHOR]

In this paper, we consider a distributed optimization scenario, motivated by peak-demand minimization, in which a set of processors aims at cooperatively solving a class of min–max optimization problems. The min–max structure and a twofold coupling make the problem challenging in a distributed setup. We propose a distributed algorithm based on the derivation of a series of dual problems and the application of properties from min–max optimization. The resulting distributed algorithm, despite its complex derivation, has a simple structure consisting of a primal optimization and a suitable dual update. We prove the convergence of the proposed algorithm in objective value, and, moreover, that every limit point of the primal sequence is an optimal (thus feasible) solution. This primal recovery property is of key importance in applications since it allows each agent to compute its portion of the global optimal strategy without resorting to any recovery mechanism. Finally, we provide numerical computations for peak-demand optimization in a network of thermostatically controlled loads and show that our algorithm outperforms a plain distributed subgradient performed on the dual. [ABSTRACT FROM AUTHOR]

A regularized optimization problem over a large unstructured graph is studied, where the regularization term is tied to the graph geometry. Typical regularization examples include the total variation and the Laplacian regularizations over the graph. When the graph is a simple path without loops, efficient off-the-shelf algorithms can be used. However, when the graph is large and unstructured, such algorithms cannot be used directly. In this paper, an algorithm, referred to as “Snake,” is proposed to solve such regularized problems over general graphs. The algorithm consists in properly selecting random simple paths in the graph and performing the proximal gradient algorithm over these simple paths. This algorithm is an instance of a new general stochastic proximal gradient algorithm, whose convergence is proven. Applications to trend filtering and graph inpainting are provided among others. Numerical experiments are conducted over large graphs. [ABSTRACT FROM AUTHOR]

This paper considers the optimization problem for partially observable Markov decision processes (POMDPs) with the continuous state, observation, and action spaces. POMDPs with the discrete spaces have emerged as a promising approach to the decision systems with imperfect state information. However, in recent applications of POMDPs, there are many problems that have continuous states, observations, and actions. For such problems, due to the infinite dimensionality of the belief space, the existing studies usually discretize the continuous spaces with the sufficient or nonsufficient statistics, which may cause the curse of dimensionality and performance degradation. In this paper, based on the sensitivity analysis of the performance criteria, we have developed a simulation-based policy iteration algorithm to find the local optimal observation-based policy for POMDPs with the continuous spaces. The proposed algorithm needs none of the specific assumptions and prior information, and has a low computational complexity. One numerical example of the complicated multiple-input multiple-output beamforming problem shows that the algorithm has a significant performance improvement. [ABSTRACT FROM AUTHOR]

This paper proposes distributed algorithms to solve robust convex optimization (RCO) when the constraints are affected by nonlinear uncertainty. We adopt a scenario approach by randomly sampling the uncertainty set. To facilitate the computational task, instead of using a single centralized processor to obtain a “global solution” of the scenario problem (SP), we resort to multiple interconnected processors that are distributed among different nodes of a network to simultaneously solve the SP. Then, we propose a primal-dual subgradient algorithm and a random projection algorithm to distributedly solve the SP over undirected and directed graphs, respectively. Both algorithms are given in an explicit recursive form with simple iterations, which are especially suited for processors with limited computational capability. We show that, if the underlying graph is strongly connected, each node asymptotically computes a common optimal solution to the SP with a convergence rate $O(1/(\sum _{t=1}^k\zeta ^t))$ , where $\lbrace \zeta ^t\rbrace$ is a sequence of appropriately decreasing stepsizes. That is, the RCO is effectively solved in a distributed way. The relations with the existing literature on robust convex programs are thoroughly discussed and an example of robust system identification is included to validate the effectiveness of our distributed algorithms. [ABSTRACT FROM AUTHOR]

This paper is concerned with a general class of distributed constrained optimization problems over a multiagent network, where the global objective function is represented by the sum of all local objective functions. Each agent in the network only knows its own local objective function, and is restricted to a global nonempty closed convex set. We discuss the scenario where the communication of the whole multiagent network is expressed as a sequence of time-varying general unbalanced directed graphs. The directed graphs are required to be uniformly jointly strongly connected and the weight matrices are only row-stochastic. To collaboratively deal with the optimization problems, existing distributed methods mostly require the communication graph to be fixed or balanced, which is impractical and hardly inevitable. In contrast, we propose a new distributed projection subgradient algorithm which is applicable to the time-varying general unbalanced directed graphs and does not need each agent to know its in-neighbors’ out-degree. When the objective functions are convex and Lipschitz continuous, it is proved that the proposed algorithm exactly converges to the optimal solution. Simulation results on a numerical experiment are shown to substantiate feasibility of the proposed algorithm and correctness of the theoretical findings. [ABSTRACT FROM AUTHOR]

Separable nonlinear least-squares (SNLLS) problems arise frequently in many research fields, such as system identification and machine learning. The variable projection (VP) method is a very powerful tool for solving such problems. In this paper, we consider the regularization of ill-conditioned SNLLS problems based on the VP method. Selecting an appropriate regularization parameter is difficult because of the nonlinear optimization procedure. We propose to determine the regularization parameter using the weighted generalized cross-validation method at every iteration. This makes the original objective function changing during the optimization procedure. To circumvent this problem, we use an inequation to produce a consistent demand of decreasing at successive iterations. The approximation of the Jacobian of the regularized problem is also discussed. The proposed regularized VP algorithm is tested by the parameter estimation problem of several statistical models. Numerical results demonstrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Partial least squares regression (PLSR) has been a popular technique to explore the linear relationship between two data sets. However, all existing approaches often optimize a PLSR model in Euclidean space and take a successive strategy to calculate all the factors one by one for keeping the mutually orthogonal PLSR factors. Thus, a suboptimal solution is often generated. To overcome the shortcoming, this paper takes statistically inspired modification of PLSR (SIMPLSR) as a representative of PLSR, proposes a novel approach to transform SIMPLSR into optimization problems on Riemannian manifolds, and develops corresponding optimization algorithms. These algorithms can calculate all the PLSR factors simultaneously to avoid any suboptimal solutions. Moreover, we propose sparse SIMPLSR on Riemannian manifolds, which is simple and intuitive. A number of experiments on classification problems have demonstrated that the proposed models and algorithms can get lower classification error rates compared with other linear regression methods in Euclidean space. We have made the experimental code public at https://github.com/Haoran2014. [ABSTRACT FROM AUTHOR]

We consider networks the nodes of which are interconnected via directed edges, each able to admit a flow (or weight) within a certain interval, with nonnegative end points that correspond to lower and upper flow limits. This paper proposes and analyzes a distributed algorithm for obtaining admissible and balanced flows, i.e., flows that are within the given intervals at each edge and are balanced (the total in-flow equals the total out-flow) at each node. The algorithm can also be viewed as a distributed method for obtaining a set of weights that balance a weighted digraph for the case when there are lower and upper limit constraints on the edge weights. The proposed iterative algorithm assumes that communication among pairs of nodes that are interconnected is bidirectional (i.e., the communication topology is captured by the undirected graph that corresponds to the network digraph), and allows the nodes to asymptotically (with geometric rate) reach a set of balanced feasible flows, as long as the (necessary and sufficient) circulation conditions on the given digraph, with the given flow/weight interval constraints on each edge, are satisfied. We also provide a methodology that can be used by the nodes to asymptotically determine, in a distributed manner, when the circulation conditions are not satisfied (thus, making the problem infeasible). Finally, we provide several examples and simulation studies to highlight the role of various parameters involved in the proposed distributed algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we study a class of robust design problems with polynomial dependence on the uncertainty. One of the main motivations for considering these problems comes from robust controller design, where one often encounters systems that depend polynomially on the uncertain parameters. This paper can be seen as integrated in the emerging area of probabilistic robustness, where a probabilistic relaxation of the original robust problem is adopted, thus requiring the satisfaction of the constraints not for all possible values of the uncertainty, but for most of them. Different from the randomized approach for tackling probabilistic relaxations, which is only guaranteed to provide soft bounds on the probability of satisfaction, we present a deterministic approach based on the novel concept of kinship function introduced in this paper. This allows the development of an original framework, which leads to easily computable deterministic convex relaxations of the probabilistic problem. In particular, optimal polynomial kinship functions are introduced, which can be computed a priori and once for all and provide the “best convex bound” on the probability of constraint violation. More importantly, it is proven that the solution of the relaxed problem converges to that of the original robust optimization problem as the degree of the optimal polynomial kinship function increases. Furthermore, by relying on quadrature formulas for computation of integrals of polynomials, it is shown that the computational complexity of the proposed approach is polynomial in the number of uncertain parameters. Finally, unlike other deterministic approaches to robust polynomial optimization, the number of variables in the ensuing optimization problem is not increased by the proposed approximation. An important feature of this approach is that a significant amount of the computational burden is shifted to a one-time offline computation whose results can be stored and provided to the end-user. [ABSTRACT FROM PUBLISHER]

In this paper, a neural-dynamic optimization-based nonlinear model predictive control (NMPC) is developed for the multiple nonholonomic mobile robots formation. First, a model-based monocular vision method is developed to obtain the location information of the leader. Then, a separation-bearing-orientation scheme (SBOS) control strategy is proposed. During the formation motion, the leader robot is controlled to track the desired trajectory and the desired leader–follower relationship can be maintained through the SBOS method. Finally, the model predictive control (MPC) is utilized to maintain the desired leader–follower relationship. To solve the MPC generated constrained quadratic programming problem, the neural-dynamic optimization approach is used to search for the global optimal solution. Compared to other existing formation control approaches, the proposed solution is that the NMPC scheme exploit prime-dual neural network for online optimization. Finally, by using several actual mobile robots, the effectiveness of the proposed approach has been verified through the experimental studies. [ABSTRACT FROM AUTHOR]

A recently introduced latent feature learning technique for time-varying dynamic phenomena analysis is the so-called slow feature analysis (SFA). SFA is a deterministic component analysis technique for multidimensional sequences that, by minimizing the variance of the first-order time derivative approximation of the latent variables, finds uncorrelated projections that extract slowly varying features ordered by their temporal consistency and constancy. In this paper, we propose a number of extensions in both the deterministic and the probabilistic SFA optimization frameworks. In particular, we derive a novel deterministic SFA algorithm that is able to identify linear projections that extract the common slowest varying features of two or more sequences. In addition, we propose an expectation maximization (EM) algorithm to perform inference in a probabilistic formulation of SFA and similarly extend it in order to handle two and more time-varying data sequences. Moreover, we demonstrate that the probabilistic SFA (EM-SFA) algorithm that discovers the common slowest varying latent space of multiple sequences can be combined with dynamic time warping techniques for robust sequence time-alignment. The proposed SFA algorithms were applied for facial behavior analysis, demonstrating their usefulness and appropriateness for this task. [ABSTRACT FROM AUTHOR]

This paper introduces a new approach for the synthesis of Markov chains with density safety constraints. Specific safety constraints considered are: (i) Density upper bound constraints; (ii) Bound on the rate of change of density. The proposed approach is based on a new mathematical result that formulates these constraints as linear, and hence convex, inequality constraints. The main contribution of the paper is that the new convex constraints are proved to be equivalent, necessary and sufficient, to the original constraints. This result enables the formulation of the Markov chain synthesis problem as a Linear Matrix Inequality (LMI) optimization problem with additional constraints on the steady state probability distribution, ergodicity, and feasible state transitions. The LMI formulation presents a necessary and sufficient design condition for reversible Markov chains, that is, it is not conservative. When the reversibility is not required, the LMI condition is only sufficient due to the ergodicity constraint. Since LMI problems can be solved to global optimality in polynomial time by using Interior Point Method algorithms, the proposed LMI-based design approach is numerically tractable. [ABSTRACT FROM PUBLISHER]

Novelty detection models aim to find the minimum volume set covering a given probability mass. This paper proposes a robust single-class support vector machine (SSVM) for novelty detection, which is mainly based on the worst case conditional value-at-risk minimization. By assuming that every input is subject to an uncertainty with a specified symmetric support, this robust formulation results in a maximization term that is similar to the regularization term in the classical SSVM. When the uncertainty set is \ell 1 -norm, $\ell _\infty $ -norm or box, its training can be reformulated to a linear program; while the uncertainty set is \ell _{2} -norm or ellipsoidal, its training is a tractable second-order cone program. The proposed method has a nice consistent statistical property. As the training size goes to infinity, the estimated normal region converges to the true provided that the magnitude of the uncertainty set decreases in a systematic way. The experimental results on three data sets clearly demonstrate its superiority over three benchmark models. [ABSTRACT FROM AUTHOR]

Motivated by recent approaches to switched linear system identification based on sparse optimization, the paper deals with the recovery of sparse solutions of underdetermined systems of linear equations. More precisely, we focus on the associated convex relaxation where the \ell1-norm of the vector of variables is minimized and propose a new iteratively reweighted scheme in order to improve the conditions under which this relaxation provides the sparsest solution. We prove the convergence of the new scheme and derive sufficient conditions for the convergence towards the sparsest solution. Experiments show that the new scheme significantly improves upon the previous approaches for compressive sensing. Then, these results are applied to switched system identification. [ABSTRACT FROM AUTHOR]

This paper considers the problem of estimating the position of a scene point from its image projections onto generalized cameras, i.e., cameras that can be modeled by a spherical projection followed by a perspective one. The sought estimate is defined through a geometric criterion, specifically the minimization of the angles between the projections on the sphere of the available image points and the corresponding projections of the estimate. A solution based on convex optimization with linear matrix inequalities (LMIs) is proposed for addressing this problem, which provides a candidate of the sought estimate. Moreover, a condition is provided for establishing exactness of the found candidate, i.e., establishing whether the found candidate is a minimizer of the considered geometric criterion. [ABSTRACT FROM AUTHOR]

This paper deals with the performance optimization of resource allocation systems with the aim of maximizing the system's throughput under a given budget for acquiring resources. Resources are assumed to be renewable, i.e., they are not consumed by the operations and become available again after they have been released. The systems under consideration are modeled by a subclass of timed Petri nets called deterministic timed weighted marked graphs. In addition, we take into account infinite server semantics, i.e., the degree of self-concurrency of each transition is infinite. We propose an approach that provides an optimal solution, but has a high computational cost. For this reason, we also present two different approaches that can find suboptimal solutions with a reduced computational cost. The performances of the proposed approaches are compared by means of numerical simulations. [ABSTRACT FROM AUTHOR]

This paper proposes a novel method, called robust latent subspace learning (RLSL), for image classification. We formulate an RLSL problem as a joint optimization problem over both the latent SL and classification model parameter predication, which simultaneously minimizes: 1) the regression loss between the learned data representation and objective outputs and 2) the reconstruction error between the learned data representation and original inputs. The latent subspace can be used as a bridge that is expected to seamlessly connect the origin visual features and their class labels and hence improve the overall prediction performance. RLSL combines feature learning with classification so that the learned data representation in the latent subspace is more discriminative for classification. To learn a robust latent subspace, we use a sparse item to compensate error, which helps suppress the interference of noise via weakening its response during regression. An efficient optimization algorithm is designed to solve the proposed optimization problem. To validate the effectiveness of the proposed RLSL method, we conduct experiments on diverse databases and encouraging recognition results are achieved compared with many state-of-the-arts methods. [ABSTRACT FROM AUTHOR]

In image classification, feature combination is often used to combine the merits of multiple complementary features and improve the classification accuracy compared with one single feature. Existing feature combination algorithms, e.g., multiple kernel learning, usually determine the weights of features based on the optimization with respect to some classifier-dependent objective function. These algorithms are often computationally expensive, and in some cases are found to perform no better than simple baselines. In this paper, we solve the feature combination problem from a totally different perspective. Our algorithm is based on the simple idea of combining only base kernels suitable to be combined. Since the very aim of feature combination is to obtain the highest possible classification accuracy, we measure the combination suitableness of two base kernels by the maximum possible cross-validation accuracy of their combined kernel. By regarding the pairwise suitableness as the kernel adjacency, we obtain a weighted graph of all base kernels and find that the base kernels suitable to be combined correspond to a cluster in the graph. We then use the dominant sets algorithm to find the cluster and determine the weights of base kernels automatically. In this way, we transform the kernel combination problem into a clustering one. Our algorithm can be implemented in parallel easily and the running time can be adjusted based on available memory to a large extent. In experiments on several data sets, our algorithm generates comparable classification accuracy with the state of the art. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of distributed control for network systems aiming to achieve optimal steady-state performance. Motivated by recent research on reengineering cyber-physical systems, such as power systems and the Internet, we propose a two-step control retrofit procedure. First, we reformulate the dynamical system as an optimization algorithm to solve a certain optimization problem. Second, we combine a predefined steady-state optimization problem and the reformulated problem to systematically (re)design the control. As a result, the system automatically tracks the optimal solution of the predefined steady-state optimization problem and the control scheme can be implemented in a distributed and closed-loop manner. In order to investigate how general this framework is, we establish necessary and sufficient conditions under which a linear dynamical system can be viewed as an optimization algorithm. These conditions are characterized using properties of system matrices and related linear matrix inequalities. A practical example of frequency control in power systems shows the effectiveness of the proposed framework. [ABSTRACT FROM PUBLISHER]

This paper studies stability- and symmetry-preserving $H^2$ optimal model reduction problems of linear systems, which include linear gradient systems as a special case. The problem is formulated as a nonlinear optimization problem on the product manifold of the manifold of symmetric positive-definite matrices and two Euclidean spaces. To solve the problem by using the trust-region method, the gradient and Hessian of the objective function are derived. Furthermore, it is shown that if we restrict our systems to gradient systems, the gradient and Hessian can be obtained more efficiently. More concretely, by symmetry, we can reduce linear matrix equations to be solved. In addition, by a simple example, we show that the solutions to our problem and a similar problem in some literature works are not unique, and the solution sets of both problems do not contain each other in general. Also, it is revealed that the attained optimal values do not coincide. Numerical experiments show that the proposed method gives a reduced system with the same structure with the original system although the balanced truncation method does not. [ABSTRACT FROM PUBLISHER]

This paper analyzes data-based online nonlinear extremum-seeker (DONE), an online optimization algorithm that iteratively minimizes an unknown function based on costly and noisy measurements. The algorithm maintains a surrogate of the unknown function in the form of a random Fourier expansion. The surrogate is updated whenever a new measurement is available, and then used to determine the next measurement point. The algorithm is comparable with Bayesian optimization algorithms, but its computational complexity per iteration does not depend on the number of measurements. We derive several theoretical results that provide insight on how the hyperparameters of the algorithm should be chosen. The algorithm is compared with a Bayesian optimization algorithm for an analytic benchmark problem and three applications, namely, optical coherence tomography, optical beam-forming network tuning, and robot arm control. It is found that the DONE algorithm is significantly faster than Bayesian optimization in the discussed problems while achieving a similar or better performance. [ABSTRACT FROM AUTHOR]

Hashing-based semantic similarity search is becoming increasingly important for building large-scale content-based retrieval system. The state-of-the-art supervised hashing techniques use flexible two-step strategy to learn hash functions. The first step learns binary codes for training data by solving binary optimization problems with millions of variables, thus usually requiring intensive computations. Despite simplicity and efficiency, locality-sensitive hashing (LSH) has never been recognized as a good way to generate such codes due to its poor performance in traditional approximate neighbor search. We claim in this paper that the true merit of LSH lies in transforming the semantic labels to obtain the binary codes, resulting in an effective and efficient two-step hashing framework. Specifically, we developed the locality-sensitive two-step hashing (LS-TSH) that generates the binary codes through LSH rather than any complex optimization technique. Theoretically, with proper assumption, LS-TSH is actually a useful LSH scheme, so that it preserves the label-based semantic similarity and possesses sublinear query complexity for hash lookup. Experimentally, LS-TSH could obtain comparable retrieval accuracy with state of the arts with two to three orders of magnitudes faster training speed. [ABSTRACT FROM AUTHOR]

In this paper, the decentralized partially observable Markov decision process (Dec-POMDP) systems with discrete state and action spaces are studied from a gradient point of view. Dec-POMDPs have recently emerged as a promising approach to optimizing multiagent decision making in the partially observable stochastic environment. However, the decentralized nature of the Dec-POMDP framework results in a lack of shared belief state, which makes the decision maker impossible to estimate the system state based on local information. In contrast to the belief-based policy, this paper focuses on optimizing the decentralized observation-based policy, which is easily to be applied and does not have the sharing problem. By analyzing the gradient of the objective function, we have developed a centralized stochastic gradient policy iteration algorithm to find the optimal policy on the basis of gradient estimates from a single sample path. This algorithm does not need any specific assumption and can be applied to most practical Dec-POMDP problems. One numerical example is provided to demonstrate the effectiveness of the algorithm. [ABSTRACT FROM AUTHOR]

Several sparsity-constrained algorithms, such as orthogonal matching pursuit (OMP) or the Frank–Wolfe (FW) algorithm, with sparsity constraints work by iteratively selecting a novel atom to add to the current nonzero set of variables. This selection step is usually performed by computing the gradient and then by looking for the gradient component with maximal absolute entry. This step can be computationally expensive especially for large-scale and high-dimensional data. In this paper, we aim at accelerating these sparsity-constrained optimization algorithms by exploiting the key observation that, for these algorithms to work, one only needs the coordinate of the gradient’s top entry. Hence, we introduce algorithms based on greedy methods and randomization approaches that aim at cheaply estimating the gradient and its top entry. Another of our contribution is to cast the problem of finding the best gradient entry as a best-arm identification in a multiarmed bandit problem. Owing to this novel insight, we are able to provide a bandit-based algorithm that directly estimates the top entry in a very efficient way. Theoretical observations stating that the resulting inexact FW or OMP algorithms act, with high probability, similar to their exact versions are also given. We have carried out several experiments showing that the greedy deterministic and the bandit approaches we propose can achieve an acceleration of an order of magnitude while being as efficient as the exact gradient when used in algorithms, such as OMP, FW, or CoSaMP. [ABSTRACT FROM PUBLISHER]

To detect the mental task of interest, spatial filtering has been widely used to enhance the spatial resolution of electroencephalography (EEG). However, the effectiveness of spatial filtering is undermined due to the significant nonstationarity of EEG. Based on regularization, most of the conventional stationary spatial filter design methods address the nonstationarity at the cost of the interclass discrimination. Moreover, spatial filter optimization is inconsistent with feature extraction when EEG covariance matrices could not be jointly diagonalized due to the regularization. In this paper, we propose a novel framework for a spatial filter design. With Fisher’s ratio in feature space directly used as the objective function, the spatial filter optimization is unified with feature extraction. Given its ratio form, the selection of the regularization parameter could be avoided. We evaluate the proposed method on a binary motor imagery data set of 16 subjects, who performed the calibration and test sessions on different days. The experimental results show that the proposed method yields improvement in classification performance for both single broadband and filter bank settings compared with conventional nonunified methods. We also provide a systematic attempt to compare different objective functions in modeling data nonstationarity with simulation studies. [ABSTRACT FROM PUBLISHER]

This paper presents a general framework for the analysis and design of a class of model-free, robust, and efficient sampled-data-based algorithms for extremization and learning in continuous-time nonlinear systems that generate response maps with an optimal operational set. In particular, we consider plants described by differential inclusions, interconnected in a sampled-data setting with a robust learning algorithm characterized by a constrained difference inclusion. In contrast to standard sampled-data-based approaches, where the learning dynamics are updated after a fixed sufficiently long sampling time has passed, we design a robust dynamic event-based mechanism that triggers the control action as soon as the rate of change of the output of the plant is sufficiently small. By using this event-based update rule, a significant improvement in the convergence time of the closed-loop system can be achieved. Using the framework of set-valued hybrid dynamical systems, we establish for the closed-loop system the existence of a uniformly asymptotically stable compact set, which, by an appropriate tuning of the control parameters, can be made arbitrarily close to the optimal operational set. Our results generalize existing results for periodic sampled-data extremum seeking, and can be used to solve model-free multivariable smooth/nonsmooth constrained optimization problems, as well as learning problems in game theoretical scenarios. [ABSTRACT FROM AUTHOR]

Nonnegative matrix factorization (NMF) is an advanced method for nonnegative feature extraction, with widespread applications. However, the NMF solution often entails to solve a global optimization problem with a nonconvex objective function and nonnegativity constraints. This paper presents a collective neurodynamic optimization (CNO) approach to this challenging problem. The proposed collective neurodynamic system consists of a population of recurrent neural networks (RNNs) at the lower level and a particle swarm optimization (PSO) algorithm with wavelet mutation at the upper level. The RNNs act as search agents carrying out precise local searches according to their neurodynamics and initial conditions. The PSO algorithm coordinates and guides the RNNs with updated initial states toward global optimal solution(s). A wavelet mutation operator is added to enhance PSO exploration diversity. Through iterative interaction and improvement of the locally best solutions of RNNs and global best positions of the whole population, the population-based neurodynamic systems are almost sure able to achieve the global optimality for the NMF problem. It is proved that the convergence of the group-best state to the global optimal solution with probability one. The experimental results substantiate the efficacy and superiority of the CNO approach to bound-constrained global optimization with several benchmark nonconvex functions and NMF-based clustering with benchmark data sets in comparison with the state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]