Search

In this paper, a two-layer recurrent neural network is proposed to solve the nonsmooth convex optimization problem subject to convex inequality and linear equality constraints. Compared with existing neural network models, the proposed neural network has a low model complexity and avoids penalty parameters. It is proved that from any initial point, the state of the proposed neural network reaches the equality feasible region in finite time and stays there thereafter. Moreover, the state is unique if the initial point lies in the equality feasible region. The equilibrium point set of the proposed neural network is proved to be equivalent to the Karush–Kuhn–Tucker optimality set of the original optimization problem. It is further proved that the equilibrium point of the proposed neural network is stable in the sense of Lyapunov. Moreover, from any initial point, the state is proved to be convergent to an equilibrium point of the proposed neural network. Finally, as applications, the proposed neural network is used to solve nonlinear convex programming with linear constraints and L1 -norm minimization problems. [ABSTRACT FROM AUTHOR]

Setting retail electricity prices is one of the significant strategies for energy management of multi-microgrid (MMG) systems integrated with renewable energy. Nevertheless, the need of privacy preservation, the uncertainties of renewable energy and loads, as well as the time-varying scenarios, bring challenges for pricing problems. In this paper, a two-level pricing framework is proposed based on interval predictions and model-free reinforcement learning to address these challenges. In particular, at the higher level, the distribution system operator (DSO) is viewed as an agent, which sets retail electricity prices without detailed user information for privacy protection to maximize the total revenue from selling energy with reinforcement learning. For time-varying scenarios with intermittent photovoltaic power generation and diverse loads, a differentiable trust region layer is considered in reinforcement learning to improve the robustness of the policy updating process. While at the lower level, operators in microgrids solve three-phase unbalanced optimal power flow (OPF) problems to minimize generation cost and network power loss. Additionally, to deal with the challenges from the uncertainties of renewable power generation and user loads, interval predictions are chosen to quantify prediction errors and improve the flexibility of pricing policies. Finally, a set of experiments are conducted to validate the effectiveness of the proposed method for pricing problems in MMG systems. [ABSTRACT FROM AUTHOR]

In this paper, we introduce a new classification algorithm called the optimization of distribution differences (ODD). The algorithm aims to find a transformation from the feature space to a new space where the instances in the same class are as close as possible to one another, whereas the gravity centers of these classes are as far as possible from one another. This aim is formulated as a multiobjective optimization problem that is solved by a hybrid of an evolutionary strategy and the quasi-Newton method. The choice of the transformation function is flexible and could be any continuous space function. We experiment with a linear and a nonlinear transformation in this paper. We show that the algorithm can outperform eight other classification methods, namely naive Bayes, support vector machines, linear discriminant analysis, multilayer perceptrons, decision trees, and $k$ -nearest neighbors, and two recently proposed classification methods, in 12 standard classification data sets. Our results show that the method is less sensitive to the imbalanced number of instances compared with these methods. We also show that ODD maintains its performance better than other classification methods in these data sets and hence offers a better generalization ability. [ABSTRACT FROM AUTHOR]

Conventional design and optimization of passive compensation network (PCN) for inductive power transfer (IPT) system are based on specific topologies. The demerits of this design method are: i) The topology is mostly chosen by experience; ii) The design parameters are not multi-objective optimal. Aiming at these issues, this paper proposes an optimal PCN design scheme based on evolutionary algorithm (EA) to synchronously optimize the topology and parameters of PCN for IPT system. Firstly, a unified mathematical model of the PCN is presented and derived by transmission matrix. Then, according to the mathematical model, the multi-objective functions (such as output fluctuation and efficiency) as well as the constraints (such as load and coupling coefficient) for the optimal PCN design are established. The EA based multi-objective optimal PCN design algorithm is further constructed. Six optimal results are obtained using the algorithm, and one optimized PCN having minimum output current fluctuation and high-efficiency is chosen to validate the effectiveness of the proposed design scheme in experiment. For the given IPT system with the optimized PCN, the maximum fluctuation of output current is no more than 11%, within 200% of load variation and about 77% of coupling variation. [ABSTRACT FROM AUTHOR]

Matrix factorization (MF) has been widely used to discover the low-rank structure and to predict the missing entries of data matrix. In many real-world learning systems, the data matrix can be very high dimensional but sparse. This poses an imbalanced learning problem since the scale of missing entries is usually much larger than that of the observed entries, but they cannot be ignored due to the valuable negative signal. For efficiency concern, existing work typically applies a uniform weight on missing entries to allow a fast learning algorithm. However, this simplification will decrease modeling fidelity, resulting in suboptimal performance for downstream applications. In this paper, we weight the missing data nonuniformly, and more generically, we allow any weighting strategy on the missing data. To address the efficiency challenge, we propose a fast learning method, for which the time complexity is determined by the number of observed entries in the data matrix rather than the matrix size. The key idea is twofold: 1) we apply truncated singular value decomposition on the weight matrix to get a more compact representation of the weights and 2) we learn MF parameters with elementwise alternating least squares (eALS) and memorize the key intermediate variables to avoid repeating computations that are unnecessary. We conduct extensive experiments on two recommendation benchmarks, demonstrating the correctness, efficiency, and effectiveness of our fast eALS method. [ABSTRACT FROM AUTHOR]

This paper considers the quadratic control problem of discrete-time Markov jump linear systems with constraints on the norm of the state and control variables. We assume that the Markov chain parameter is not available, and instead, there is a detector, which emits signals providing information on this parameter. It is desired to derive a feedback linear control using the information provided by this detector in order to stochastically stabilize the closed-loop system, satisfy the constraints whenever the initial conditions belong to an invariant set, and minimize an upper bound for the quadratic cost. We show that a linear matrix inequality (LMI) optimization problem can be formulated in order to obtain a solution for this problem. Two other related problems, one for minimizing the guaranteed quadratic cost considering fixed initial conditions and the other for maximizing an estimate of the domain of an invariant set, can also be formulated using our LMI approach. This paper is concluded with some numerical simulations. [ABSTRACT FROM AUTHOR]

The memristor-based neuromorphic computing system (NCS) with emerging storage and computing integration architecture has drawn extensive attention. Because of the unique nonvolatility and programmability, the memristor is an ideal nano-device to realize neural synapses in VLSI circuit implementation of neural networks. However, in the hardware implementation, the performance of the memristive neural network is always affected by quantization error, writing error, and conductance drift, which seriously hinders its applications in practice. In this paper, a novel weight optimization scheme combining quantization and Bayesian inference is proposed to alleviate this problem. Specifically, the weight deviation in the memristive neural network is transformed into the weight uncertainty in the Bayesian neural network, which can make the network insensitive to unexpected weight changes. A quantization regularization term is designed and utilized during the training process of the Bayesian neural network, reducing the quantization error and improving the robustness of the network. Furthermore, a partial training method is raised to extend the applicability of the proposed scheme in large-scale neural networks. Finally, the experiments on a Multilayer Perceptron and LeNet demonstrate that the proposed weight optimization scheme can significantly enhance the robustness of memristive neural networks. [ABSTRACT FROM AUTHOR]

The articles in this special issue focus on security and privacy of distributed algorithms and network systems. The integration of computation, communication, and control technologies has led to the widespread emergence of large scale engineering systems. These network systems are deployed in numerous fields, including electric and smart grids, transportation and smart cities, health-care and manufacturing, and so forth. Due to the rapid growth in number of geographically deployed units, such as sensors, computers and controllers, traditional centralized control and optimization algorithms may not be efficient, robust, or even applicable at all. Instead, distributed control and optimization algorithms offer a promising and desirable approach to operate and guarantee the well functioning of network systems. The aim of this special issue is to provide a window into the recent developments of the fundamentals and applications of security and privacy of distributed algorithms in network systems. [ABSTRACT FROM AUTHOR]

The computation-intensive circuit simulation makes the analog circuit sizing challenging for large-scale/complicated analog/RF circuits. A Bayesian optimization approach has been proposed recently for the optimization problems involving the evaluations of black-box functions with high computational cost in either objective functions or constraints. In this paper, we propose a weighted expected improvement-based Bayesian optimization approach for automated analog circuit sizing. Gaussian processes (GP) are used as the online surrogate models for circuit performances. Expected improvement is selected as the acquisition function to balance the exploration and exploitation during the optimization procedure. The expected improvement is weighted by the probability of satisfying the constraints. In this paper, we propose a complete Bayesian optimization framework for the optimization of analog circuits with constraints for the first time. The existing GP model-based optimization methods for analog circuits take the GP models as either offline models or as assistance for the evolutionary algorithms. We also extend the Bayesian optimization algorithm to handle multi-objective optimization problems. Compared with the state-of-the-art approaches listed in this paper, the proposed Bayesian optimization method achieves better optimization results with significantly less number of simulations. [ABSTRACT FROM PUBLISHER]

Compressed sensing (CS) has been proposed to reduce operating cost (e.g., energy requirements) of acquisition devices by leveraging its capability of sampling and compressing an input signal at the same time. This paper aims at increasing CS performance (i.e., either achieving a better compression or allowing a higher signal reconstruction quality) and proposes two novel methods. Our first approach (Nearly Orthogonal CS) is based on a geometric constraint enforcing diversity between compressed measurements, while the second one (Maximum-Energy CS) on a heuristic screening of candidate measurements that acts as a run-time self-adapted optimization technique. Intensive simulation results show that the proposed approaches have different applications, and ensure an appreciable performance boost with respect to the state-of-the-art. [ABSTRACT FROM AUTHOR]

Self-organizing map (SOM)-based motif mining, despite being a promising approach for problem solving, mostly fails to offer a consistent interpretation of clusters with respect to the mixed composition of signal and noise in the nodes. The main reason behind this shortcoming comes from the similarity metrics used in data assignment, specially designed with the biological interpretation for this domain, which are not meant to consider the inevitable noise mixture in the clusters. This limits the explicability of the majority of clusters that are supposedly noise dominated, degrading the overall system clarity in motif discovery. This paper aims to improve the explicability aspect of learning process by introducing a composite similarity function (CSF) that is specially designed for the k -mer-to-cluster similarity measure with respect to the degree of motif properties and embedded noise in the cluster. Our proposed motif finding algorithm in this paper is built on our previous work robust elicitation algorithms for discovering (READ) [1] and termed READ Deoxyribonucleic acid motifs using CSFs (READ _{\rm csf} ), which performs slightly better than READ and shows some remarkable improvements over SOM-based SOMBRERO and SOMEA tools in terms of F$ -measure on the testing data sets. A real data set containing multiple motifs is used to explore the potential of the READ _{\rm csf} for more challenging biological data mining tasks. Visual comparisons with the verified logos extracted from JASPAR database demonstrate that our algorithm is promising to discover multiple motifs simultaneously. [ABSTRACT FROM PUBLISHER]

The main focus of this paper is the presentation of reliable methods for the determination of the optimum coloring of a graph, commonly known in the literature as vertex coloring problem. It has been shown that networks of capacitively coupled oscillators can be used to solve vertex coloring problems. In this paper we address the negative impact of an unbalanced number of couplings for the oscillators on the performance of the network and compensate for this non-uniform coupling structure by an adjustment in the network itself. The negative effect of the memristor device-to-device variability of the NbOx memristor on the array functionality will be investigated and reduced via an adaptation of the memristor operating point. The main improvement in network performance is achieved by setting up a control procedure allowing the network to bypass the local solutions and converge to the global one. Two strategies inspired by global optimization algorithms will be proposed to allow the network to overcome sub-optimal solutions, and find the solution corresponding to the absolute minimum of a performance measure function of the vertex coloring problem. [ABSTRACT FROM AUTHOR]

Using reinforcement learning (RL) algorithm to optimize guidance law can address non-idealities in complex environment. However, the optimization is difficult due to huge state-action space, unstable training, and high requirements on expertise. In this paper, the constrained guidance policy of a neural guidance system is optimized using improved RL algorithm, which is motivated by the idea of traditional model-based guidance method. A novel optimization objective with minimum overload regularization is developed to restrain the guidance policy directly from generating redundant missile maneuver. Moreover, a bi-level curriculum learning is designed to facilitate the policy optimization. Experiment results show that the proposed minimum overload regularization can reduce the vertical overloads of missile significantly, and the bi-level curriculum learning can further accelerate the optimization of guidance policy. [ABSTRACT FROM AUTHOR]

We study in this paper, a class of constrained linear-quadratic (LQ) optimal control problem formulations for the scalar-state stochastic system with multiplicative noise, which has various applications, especially in the financial risk management. The linear constraint on both the control and state variables considered in our model destroys the elegant structure of the conventional LQ formulation and has blocked the derivation of an explicit control policy so far in the literature. We successfully derive in this paper, the analytical control policy for such a class of problems by utilizing the state separation property induced from its structure. We reveal that the optimal control policy is a piecewise affine function of the state and can be computed offline efficiently by solving two coupled Riccati equations. Under some mild conditions, we also obtain the stationary control policy for an infinite time horizon. We demonstrate the implementation of our method via some illustrative examples and show how to calibrate our model to solve dynamic constrained portfolio optimization problems. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) are widely employed in modern computer vision algorithms, where the input image is convolved iteratively by many filters to extract the knowledge behind it. However, while the depth of convolutional layers gets deeper and deeper in recent years, the enormous computational complexity makes it difficult to be deployed on embedded systems with limited hardware resources. In this paper, inspired by rate-distortion optimization in image and video coding, we propose a computation-performance optimization (CPO) method to remove the redundant convolution filters in a CNN with performance constraints. To prove the effectiveness of the proposed method, CPO is applied to the networks for image super-resolution and image classification. Under almost the same PSNR drop and accuracy drop for performance evaluation in these two tasks, we can achieve the best parameter and computation reduction when compared with previous works. [ABSTRACT FROM AUTHOR]

In this paper, the first dual-band sub-sampling receiver front end with sampling frequency optimization to meet the ultimate receiver error vector magnitude (EVM) of −40 dB over wide input power range of 19 dB is proposed. A systematic sub-sampling receiver chain EVM optimization with respect to major system-level impairments, such as noise folding, sampling frequency, IQ mismatches, phase noise of the sub-sampling clock, and unit capacitor value realizable at the decimation filter, is presented. The proposed dual-band sub-sampling receiver has a 26–41 dB continuously tunable gain for 2.4 GHz and 26–38.5 dB for the 5 GHz WLAN band. Continuously tunable gain ensures the ultimate receiver EVM performance over wider input power levels. In addition, the 5 GHz band is continuously tunable from 4.5 to 5.7 GHz. An active balun feedback low-noise amplifier followed by a sub-sampling down-conversion mixer is implemented to down-convert both WLAN bands to an intermediate frequency in the range from 445 to 538 MHz. Sub-sampling frequency optimization proposed in this paper down-converts both WLAN bands with the sampling frequency from 1.78 to 2.15 GHz to reach the target EVM. Additionally, a switched capacitor decimation filter running at 90 MHz is implemented to provide dual functionalities of down-conversion to baseband and band selection. A test-chip is implemented in a 1.2 V 65-nm CMOS technology. The proposed dual-band sub-sampling receiver occupies a total active area of 0.72 mm2 and has a total power dissipation of 55.6 mW. The overall receiver chain shows a noise figure of 11.5 dB at the highest gain and an IIP3 of −8 dBm at the lowest gain. [ABSTRACT FROM AUTHOR]

Visual feature detection has been widely used in many computer vision applications, with increasing concern on feature robustness, processing speed, and power efficiency. In comparison with popular feature detection algorithms, affine-SIFT achieves the strongest robustness on the image illumination, image rotation, and image scale transformation, but exhibits extreme high computation complexity. To improve its computing efficiency, this work first proposes three hardware optimization methods to address three main performance bottlenecks. The first method is the reverse affine-based pipelined computing with optimized memory accessing. The second method is about stream processing with full parallel Gaussian pyramid. The third method is the rotation invariant binary pattern based feature vector generation. Then by incorporating these three optimization methods, this paper designs a high-efficient pipelined and parallel hardware architecture with optimized parallel memory accessing. Postlayout simulations using TSMC 65-nm 1P9M low power process show that this work achieves a processing speed of 97 fps at 1080p (1000 feature points per frame on average) under 200 MHz, with power consumption at 300 mW. In comparison, its computing efficiency (1005.6K pixels/s at 1 MHz) and power efficiency (670.5K pixels/s at 1 mW) are higher than state-of-the-art works and it is more promising for broad vision applications especially the embedded vision and mobile vision applications. [ABSTRACT FROM AUTHOR]

In a discrete time nonhomogeneous Markov chain (TNHMC), the states spaces, transition probabilities, and reward functions at different times may be different. In this paper, with the confluencity previously introduced, we show that the states of a TNHMC can be classified into the branching states and a number of classes of confluent states (versus the transient and recurrent states in the time homogeneous case). The optimization of average reward in TNHMC's consisting of a single confluent class (uni-chain) have been addressed in a previous paper by the author. In this paper, we show that with confluencity and the state classification and under some bound conditions, we can obtain the necessary and sufficient conditions for optimal policies of the average reward of TNHMCs consisting of multiple confluent classes (multi-chains). Just like in the uni-chain TNHMC case, the sufficient condition does not need to hold in any “zero frequently visited” time sequence. This “under-selectivity” makes the problem not amenable to dynamic programming. A direct comparison based approach is used to prove the results. The results enhance our understanding of state classification and performance optimization with the notion of confluencity. [ABSTRACT FROM AUTHOR]

This paper deals with the frequency design of lossless passive electronic filters under magnitude constraints. With the huge increase in design complexity for mobile applications, new systematic and efficient methods are required. This paper focuses on the direct synthesis approach, an historical design approach that has not been recently updated. It consists in directly synthesizing the $LC$ values of a pre-specified circuit until the spectral mask is satisfied. While beneficial in practice, this approach typically leads to an important computational time and requires an initial guess to reduce it. Based on recent developments of the System and Control community, that led to efficient methods for system design, the direct synthesis approach is revisited. To achieve this, the port-Hamiltonian Differential Algebraic Equation (pHDAE) representation, that particularly fits the design problem, is introduced. A synthesis method is then developed, leading to solve an optimization problem of moderate complexity. For particular cases, this complexity happens to be remarkably low. Based on this observation, a second method reveals how to obtain such complexity for the more general case, using an original combination between the pHDAE and the LFT representations. Finally, a numerical example shows the validity and illustrates the benefits of this work. [ABSTRACT FROM AUTHOR]

Residual block is a very common component in recent state-of-the art CNNs such as EfficientNet/EfficientDet. Shortcut data accounts for nearly 40% of feature-maps access in ResNet152. Most of the previous DNN compilers/accelerators ignore the shortcut data optimization. This paper presents ShortcutFusion, an optimization tool for FPGA-based accelerator with a reuse-aware static memory allocation for shortcut data, to maximize on-chip data reuse given resource constraints. From TensorFlow DNN models, the proposed design generates instruction sets for a group of nodes which uses an optimized data reuse for each residual block. The accelerator design implemented on the Xilinx KCU1500 FPGA card $2.8\times $ faster and $9.9\times $ more power efficient than NVIDIA RTX 2080 Ti for $256\times 256$ input size. Compared to the result from baseline, in which the weights/inputs/outputs are accessed from the off-chip memory exactly once per each layer, ShortcutFusion reduces the DRAM access by 47.8-84.8% for RetinaNet, Yolov3, ResNet152, and EfficientNet. Given a similar buffer size to ShortcutMining, which also “mine” the shortcut data in hardware, the proposed work reduces off-chip access for feature-maps $5.27\times $ while accessing weight from off-chip memory exactly once. [ABSTRACT FROM AUTHOR]

Quantization is a promising technique to compress the size of Convolutional Neural Network (CNN) models. Recently, various precision-scalable designs have been presented to reduce the computational complexity in CNNs. However, most of them adopt straightforward calculation scheme to implement the CNN, which causes high bandwidth requirement and low hardware utilization efficiency. This paper proposes a new precision-scalable architecture which can fully reduce the computational complexity in CNN inference and meanwhile has a finely simplified calculation scheme. Based on the proposed scheme, a well-optimized multiplier called Compositional Processing Element (C-PE) is devised. Compared with the previous multipliers, the new C-PE requires less area and power. Furthermore, two levels of optimization are introduced to the design to relieve the bandwidth problem and increase the hardware utilization efficiency. Implemented under the TSMC 90nm CMOS technology, the whole design achieves 6-68.1 fps in various precisions on VGG16 benchmark and a 49.8TOPS/W energy efficiency at 500MHz when scaled to 28nm, which is much better than previous precision-scalable ones. [ABSTRACT FROM AUTHOR]

To better understand the correlation between network topological features and the robustness of network controllability in a general setting, this paper suggests a practical approach to searching for optimal network topologies with given numbers of nodes and edges. Since theoretical analysis seems impossible at least in the present time, exhaustive search based on optimization techniques is employed, firstly for a group of small-sized networks that are realistically workable, where exhaustive means 1) all possible network structures with the given numbers of nodes and edges are computed and compared, and 2) all possible node-removal sequences are considered. A main contribution of this paper is the observation of an empirical necessary condition (ENC) from the results of exhaustive search, which shrinks the search space to quickly find an optimal solution. ENC shows that the maximum and minimum in- and out-degrees of an optimal network structure should be almost identical, or within a very narrow range, i.e., the network should be extremely homogeneous. Edge rectification towards the satisfaction of the ENC is then designed and evaluated. Simulation results on large-sized synthetic and real-world networks verify the effectiveness of both the observed ENC and the edge rectification scheme. As more operations of edge rectification are performed, the network is getting closer to exactly satisfying the ENC, and consequently the robustness of the network controllability is enhanced towards optimum. [ABSTRACT FROM AUTHOR]

We study distributed optimization to minimize a sum of smooth and strongly-convex functions. Recent work on this problem uses gradient tracking to achieve linear convergence to the exact global minimizer. However, a connection among different approaches has been unclear. In this paper, we first show that many of the existing first-order algorithms are related with a simple state transformation, at the heart of which lies a recently introduced algorithm known as  $\mathcal {AB}$. We then present distributed heavy-ball, denoted as  $\mathcal {AB}m$ , that combines  $\mathcal {AB}$ with a momentum term and uses nonidentical local step-sizes. By simultaneously implementing both row- and column-stochastic weights,  $\mathcal {AB}m$ removes the conservatism in the related work due to doubly stochastic weights or eigenvector estimation.  $\mathcal {AB}m$ thus naturally leads to optimization and average consensus over both undirected and directed graphs. We show that  $\mathcal {AB}m$ has a global $R$ -linear rate when the largest step-size and momentum parameter are positive and sufficiently small. We numerically show that  $\mathcal {AB}m$ achieves acceleration, particularly when the objective functions are ill-conditioned. [ABSTRACT FROM AUTHOR]

This paper presents the stability and criticality analysis of integer linear programs with respect to perturbations in stochastic data given as Markov chains. These perturbations affect the initial distribution, the transition matrix, or the stationary distribution of Markov chains. Stability analysis is concerned with obtaining the set of all perturbations for which a solution remains optimal. This paper gives expressions for stability regions for perturbations in the initial distribution, the transition matrix, the stationary distribution, and the product of elements of the transition matrix and the stationary distribution. Furthermore, criticality measures that describe the sensitivity of the objective function with respect to an element of the problem data are derived. Stability regions that preserve the stochasticity of the problem data are given. Finally, stability regions for perturbations of elements of the transition matrix, given that the problem is not linear in the initial distribution or the transition matrix, are obtained using a small perturbation analysis. The results are applied to sensor placement problems and numerical examples are given. [ABSTRACT FROM PUBLISHER]

Recently, there has been significant interest in convex relaxations of the optimal power flow (OPF) problem. A semidefinite programming (SDP) relaxation globally solves many OPF problems. However, there exist practical problems for which the SDP relaxation fails to yield the global solution. Conditions for the success or failure of the SDP relaxation are valuable for determining whether the relaxation is appropriate for a given OPF problem. To move beyond existing conditions, which only apply to a limited class of problems, a typical conjecture is that failure of the SDP relaxation can be related to physical characteristics of the system. By presenting an example OPF problem with two equivalent formulations, this paper demonstrates that physically based conditions cannot universally explain algorithm behavior. The SDP relaxation fails for one formulation but succeeds in finding the global solution to the other formulation. Since these formulations represent the same system, success (or otherwise) of the SDP relaxation must involve factors beyond just the network physics. The lack of universal physical conditions for success of the SDP relaxation motivates the development of tighter convex relaxations capable of solving a broader class of problems. Tools from polynomial optimization theory provide a means of developing tighter relaxations. This paper uses the example problem to illustrate relaxations from the Lasserre hierarchy for polynomial optimization and a related “mixed semidefinite/second-order cone programming” hierarchy. [ABSTRACT FROM AUTHOR]

The paper considers normalized gradient descent (NGD), a natural modification of classical gradient descent (GD) in optimization problems. It is shown that, contrary to GD, NGD escapes saddle points “quickly.” A serious shortcoming of GD in nonconvex problems is that it can take arbitrarily long to escape from the neighborhood of a saddle point. In practice, this issue can significantly slow the convergence of GD, particularly in high-dimensional nonconvex problems. The paper focuses on continuous-time dynamics. It is shown that 1) NGD “almost never” converges to saddle points and 2) the time required for NGD to escape from a ball of radius $r$ about a saddle point $x^*$ is at most $5\sqrt{\kappa }r$ , where $\kappa$ is the condition number of the Hessian of $f$ at $x^*$. As a simple application of these results, a global convergence-time bound is established for NGD under mild assumptions. [ABSTRACT FROM AUTHOR]

The limited memory version of the Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) algorithm is the most popular quasi-Newton algorithm in machine learning and optimization. Recently, it was shown that the stochastic L-BFGS (sL-BFGS) algorithm with the variance-reduced stochastic gradient converges linearly. In this paper, we propose a new sL-BFGS algorithm by importing a proper momentum. We prove an accelerated linear convergence rate under mild conditions. The experimental results on different data sets also verify this acceleration advantage. [ABSTRACT FROM AUTHOR]

This paper studies multi-stage capacitive mixed-signal matrix-vector multiplying digital-to-analog (MDAC) conversion topologies for highly energy-efficient, high-resolution, and high-dimensional MIMO analog processing systems. In order to mitigate nonlinearity due to radix errors and capacitive mismatch encountered in compact low-power MDAC realizations, we introduce stochastic successive approximation, or S2A, as an online optimization algorithm for adaptive array analog signal processing amenable to efficient implementation in massively parallel mixed-signal hardware. S2A offers a direct alternative to stochastic gradient descent overcoming several of its shortcomings, such as its sensitivity to model error, while improving on the rate and quality of convergence. S2A overcomes non-convergence typically encountered with gradient descent for non-convex optimization landscapes induced by a mismatch in capacitive multiplying digital-to-analog converter components when applied to adaptive analog signal processing. Experimental validation of S2A in mixed-signal hardware for real-time RF adaptive beamforming demonstrates 65 dB of over-the-air, multipath interferer suppression in fewer than 25 S2A iterations. [ABSTRACT FROM AUTHOR]

Recently, Thompson sampling has been shown to achieve good theoretical performance guarantees for stochastic control problems with parameter uncertainty when the state, control, and parameter spaces are all finite. Much less is known however about the performance of Thompson sampling when applied to continuous or more general spaces, which constitutes an important class of problems in practice. In this paper, we study Thompson sampling when applied to a broad class of average cost stochastic control problems where the state, control, and parameter spaces are all general measurable spaces. The main contributions of our paper are establishing theoretical performance guarantees for Thompson sampling as measured by: first, expected posterior sampling error; and second, average per period regret. [ABSTRACT FROM AUTHOR]

In this paper, we address the so-called Liouville control problem for discrete-time SISO linear systems in the class of Gaussian distributions. In particular, we propose a systematic procedure for the characterization of a dynamic output feedback policy that will transfer the output of the system, which is a known Gaussian random variable, to a goal Gaussian distribution after a finite number of stages. In the proposed approach, the Liouville control problem is reduced to two decoupled (finite-dimensional) quadratic programs, one of which is subject to a single affine constraint, which is a convex program, whereas the other one is subject to a single quadratic equality constraint. Despite the fact that the second optimization problem is not convex, one can characterize its exact solution via a systematic procedure without resorting to convex relaxation techniques, which may yield suboptimal or even infeasible solutions to the original (nonconvex) optimization problem. Finally, we present numerical simulations that illustrate the key ideas of this paper. [ABSTRACT FROM AUTHOR]

This paper deals with the estimation of convergence rate and domain of attraction of control-Lyapunov functions in Lyapunov-based control. This pair of estimation problems has been considered only for input-affine systems with constraints on the input norm. In this paper, we propose a novel optimization framework to address the estimation of convergence rate and domain of attraction. Specifically, we formulate the estimation problems as min–max bilevel programs for the decay rate of the Lyapunov function, where the inner problem can be resolved using Karush–Kuhn–Tucker optimality conditions, and the resulting single-level programs can be transformed into and solved as mixed-integer nonlinear programs. The proposed approach is applicable to systems with input-nonaffinity or more general forms of input constraints under an input-convexity assumption. [ABSTRACT FROM AUTHOR]

While multiple patterning lithography suffers from considerable and increasing mask manufacturing cost, next generation lithography technologies are urgently required for sub-10 nm technology nodes, where directed self-assembly (DSA) is one of the most promising candidates for contact/via layer fabrication. In addition, redundant via insertion is regarded as an important step in the circuit design flow to improve circuit reliability and yield. In this paper, we propose to adopt wire perturbation to further enhance via manufacturability and redundant via insertion rates. In addition, an improved DSA-compliant and redundant via-aware routing graph model is proposed and a systematic via graph update approach is developed to facilitate the implementation of the router. Experimental results demonstrate that compared with a state-of-the-art work, our wire perturbation approach can averagely increase inserted redundant vias by 6% and reduce unmanufacturable vias by 23% with only 0.9% wirelength overhead, and our routing graph model achieves the same performance as the one proposed in a state-of-the-art work and is much more efficient. [ABSTRACT FROM AUTHOR]

For the design of ring oscillator-based ADCs, little has been reported on how to optimally co-design the readout scheme and the ring oscillator core towards optimal energy efficiency. This paper describes a methodology to find this optimum for a target SQNR and signal bandwidth, for the case of 1st-order quantization noise shaping. In short, starting from initial assumptions on the VCO, the readout optimization boils down to finding the best VCO frequency. From this, the number of readout phases, the sampling frequency of the readout and the number of bits in the quantizers are derived. Then, it is explained how to adjust the VCO core toward the corresponding optimal VCO configuration. This is discussed for the case of a current-controlled ring oscillator, indicating the main constraints that define the design space for such a VCO optimization. If the actual VCO configuration is bounded by practical constraints, this approach can be re-iterated where a new readout optimization is performed taking these constraints into account. [ABSTRACT FROM AUTHOR]

In this paper, we revisit Boole’s expansion theorem to propose a new synthesis method for implication logic circuits based on memristors. By rewriting the sum-of-products form of Boole’s expansion theorem in terms that are best suited for the implication logic, we develop a generalized rule to derive the sequence of operations needed to realize any logic function written in the classical AND-OR form. The proposed method leverages on multi-input operation, minimizing both the number of steps required to compute a given Boolean function and the number of memristors involved. Moreover, it allows using well-established methods of logic circuit optimization like binary decision diagrams, Karnaugh maps, and heuristic algorithms, that are already implemented in commercial CAD software. The proposed method allows a fair comparison between the performance of CMOS and implication logic implementations of the same logic function under the same degree of optimization, and is shown to outperform existing approaches. Possible device-circuit co-design strategies to optimize circuit performance are finally discussed. [ABSTRACT FROM AUTHOR]

This paper develops and validates a comprehensive and universally applicable computational concept for solving nonlinear differential equations (NDEs) through a neurocomputing concept based on cellular neural networks (CNNs). High-precision, stability, convergence, and lowest-possible memory requirements are ensured by the CNN processor architecture. A significant challenge solved in this paper is that all these cited computing features are ensured in all system-states (regular or chaotic ones) and in all bifurcation conditions that may be experienced by NDEs.One particular quintessence of this paper is to develop and demonstrate a solver concept that shows and ensures that CNN processors (realized either in hardware or in software) are universal solvers of NDE models. The solving logic or algorithm of given NDEs (possible examples are: Duffing, Mathieu, Van der Pol, Jerk, Chua, Rössler, Lorenz, Burgers, and the transport equations) through a CNN processor system is provided by a set of templates that are computed by our comprehensive templates calculation technique that we call nonlinear adaptive optimization. This paper is therefore a significant contribution and represents a cutting-edge real-time computational engineering approach, especially while considering the various scientific and engineering applications of this ultrafast, energy-and-memory-efficient, and high-precise NDE solver concept. For illustration purposes, three NDE models are demonstratively solved, and related CNN templates are derived and used: the periodically excited Duffing equation, the Mathieu equation, and the transport equation. [ABSTRACT FROM PUBLISHER]

This paper shows how to choose compensation capacitors for optimal performance in two-coil fixed-frequency inductive wireless power links having a series-parallel (SP) configuration. First, for the SP circuit with given coils, coupling and sinusoidal input voltage, it is shown how to calculate nonnegative valued capacitors to maximize power delivered to a given resistive load. Exact conditions for such a compensated system to deliver the maximum power obtainable from a circuit with the same resistances and an ideal transformer with optimal turns ratio are presented. The second main contribution is a method for selecting compensation capacitors to maximize a weighted sum of efficiency and delivered power to obtain a trade-off between these two quantities. This can allow flexible performance with a given set of coils. All of this can be implemented without numerical optimization software. The results are illustrated by numerical examples and supported by experimental measurements. [ABSTRACT FROM AUTHOR]

In this paper, we present a unified framework for solving a general class of problems arising in the context of set-membership estimation/identification theory. More precisely, the paper aims at providing an original approach for the computation of optimal conditional and robust projection estimates in a nonlinear estimation setting, where the operator relating the data and the parameter to be estimated is assumed to be a generic multivariate polynomial function, and the uncertainties affecting the data are assumed to belong to semialgebraic sets. By noticing that the computation of both the conditional and the robust projection optimal estimators requires the solution to min-max optimization problems that share the same structure, we propose a unified two-stage approach based on semidefinite-relaxation techniques for solving such estimation problems. The key idea of the proposed procedure is to recognize that the optimal functional of the inner optimization problems can be approximated to any desired precision by a multivariate polynomial function by suitably exploiting recently proposed results in the field of parametric optimization. Two simulation examples are reported to show the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

Checking validity of a model is a crucial step in the process of system identification. This is especially true when dealing with switched affine systems since, in this case, the problem of system identification from noisy data is known to be generically NP-Hard and can only be solved in practice by using heuristics and relaxations. Therefore, before the identified models can be used for instance for controller design, they should be systematically validated against additional experimental data. In this paper we address the problem of model (in)validation for multi-input multi-output switched affine systems in output error form with unknown switches. As a first step, we prove that necessary and sufficient invalidation certificates can be obtained by solving a sequence of convex optimization problems. In principle, these problems involve increasingly large matrices. However, as we show in the paper by exploiting recent results from semialgebraic geometry, the proposed algorithm is guaranteed to stop after a finite number of steps that can be be explicitly computed from the a priori information. In addition, this algorithm exploits the sparse structure of the underlying optimization problem to substantially reduce the computational burden. The effectiveness of the proposed method is illustrated using both academic examples and a non-trivial problem arising in computer vision: activity monitoring. [ABSTRACT FROM AUTHOR]

The existing, semisupervised, spectral clustering approaches have two major drawbacks, i.e., either they cannot cope with multiple categories of supervision or they sometimes exhibit unstable effectiveness. To address these issues, two normalized affinity and penalty jointly constrained spectral clustering frameworks as well as their corresponding algorithms, referred to as type-I affinity and penalty jointly constrained spectral clustering (TI-APJCSC) and type-II affinity and penalty jointly constrained spectral clustering (TII-APJCSC), respectively, are proposed in this paper. TI refers to type-I and TII to type-II. The significance of this paper is fourfold. First, benefiting from the distinctive affinity and penalty jointly constrained strategies, both TI-APJCSC and TII-APJCSC are substantially more effective than the existing methods. Second, both TI-APJCSC and TII-APJCSC are fully compatible with the three well-known categories of supervision, i.e., class labels, pairwise constraints, and grouping information. Third, owing to the delicate framework normalization, both TI-APJCSC and TII-APJCSC are quite flexible. With a simple tradeoff factor varying in the small fixed interval (0, 1], they can self-adapt to any semisupervised scenario. Finally, both TI-APJCSC and TII-APJCSC demonstrate strong robustness, not only to the number of pairwise constraints but also to the parameter for affinity measurement. As such, the novel TI-APJCSC and TII-APJCSC algorithms are very practical for medium- and small-scale semisupervised data sets. The experimental studies thoroughly evaluated and demonstrated these advantages on both synthetic and real-life semisupervised data sets. [ABSTRACT FROM AUTHOR]

A low-power continuous-time delta-sigma modulator (CTDSM) incorporating a multi-bit feedback-assisted quantizer (FBAQ) is presented in this paper. The proposed multi-bit quantizer is placed in a negative feedback loop to reduce the signal swing at its input. As a result, the number of comparator required for signal quantization is reduced. Furthermore, the modulator is optimized for low-voltage swing operation, in which the excess-loop-delay compensation is embedded without requiring additional hardware. With a 240-MHz sampling clock, this CTDSM achieves a peak SNDR of 68.3 dB and a dynamic range of 71 dB over a 5-MHz signal bandwidth. Fabricated in a 90-nm CMOS process, this chip consumes 4.6 mW from a 1-V supply, which corresponds to a figure of merit (FoM) of 216 fJ/conversion-step. [ABSTRACT FROM AUTHOR]

The aim of this paper is to approximate a Finite-State Markov (FSM) process by another process defined on a lower dimensional state space, called the approximating process, with respect to a total variation distance fidelity criterion. The approximation problem is formulated as an optimization problem using two different approaches. The first approach is based on approximating the transition probability matrix of the FSM process by a lower-dimensional transition probability matrix, resulting in an approximating process which is a Finite-State Hidden Markov (FSHM) process. The second approach is based on approximating the invariant probability vector of the original FSM process by another invariant probability vector defined on a lower-dimensional state space. Going a step further, a method is proposed based on optimizing a Kullback-Leibler divergence to approximate the FSHM processes by FSM processes. The solutions of these optimization problems are described by optimal partition functions which aggregate the states of the FSM process via a corresponding water-filling solution, resulting in lower-dimensional approximating processes which are FSHM or FSM processes. Throughout the paper, the theoretical results are justified by illustrative examples that demonstrate our proposed methodology. [ABSTRACT FROM AUTHOR]

This paper is concerned with the optimal distributed control (ODC) problem for linear discrete-time deterministic and stochastic systems. The objective is to design a static distributed controller with a prespecified structure that is globally optimal with respect to a quadratic cost functional. It is shown that this NP-hard problem has a quadratic formulation, which can be relaxed to a semidefinite program (SDP). If the SDP relaxation has a rank-1 solution, a globally optimal distributed controller can be recovered from this solution. By utilizing the notion of treewidth, it is proved that the nonlinearity of the ODC problem appears in such a sparse way that an SDP relaxation of this problem has a matrix solution with rank at most 3. Since the proposed SDP relaxation is computationally expensive for a large-scale system, a computationally cheap SDP relaxation is also developed with the property that its objective function indirectly penalizes the rank of the SDP solution. Various techniques are proposed to approximate a low-rank SDP solution with a rank-1 matrix, leading to near globally optimal controllers together with a bound on the optimality degree of each controller. The above results are developed for both finite-horizon and infinite-horizon ODC problems. The SDP relaxations developed in this work are exact for the design of a centralized controller, hence serving as an alternative for solving Riccati equations. The efficacy of the proposed SDP relaxations is elucidated through a case study on the distributed frequency control of power systems. [ABSTRACT FROM PUBLISHER]

Due to their superior material properties over bulk silicon, low-dimensional materials have been actively explored as potential candidates to replace or complement silicon technology in post-Moore era. As the first low-dimensional based transistor technology widely studied, carbon nanotube field effect transistor (CNFET) technology is among the most promising candidates to outperform silicon in the advanced technology nodes, because of its potential to achieve higher speed, better gate control, and better energy efficiency. However, existing fabrication processes for most emerging low-dimensional material based technologies including CNFETs, suffer from imperfections, which lowers the device and circuit yield. This paper presents a quantitative methodology for estimating circuit-level yield of CNFET CMOS circuits, which takes into account imperfections of CNT substrates based on a recently developed CMOS-compatible self-assembly process. The impact of different types of process imperfection on circuit-level yield is analyzed. The paper also provides a methodology using device sizing optimization to effectively improve circuit level yields with minimal impact on area and energy delay product, consequently relaxing the process requirements to satisfy certain circuit-level yield target. It is shown that an 80% pass rate can be achieved even with today’s process for various 4-stage cascade CMOS circuits through the proposed sizing optimization approach. [ABSTRACT FROM PUBLISHER]

In this paper, a neurodynamic approach for image restoration is proposed. Image restoration is a process of estimating original images from blurred and/or noisy images. It can be considered as a mapping problem that can be solved by neural networks. Echo state network (ESN) is a recurrent neural network with a simplified training process, which is adopted to estimate the original images in this paper. The parameter selection is important to the performance of the ESN. Thus, the pigeon-inspired optimization (PIO) approach is employed in the training process of the ESN to obtain desired parameters. Moreover, the orthogonal design strategy is utilized in the initialization of PIO to improve the diversity of individuals. The proposed method is tested on several deteriorated images with different sorts and levels of blur and/or noise. Results obtained by the improved ESN are compared with those obtained by several state-of-the-art methods. It is verified experimentally that better image restorations can be obtained for different blurred and/or noisy instances with the proposed neurodynamic method. In addition, the performance of the orthogonal PIO algorithm is compared with that of several existing bioinspired optimization algorithms to confirm its superiority. [ABSTRACT FROM AUTHOR]

This paper proposes a power and area efficient Laplacian Pyramid processing engine (LPPE) for real-time portable image/video processing applications. On the architecture level, an adaptive data compression technique is proposed to reduce the power and area of FIFOs in the LPPE while maintaining small mean square error (MSE). A new filtering extension method is proposed to reduce the output errors caused by image boundary pixels. On the circuit level, near-threshold operation is adopted to further reduce the power consumption. The proposed LPPE is fabricated in 0.18- \mu\textm CMOS process technology and consumes only 452 \mu\textW/frame with a clock frequency of 3.68 MHz and 112 frames per second at 0.5 V. The area is reduced by 18.98%–36.06% compared to conventional counterpart. [ABSTRACT FROM PUBLISHER]

This paper presents a transimpedance-to-noise optimization approach for design of a resistive shunt-feedback TIA. This optimization offers an enhancement in the transimpedance and a noise performance very close to the theoretical minimum noise of the TIA. In addition, the transimpedance-to-noise optimization approach results in a small front-end FET size which enables a further reduction in power and area. Moreover, this approach enables using a fewer number of stages in the receiver chain which makes a high PSRR feasible and obviates the necessity for using an offset cancellation circuitry. Building on this approach, a fully differential analog front-end including a resistive shunt-feedback TIA and a post amplifier (PA) for time-of-flight (ToF) Lidar receivers is designed and implemented, achieving 94dB Ω transimpedance gain, 71nA input-referred rms noise current, −3dB bandwidth of 340MHz, and power supply rejection ratio (PSRR) of more than 87dB in a 0.11 μm CMOS process. The associated DC power consumption is 19.4mW with V DD of 1.8V. Moreover, a push-pull buffer with 1V output swing is integrated for driving 50 Ω loads, such as off-chip time discriminators, which also additionally amplifies the signal with a gain of 5dB while consuming an extra 20.9mW of DC power. The whole chip (excluding pads) occupies 210 μm × 110 μm in area. [ABSTRACT FROM AUTHOR]

In this paper, we present a framework for risk-sensitive model predictive control (MPC) of linear systems affected by stochastic multiplicative uncertainty. Our key innovation is to consider a time-consistent, dynamic risk evaluation of the cumulative cost as the objective function to be minimized. This framework is axiomatically justified in terms of time-consistency of risk assessments, is amenable to dynamic optimization, and is unifying in the sense that it captures a full range of risk preferences from risk neutral (i.e., expectation) to worst case. Within this framework, we propose and analyze an online risk-sensitive MPC algorithm that is provably stabilizing. Furthermore, by exploiting the dual representation of time-consistent, dynamic risk measures, we cast the computation of the MPC control law as a convex optimization problem amenable to real-time implementation. Simulation results are presented and discussed. [ABSTRACT FROM AUTHOR]

This paper investigates the distributed computation of the well-known linear matrix equation in the form of ${{AXB}} = F$ , with the matrices $A$ , $B$ , $X$ , and $F$ of appropriate dimensions, over multiagent networks from an optimization perspective. In this paper, we consider the standard distributed matrix-information structures, where each agent of the considered multiagent network has access to one of the subblock matrices of $A$ , $B$ , and $F$. To be specific, we first propose different decomposition methods to reformulate the matrix equations in standard structures as distributed constrained optimization problems by introducing substitutional variables; we show that the solutions of the reformulated distributed optimization problems are equivalent to least squares solutions to original matrix equations; and we design distributed continuous-time algorithms for the constrained optimization problems, even by using augmented matrices and a derivative feedback technique. Moreover, we prove the exponential convergence of the algorithms to a least squares solution to the matrix equation for any initial condition. [ABSTRACT FROM AUTHOR]

Traffic congestion often incurs environmental problems. One of the most effective ways to mitigate this is carpooling transportation, which substantially reduces automobile demands. Due to the popularization of smartphones and mobile applications, a carpool service can be conveniently accessed via the intelligent carpool system. In this system, the service optimization required to intelligently and adaptively distribute the carpool participant resources is called the carpool service problem (CSP). Several previous studies have examined viable and preliminary solutions to the CSP by using exact and metaheuristic optimization approaches. For CSP-solving, evolutionary computation (e.g., metaheuristics) is a more promising option in comparison to exact-type approaches. However, all the previous state-of-the-art approaches use pure optimization to solve the CSP. In this paper, we employ the framework of neuroevolution to propose the self-organizing map-based neuroevolution (SOMNE) solver by which the SOM-like network represents the abstract CSP solution and is well-trained by using neural learning and evolutionary mechanism. The experimental section of this paper investigates the comparisons and analyses of two objective functions of the CSP and demonstrates that the proposed SOMNE solver achieves superior results when compared against those the other approaches produce, especially in regard to the optimization of the primary objective functions of the CSP. Finally, the visual results of the SOM are illustrated to show the effectiveness and efficiency of the evolutionary neural learning process. [ABSTRACT FROM AUTHOR]

In this paper, an exchange algorithm is proposed to design sparse linear phase finite impulse response (FIR) filters with reduced effective length. The sparse FIR filter design problem is formally an ${l} _{0}$ -norm minimization problem. This original design problem is re-formulated by encoding the filter coefficients using a binary encoding vector, which represents the locations of the zero and non-zero filter coefficients. An iterative 0-1 exchange process with proper direction control is proposed to propel the minimax approximation error toward the specified upper bound of error for sparsity maximization. The effective length is optimized with a lower priority than sparsity in the proposed algorithm. Simulation results show that the proposed algorithm is superior to the existing algorithms in terms of both sparsity and/or effective length in most cases. [ABSTRACT FROM AUTHOR]