Search

In this paper, different methodologies are employed to improve the linearity performance of a prototype CMOS image sensor (CIS). First, several pixel structures, including a novel pixel design based on a capacitive trans-impedance amplifier (CTIA), are proposed to achieve a higher pixel-level linearity. Furthermore, three types of digital linearity calibration methods are explored. A prototype image sensor designed in 0.18- $\mu \text{m}$ , 1-poly, and 4-metal CIS technology with a pixel array of $128 \times 160$ is used to verify these linearity improvement techniques. The measurement results show that the proposed CTIA pixel has the best linearity result out of all pixel structures. Meanwhile, the proposed calibration methods further improved the linearity of the CIS without changing the pixel structure. The pixel mode method achieves the most significant improvement on the linearity. One type of 4T pixel attains a nonlinearity of 0.028% with pixel mode calibration, which is two times better than the state of the art. Voltage mode (VM) and current mode (CM) calibration methods get rid of the limitation on the illumination condition during calibration operation; especially, CM calibration can further suppress the nonlinearity caused by the integration capacitor $C_{\textit {FD}}$ on the floating diffusion node, which is remnant in VM. [ABSTRACT FROM AUTHOR]

This paper is concerned with the optimal static distributed control problem for linear discrete-time deterministic and stochastic systems. The objective is to design a stabilizing static distributed controller whose performance is close to that of the optimal centralized controller. To this end, we first consider deterministic systems, where the initial state is either given or belongs to a known bounded region. Given an arbitrary centralized controller, we derive a condition under which there exists a distributed controller that generates input and state trajectories close to their counterparts in the centralized closed-loop system. This condition for the design of a distributed controller is translated into an optimization problem, where the optimal objective value of this problem quantifies the closeness of the designed distributed and given centralized control systems. The results are then extended to stochastic systems that are subject to input disturbance and measurement noise. The proposed optimization problem has a closed-form solution (explicit formula) and can be efficiently solved for large-scale systems. The mathematical framework developed in this paper is utilized to design a near-globally optimal distributed controller based on the optimal centralized controller, and strong theoretical lower bounds on the global optimality guarantee of the obtained distributed controller are derived. We show that if the optimal objective value of the proposed convex program is sufficiently small, the designed controller is stabilizing and nearly globally optimal. To illustrate the results, case studies on aircraft formation and frequency control of power systems are offered. [ABSTRACT FROM AUTHOR]

Imbalanced data with a skewed class distribution are common in many real-world applications. Deep Belief Network (DBN) is a machine learning technique that is effective in classification tasks. However, conventional DBN does not work well for imbalanced data classification because it assumes equal costs for each class. To deal with this problem, cost-sensitive approaches assign different misclassification costs for different classes without disrupting the true data sample distributions. However, due to lack of prior knowledge, the misclassification costs are usually unknown and hard to choose in practice. Moreover, it has not been well studied as to how cost-sensitive learning could improve DBN performance on imbalanced data problems. This paper proposes an evolutionary cost-sensitive deep belief network (ECS-DBN) for imbalanced classification. ECS-DBN uses adaptive differential evolution to optimize the misclassification costs based on the training data that presents an effective approach to incorporating the evaluation measure (i.e., G-mean) into the objective function. We first optimize the misclassification costs, and then apply them to DBN. Adaptive differential evolution optimization is implemented as the optimization algorithm that automatically updates its corresponding parameters without the need of prior domain knowledge. The experiments have shown that the proposed approach consistently outperforms the state of the art on both benchmark data sets and real-world data set for fault diagnosis in tool condition monitoring. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel iterative optimization model for bottom-up saliency detection. By exploring bottom-up saliency principles and semisupervised learning approaches, we design a high-performance saliency analysis method for wide ranging scenes. The proposed algorithm consists of two stages: 1) we develop a boundary homogeneity model to characterize the general position and the contour of the salient objects and 2) we propose a novel iterative optimization model, termed gradual saliency optimization, for further performance improvement. Our main contribution falls on the second stage, where we propose an iterative framework with self-repairing mechanisms for refining saliency maps. In this framework, we further develop a more comprehensive optimization function applying a novel semisupervised learning scheme to enhance the traditional saliency measure. More elaborately, the iterative method can gradually improve the output in each iteration and finally converge to high-quality saliency maps. Based on our experiments on four different public data sets, it can be demonstrated that our approach significantly outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In this paper, we present a framework for analytically estimating the output quality of common digital signal processing (DSP) blocks that utilize approximate adders. The framework is based on considering the error of approximate adders as an additive noise (approximation noise) that disturbs the output of the DSP block in question. A signal processing theoretical modeling approach for describing the power of the approximation noise which is the integral of error spectral density over the bandwidth, is developed. The output qualities of DSP blocks, such as finite impulse response filter, discrete cosine transform, and fast Fourier transform, which utilize approximate adders, are thus estimated. The accuracy of the proposed framework is evaluated by comparing mathematical model predictions to simulation results by using the signal-to-noise ratio (SNR) metric. The inaccuracy of the SNRs predicted by the framework was, on average, less than 2.5dB compared with that obtained from simulations. Therefore, a mathematical optimization approach based on Lagrange Multipliers for optimizing design parameters is also presented. The optimization is realized by choosing a proper configuration of the target block, such as determining the data width of the inexact computation part for each approximate adder in the design. [ABSTRACT FROM AUTHOR]

The reduced-order filtering problems are investigated in this paper for static neural networks with Markovian jumping parameters and mode-dependent time-varying delays. By fully making use of integral inequalities, the designs of reduced-order $H_{\infty }$ and $L_{2}-L_{\infty }$ filters are discussed. The proper gain matrices of filters and the optimal performance indices are efficiently obtained by resolving corresponding convex optimization problems with the constraints of linear matrix inequalities. It is verified that the computational complexity for the reduced-order filter design is significantly reduced when compared with the full-order one. Furthermore, the nonfragile reduced-order filtering problems are also resolved in this paper. Two examples with simulation results are presented to demonstrate the feasibility and application of the established results. [ABSTRACT FROM AUTHOR]

Due to the emerging systems with constraints in terms of power and costs, such as smart sensor interfaces for the Internet-of-Things, the design of the ADCs becomes very challenging. In this paper, energy and area efficient techniques for continuous-time (CT) delta-sigma modulators ($\Delta \Sigma $ Ms) are discussed. These techniques are based on increasing the contribution of the 1-bit comparator to the loop gain by using passive RC integrators together with low gain amplifiers in the $\Delta \Sigma \text{M}$ loop filter. A third-order CT $\Delta \Sigma \text{M}$ is designed using these techniques to demonstrate their validity, and it achieves 27.5 fJ/conv.-step of energy efficiency. Due to the many design issues, such as the tradeoff between RC variations and loop stability, the design of this modulator has been optimized using a genetic algorithm. The 65-nm CMOS $\Delta \Sigma \text{M}$ occupies only $0.013~\mathrm {mm^{2}}$ , dissipates 256 $\mu \text{W}$ from a 0.7-V supply and it achieves a peak SNDR of 69.1 dB in a 2-MHz bandwidth. The dynamic range reaches 76.2 dB, which corresponds to a $\mathrm {FoM_{Schreier}}$ of 175.1 dB. [ABSTRACT FROM AUTHOR]

In this paper, a built-in self-test and in situ analog circuit optimization platform is proposed and characterized. By integrating a fully digital optimization engine and self-test circuits along with circuits-under-optimization (CUO), this platform can automatically find an operating point that makes a good balance among multiple competing characteristic goals. Therefore, the arbitrary linear time-invariant CUO can be optimized even when there are large process-voltage-temperature variations and aging effects of devices. This platform is analyzed to determine the required accuracy of its building blocks in terms of noise and linearity. The feasibility of this platform is proved by a case study utilizing a Tow-Thomas bandpass biquad. [ABSTRACT FROM AUTHOR]

A 14 bit 500 MS/s current-steering digital-to-analog converter (DAC) was designed and fabricated in 0.13 \mum CMOS process. For traditional wide-band current-steering DACs, the spurious-free dynamic range (SFDR) is limited by nonlinear distortions from the code-dependent load variations and the code-dependent switching glitches. They are analyzed in this paper and mitigated by the proposed complementary switched current sources (CSCS) and time-relaxed interleaving digital- random-return-to-zero (TRI-DRRZ), respectively. The proposed techniques are fabricated and measured, with an SFDR of 84.8 dB at 11 MHz signal frequency and 73.5 dB at 244 MHz. The DAC consumes 299 mW from a mixed power supply of 1.2 V and 2.5 V with an active area of 1.85\times 0.65\ mm^2. [ABSTRACT FROM AUTHOR]

This paper presents a human-inspired control approach to bipedal robotic walking: utilizing human data and output functions that appear to be intrinsic to human walking in order to formally design controllers that provably result in stable robotic walking. Beginning with human walking data, outputs—or functions of the kinematics—are determined that result in a low-dimensional representation of human locomotion. These same outputs can be considered on a robot, and human-inspired control is used to drive the outputs of the robot to the outputs of the human. The main results of this paper are that, in the case of both under and full actuation, the parameters of this controller can be determined through a human-inspired optimization problem that provides the best fit of the human data while simultaneously provably guaranteeing stable robotic walking for which the initial condition can be computed in closed form. These formal results are demonstrated in simulation by considering two bipedal robots—an underactuated 2-D bipedal robot, AMBER, and fully actuated 3-D bipedal robot, NAO—for which stable robotic walking is automatically obtained using only human data. Moreover, in both cases, these simulated walking gaits are realized experimentally to obtain human-inspired bipedal walking on the actual robots. [ABSTRACT FROM PUBLISHER]

Once there are constant or infrequently changed bits (COIBs) in two adjacent input messages of SHA-2, the switching power of input messages data registers (IMD-REGs) used for COIBs will disappear. Meanwhile, when full pipeline SHA-2 is applied in a certain application scenario where the IMD-REGs used for COIBs can be removed, more area of full pipeline SHA-2 can be saved as the proportion of IMD-REGs in message word registers increases. This paper proposes a new message expansion structure for full pipeline SHA-2 to increase the proportion of IMD-REGs. By inserting two expanders in last part of expansion structure pipeline stages and rescheduling the expander, the consumption rate of input messages will be decreased and the proportion of IMD-REGs will be increased. Compared with normal message expansion structure, the ratio of IMD-REGs to total message word registers in the proposed structure is increased from 15.1% to 41.6% for full pipeline SHA256, and 11.2% to 32.4% for full pipeline SHA512. When COIBs exists in adjacent input messages, the power and area advantages of proposed new message expansion structure have been demonstrated by FPGA and ASIC implementations. [ABSTRACT FROM AUTHOR]

Training of Deep Neural Network (DNN) at the edge faces the challenge of high energy consumption due to the requirements of a large number of memory accesses for gradient calculations. Therefore, it is necessary to minimize data fetches to perform training of a DNN model on the edge. In this paper, a novel technique has been proposed to reduce the memory access for the training of fully connected layers in transfer learning. By analyzing the memory access patterns in the backpropagation phase in fully connected layers, the memory access can be optimized. We introduce a new method to update the weights by introducing the delta term for every node of output and fully connected layer. Delta term aims to reduce memory access for the parameters which are required to access repeatedly during the training process of fully connected layers. The proposed technique shows 0.13x-13.93x energy savings for the training of fully connected layers for famous DNN architectures on multiple processor architectures. The proposed technique can be used to perform transfer learning on-chip to reduce energy consumption as well as memory access. [ABSTRACT FROM AUTHOR]

Under process, voltage, and temperature variations, SRAM cell stability largely fluctuates from the nominal value. In the design step, SRAM cell optimization while ignoring the fluctuation induces the yield loss for the stability. Variation-aware optimization of an SRAM cell can prevent the yield loss problem by considering the mean and variance of SRAM cell stability when finding optimal design parameters. This paper proposes a novel SRAM optimization method that uses a deep neural network (DNN). Multiple DNNs from ensemble techniques represent the mean and variance of SRAM cell stability for the nominal design parameters. Subsequent sensitivity analysis of DNN extracts the ${K}$ design parameters that have the most dominant effects on the mean and variance of SRAM cell stability. Then multidimensional optimization is used to find the optimal values of these ${K}$ parameters to maximize the mean stability while minimizing its variance. The proposed method achieved an average of 2% error compared to MC simulation. The proposed optimization method takes only 561 s to provide the most optimal design parameter values of an SRAM cell. [ABSTRACT FROM AUTHOR]

The paper proposes an identification procedure for autoregressive Gaussian stationary stochastic processes under the assumption that the manifest (or observed) variables are nearly independent when conditioned on a limited number of latent (or hidden) variables. The method exploits the sparse plus low-rank decomposition of the inverse of the manifest spectral density and the efficient convex relaxations recently proposed for such decompositions. [ABSTRACT FROM PUBLISHER]

This paper presents a 14-bit 250 MS/s ADC fabricated in a 180 nm CMOS process, which aims at optimizing its linearity, operating speed, and power efficiency. The implemented ADC employs an improved SHA with parasitic optimized bootstrapped switches to achieve high sampling linearity over a wide input frequency range. It also explores a dedicated foreground calibration to correct the capacitor mismatches and the gain error of residue amplifier, where a novel configuration scheme with little cost for analog front-end is developed. Moreover, a partial non-overlapping clock scheme associated with a high-speed reference buffer and fast comparators is proposed to maximize the residue settling time. The implemented ADC is measured under different input frequencies with a sampling rate of 250 MS/s and it consumes 300 mW from a 1.8 V supply. For 30 MHz input, the measured SFDR and SNDR of the ADC is 94.7 dB and 68.5 dB, which can remain over 84.3 dB and 65.4 dB for up to 400 MHz. The measured DNL and INL after calibration are optimized to 0.15 LSB and 1.00 LSB, respectively, while the Walden FOM at Nyquist frequency is 0.57 pJ/step. [ABSTRACT FROM AUTHOR]

In this paper, we consider the l\infty gain characterizations of linear switched systems (LSS) and present various relevant results on their exact computation and optimization. Depending on the role of the switching sequence, we study two broad cases: first, when the switching sequence attempts to maximize, and second, when it attempts to minimize the l\infty gain. The first, named as worst-case throughout the paper, can be related to robustness of the system to uncontrolled switching; the second relates to situations when the switching can be part to the overall decision making. Although, in general, the exact computation of l\infty gains is difficult, we provide specific classes, the input-output switching systems, for which it is shown that linear programming can be used to obtain the worst-case l\infty gain. This is a sufficiently rich class of systems as any stable LSS can be approximated by one. Certain applications to robust control design are provided where we show that a switched compensation independently of the plant has no advantage over a linear time invariant (LTI) compensation, and further, if the plant is strictly causal, even a switched compensation which has a matched switching with the plant does not provide a better performance over an LTI compensation. Also, we present a new necessary and sufficient condition to check the stability of LSS in form of a model matching problem. On the other hand, if one is interested in minimizing the l\infty gain over the switching sequences, we show that, for finite impulse response (FIR) switching systems the minimizing switching sequence can be chosen to be periodic. For input-only or output-only switching an exact, readily computable, characterization of the minimal l\infty gain is provided, and it is shown that the minimizing switching sequence is constant, which, as also shown, is not true for input-output switching. [ABSTRACT FROM AUTHOR]

This paper proposes an equalization approach to mean-square (MS) feedback stabilization for multi-input/multi-output (MIMO) discrete-time systems over fading channels. An orthogonal encode/decode matrix pair is placed at the two ends of the fading channel, thereby equalizing the complementary sensitivity in the MS sense. By adopting the notion of the MS stability in an input-output setting, and using the signal-to-noise ratio (SNR) and encode/decode matrix as additional design parameters, we are able to convert the networked stabilization over fading channels into an equivalent constrained \mathcal H2 optimal control, and derive the MS stabilizability conditions over linear time-invariant controllers. Our results recover the existing MS stabilizability condition under the state feedback. For output feedback control, we provide a characterization of the minimum network resource for the SNR required to stabilize the networked control system over the fading channel, and develop the synthesis algorithms for design of the orthogonal coding matrix and output feedback controller in different scenarios. Our results show that the MS stabilization problem as formulated in this paper admits the closed-form solution for MIMO plants with unstable block zeros, and is mathematically tractable for more general MIMO plants. Two numerical examples are worked out to illustrate our results. [ABSTRACT FROM PUBLISHER]

Consider a connected network of agents endowed with local cost functions representing private objectives. Agents seek to find an agreement on some minimizer of the aggregate cost, by means of repeated communications between neighbors. Consensus on the average over the network, usually addressed by gossip algorithms, is a special instance of this problem, corresponding to quadratic private objectives. Consensus on the median, or more generally, consensus on a given quantile, is also a special instance of this problem. In this paper we show that optimizing the aggregate cost function regularized by a total variation (TV) term has appealing properties. First, it can be done very naturally in a distributed way, yielding algorithms that are efficient on numerical simulations. Secondly, the optimum for the regularized cost is shown to be also the optimum for the initial aggregate cost function under assumptions that are simple to state. Finally, these algorithms are robust to unreliable agents that keep injecting some false value in the network. This is remarkable enough, and is not the case, for instance, of gossip algorithms that are entirely ruled by unreliable agents as detailed in the paper. [ABSTRACT FROM PUBLISHER]

This paper presents comprehensive analyses of power-performance tradeoffs in current-mode logic (CML)-based transmitters via equation-based optimization and provides practical design guidelines that can help achieving optimum power efficiency. An accurate equation-based optimization framework was developed for this purpose, and using this framework, key circuit-level design parameters were found with specific design specifications and process technologies. The proposed optimization framework can determine optimum clock edge-rate (i.e., ratio of rise/fall time to clock period) and inter-stage voltage swings that enable significant power savings without affecting performance. Various transmitter design tradeoffs were analyzed for 45-nm, 65-nm, and 90-nm technologies considering data rate, clock edge-rate, and inter-stage swing optimization. Based on the analysis results, we determined the most energy-efficient data rate that can be achieved with acceptable power overhead for each technology node. Also, we identified the optimal clock edge-rate and inter-stage voltage swings in CML gates that achieved minimum transmitter power dissipation. Our analysis results indicate that, when properly optimized, overall transmitter power consumption can be reduced by up to 43% with inter-stage swing optimization with all other design constraints held constant. [ABSTRACT FROM AUTHOR]

This paper studies the problem of minimizing a sum of (possible nonsmooth) convex functions that are corresponding to multiple interacting nodes, subject to a convex state constraint set. Time-varying directed network is considered here. Two types of computational constraints are investigated in this paper: one where the information of gradients is not available and the other where the projection steps can only be calculated approximately. We devise a distributed zeroth-order method, the implementation of which needs only functional evaluations and approximate projection. In particular, we show that the proposed method generates expected function value sequences that converge to the optimal value, provided that the projection errors decrease at appropriate rates. [ABSTRACT FROM AUTHOR]

This paper develops a neural network architecture and a new processing method for solving in real time, the nonlinear equality constrained multiobjective optimization problem (NECMOP), where several nonlinear objective functions must be optimized in a conflicting situation. In this processing method, the NECMOP is converted to an equivalent scalar optimization problem (SOP). The SOP is then decomposed into several-separable subproblems processable in parallel and in a reasonable time by multiplexing switched capacitor circuits. The approach which we propose makes use of a decomposition–coordination principle that allows nonlinearity to be treated at a local level and where coordination is achieved through the use of Lagrange multipliers. The modularity and the regularity of the neural networks architecture herein proposed make it suitable for very large scale integration implementation. An application to the resolution of a physical problem is given to show that the approach used here possesses some advantages of the point of algorithmic view, and provides processes of resolution often simpler than the usual techniques. [ABSTRACT FROM PUBLISHER]

Comparator power consumption is a major bottleneck to the power efficiency of a high resolution successive approximation register (SAR) analog-to-digital converter (ADC) used in low-power applications. This paper analyzes theoretically the optimal comparators that need to be used to achieve a desired overall performance at minimum power levels. A simple and accurate mathematical model of the SAR ADC that is amenable to analysis is first presented. The mathematical formulation suggests that the comparison power allocated to each bit step need not be the same. The optimization problem, therefore, is solved so that the comparator power budget is optimally distributed among all bit steps. Simulation results show that up to 50% and 60% power savings can be achieved when 10 and 12 comparators are employed for 10 b and 12 b SAR ADCs, respectively. To reduce the implementation complexity, comparator noise allocation problem is also solved when fewer than N comparators are employed in an N-bit SAR ADC. Simulation results suggest that two comparators is sufficient to achieve near ideal performance in 10 b and 12 b SAR ADCs. [ABSTRACT FROM AUTHOR]

This article discusses methods of parametric and structural-parametric synthesis of resonators, which contain transmission line segments of the same length. These methods allow optimizing resonators of various configurations with different segments number $N$ and characteristic impedances $Z_{0i}$. We will call optimal resonators if they have a minimum ratio $m = Z_{\mathrm {0max}}/Z_{\mathrm {0min}}$ at given ratio between the critical frequencies (poles and zeros of the input impedance). The input impedances of the resonators under consideration at the Richards’ frequency variable $S$ are described by the reactance functions of $N$ degree: $Z^{N}(S)$. The parametric synthesis refers to stepped impedance resonators (SIRs). The structural-parametric synthesis refers to resonators of a more general form, which contain open- and short-circuited stubs. The methods are based on two states established in this paper: the order of the reactance function $Z^{N}(S)$ should be the minimum possible ($N = N_{\mathrm {min}}$) to implement the required ratio between certain critical frequencies; the characteristic impedances of the composite sections $Z_{0i}$ take only two values $Z_{\mathrm {0min}}$ or $Z_{\mathrm {0max}}$. These methods were used to synthesize different resonators useful for practical application. With the help of the structural-parametric synthesis, the resonators of a more complex configuration are synthesized, which significantly exceed SIRs. The results of the measurements and simulation are presented. [ABSTRACT FROM AUTHOR]

This paper presents a simple linear programming (LP) technique for designing high-accuracy low-complexity finite-impulse-response (FIR) variable fractional-delay (VFD) digital filters in the minimax error sense. The objective of the minimax design is to minimize the maximum absolute error of the variable frequency response (VFR) of an FIR VFD filter, which is a nonlinear problem and difficult to solve. This paper shows that the minimax design can be approximately decomposed into a pair of separate LP subproblems by decoupling the minimization of the real-part VFR error from that of the imaginary-part error. As a result, the original nonlinear minimax design problem can be easily solved by solving the two LP subproblems separately. To reduce the VFD filter complexity, we also propose a one-by-one increase scheme for optimizing the subfilter orders in the Farrow structure such that a given design specification (maximum absolute error of VFR) can be exactly satisfied. Both even-order and odd-order design examples are given to illustrate that the decoupling minimax method is not only simple, but also can achieve excellent high-accuracy low-complexity FIR VFD filters. [ABSTRACT FROM PUBLISHER]

The first part of this note identifies and analyzes two weak points of the commented paper refid="ref1"/. i) Example 1 reports an incorrect result for which we provide a complete explanation and correction. ii) The practical use of Theorem 1 reveals major drawbacks such as a severe limitation in the applicability (due to the restrictive conditions requested by the hypothesis) and a relatively low accuracy of the results (i.e., rough values for the right outer bounds of the eigenvalue ranges of interval matrices). The second part of the note presents three methods that circumvent the use of Theorem 1 refid="ref1"/ and provide the following information about the eigenvalue range of an interval matrix: \bullet a right outer bound—relying on an inequality proved by J. Rohn (see References); \bullet the right end point—by solving a constrained global maximization problem; \bullet a right outer bound (for arbitrary interval matrices) and the right end point (for some classes of interval matrices)—by calculating the spectral abscissa of a constant matrix that majorizes the considered interval matrix. For illustration by numerical examples, throughout the note we use three interval matrices related to Example 1 from the commented paper. [ABSTRACT FROM AUTHOR]

Advanced power amplifier (PA) architectures are critical for 5G communication which requires PAs to have sufficient output power, high linearity, and high energy efficiency. The time-variant operational environment further demands the self-reconfigurability in the PA design. Recent developments in bandit-problems and reinforcement learning (RL) have led to data-driven control algorithms. This paper presents various RL-based algorithms for Doherty PA control, which achieve robust adaptive operation over environmental changes. Multiple RL frameworks are incorporated in the control algorithms including multi-armed bandit (MAB), continuum-armed bandit (CAB), contextual-bandit (CB), and actor-critic with experience replay (AC). The control algorithms based on the latter three frameworks leverage on the prior information about the Doherty PA’s characteristics to improve learning efficiency. In our simulation test where the optimal policy needs to be adjusted due to transmitter output load impedance mismatch, the MAB-based control learns the optimal policy within 25,000 samples. The control algorithms based on CAB and CB learn the optimal policy within 5,000 samples. The fastest learning rate is achieved by the control algorithm based on AC, which learns the optimal policy within 1,500 samples. [ABSTRACT FROM AUTHOR]

Deep neural network (DNN) has been widely studied due to its high performance and usability for various applications such as image classification, detection, segmentation, translation, and action recognition. Thanks to the universal applications and high performance of DNN algorithm, DNN is adopted for various AI platforms, including edge/mobile devices as well as cloud servers. However, high-performance DNN requires a large amount of computation and memory access, making it challenging to implement DNN operation on edge/mobile. There have been several ways to solve these problems, including algorithms as well as hardware for DNN. Algorithms that help accelerate DNN in hardware enable much more efficient operation of high-performance AI. This article aims to provide an overview of the recent hardware and algorithm co-design schemes enabling efficient processing of DNNs. Specifically, it will provide algorithm optimization methods for DNN structure, neurons, synapses, and data types. This paper also introduces optimization methods for hardware architectures, PE array, data-path control, and microarchitecture of PE. And we will also show examples of DNN algorithm and hardware co-designed ASICs. [ABSTRACT FROM AUTHOR]

This paper presents an ultra-broadband 5-bit switched-type attenuator (STA). Three attenuation topologies are employed for the design of the attenuation cells, including the T-type, simplified T-type and Π-type topologies. For each attenuation topology, the optimal values of the series and shunt resistors are derived to achieve accurate amplitude tuning and ensure good impedance matching. The capacitive compensation technique is adopted by both the T-type and Π-type topologies to enhance the high-frequency performance. Based on the proposed optimization techniques, a systematic design methodology is developed for the STA. The 5-bit STA is implemented in 65-nm CMOS technology and occupies a core chip area of only 0.036 mm2. It exhibits ultra-broadband operation with 15.5-dB amplitude tuning range and 0.5-dB tuning step. The insertion loss of the reference state is 1.5 – 5.9 dB from DC to 50 GHz. The return loss is better than 12 dB for all the 32 states. The RMS amplitude error and phase error are less than 0.25 dB and 3.5° over the DC – 50 GHz band. [ABSTRACT FROM AUTHOR]

This paper considers the problem of selecting the top-m designs using simulation with input uncertainty. The performance of each design is measured by its worst case performance. The objective of this paper is to maximize the probability of correctly selecting the top-m designs given a fixed simulation budget. Due to the complexity of probability of correct selection (PCS), we develop a lower bound for the PCS and derive an asymptotically optimal budget allocation rule. Useful insights on characterizing the efficient budget allocation rule with input uncertainty are provided. Meanwhile, a sequential simulation procedure is suggested to implement the allocation rule. A series of numerical experiments indicate that the proposed simulation budget allocation rule can outperform all existing selection rules. [ABSTRACT FROM AUTHOR]

Continuous-time and discrete-time forms of Zhang dynamics (ZD) for time-varying nonlinear optimization have been developed recently. In this paper, a novel discrete-time ZD (DTZD) algorithm is proposed and investigated based on the previous research. Specifically, the DTZD algorithm for time-varying nonlinear optimization is developed by adopting a new Taylor-type difference rule. This algorithm is a five-step iteration process, and thus, is referred to as the five-step DTZD algorithm in this paper. Theoretical analysis and results of the proposed five-step DTZD algorithm are presented to highlight its excellent computational performance. The geometric representation of the proposed algorithm for time-varying nonlinear optimization is also provided. Comparative numerical results are illustrated with four examples to substantiate the efficacy and superiority of the proposed five-step DTZD algorithm for time-varying nonlinear optimization compared with the previous DTZD algorithms. [ABSTRACT FROM AUTHOR]

The vast majority of dimensionality reduction (DR) techniques rely on the second-order statistics to define their optimization objective. Even though this provides adequate results in most cases, it comes with several shortcomings. The methods require carefully designed regularizers and they are usually prone to outliers. In this paper, a new DR framework that can directly model the target distribution using the notion of similarity instead of distance is introduced. The proposed framework, called similarity embedding framework (SEF), can overcome the aforementioned limitations and provides a conceptually simpler way to express optimization targets similar to existing DR techniques. Deriving a new DR technique using the SEF becomes simply a matter of choosing an appropriate target similarity matrix. A variety of classical tasks, such as performing supervised DR and providing out-of-sample extensions, as well as, new novel techniques, such as providing fast linear embeddings for complex techniques, are demonstrated in this paper using the proposed framework. Six data sets from a diverse range of domains are used to evaluate the proposed method and it is demonstrated that it can outperform many existing DR techniques. [ABSTRACT FROM AUTHOR]

The truncated regular $L_{1}$ -loss support vector machine can eliminate the excessive number of support vectors (SVs); thus, it has significant advantages in robustness and scalability. However, in this paper, we discover that the associated state-of-the-art solvers, such as difference convex algorithm and concave–convex procedure, not only have limited sparsity promoting property for general truncated losses especially the $L_{2}$ -loss but also have poor scalability for large-scale problems. To circumvent these drawbacks, we present a general multistage scheme with explicit interpretation regarding SVs as well as outliers. In particular, we solve the general nonconvex truncated loss minimization through a sequence of associated convex subproblems, in which the outliers are removed in advance. The proposed algorithm can be regarded as a structural optimization attempt carefully considering sparsity imposed by the nonconvex truncated losses. We show that this general multistage algorithm offers sufficient sparsity especially for the truncated $L_{2}$ -loss. To further improve the scalability, we propose a linear multistep algorithm by employing a single iteration of coordinate descent to monotonically decrease the objective function at each stage and a kernel algorithm by using the Karush–Kuhn–Tucker conditions to cheaply find most part of the outliers for the next stage. Comparison experiments demonstrate that our methods have superiority in sparsity as well as efficiency in scalability. [ABSTRACT FROM AUTHOR]

Hashing is emerging as a powerful tool for building highly efficient indices in large-scale search systems. In this paper, we study spectral hashing (SH), which is a classical method of unsupervised hashing. In general, SH solves for the hash codes by minimizing an objective function that tries to preserve the similarity structure of the data given. Although computationally simple, very often SH performs unsatisfactorily and lags distinctly behind the state-of-the-art methods. We observe that the inferior performance of SH is mainly due to its imperfect formulation; that is, the optimization of the minimization problem in SH actually cannot ensure that the similarity structure of the high-dimensional data is really preserved in the low-dimensional hash code space. In this paper, we, therefore, introduce reversed SH (ReSH), which is SH with its input and output interchanged. Unlike SH, which estimates the similarity structure from the given high-dimensional data, our ReSH defines the similarities between data points according to the unknown low-dimensional hash codes. Equipped with such a reversal mechanism, ReSH can seamlessly overcome the drawback of SH. More precisely, the minimization problem in our ReSH can be optimized if and only if similar data points are mapped to adjacent hash codes, and mostly important, dissimilar data points are considerably separated from each other in the code space. Finally, we solve the minimization problem in ReSH by multilayer neural networks and obtain state-of-the-art retrieval results on three benchmark data sets. [ABSTRACT FROM AUTHOR]

This paper investigates the dynamical properties of a network of neurons, each of which implements an asynchronous mapping based on polynomial growth transforms. In the first part of this paper, we present a geometric approach for visualizing the dynamics of the network where each of the neurons traverses a trajectory in a dual optimization space, whereas the network itself traverses a trajectory in an equivalent primal optimization space. We show that as the network learns to solve basic classification tasks, different choices of primal-dual mapping produce unique but interpretable neural dynamics like noise shaping, spiking, and bursting. While the proposed framework is general enough, in this paper, we demonstrate its use for designing support vector machines (SVMs) that exhibit noise-shaping properties similar to those of $\Sigma \Delta $ modulators, and for designing SVMs that learn to encode information using spikes and bursts. It is demonstrated that the emergent switching, spiking, and burst dynamics produced by each neuron encodes its respective margin of separation from a classification hyperplane whose parameters are encoded by the network population dynamics. We believe that the proposed growth transform neuron model and the underlying geometric framework could serve as an important tool to connect well-established machine learning algorithms like SVMs to neuromorphic principles like spiking, bursting, population encoding, and noise shaping. [ABSTRACT FROM AUTHOR]

This paper presents the hardware design and implementation of 1-D median filter that uses modular architecture, which produces median results hierarchically. Different types of submodules could be applied to form a customized architecture in order to meet different constraints and requirements. Complete analysis and hardware-oriented optimization were performed to achieve the optimal configurations when the input size and data length were changed. As the data length increases, the required resources and latency increase linearly, while the maximal operation frequency is nearly independent to data length. When our filter is synthesized using 90-nm process technology, its operating frequency could achieve more than 2000 MHz and resource consumption is reduced by 23.29% when compared with the state-of-the-art design. The experimental results show that the proposed cascaded architecture is superior to existing designs in terms of maximal operating speed and resource costs. [ABSTRACT FROM PUBLISHER]

Multitask learning (MTL) aims to learn multiple tasks simultaneously through the interdependence between different tasks. The way to measure the relatedness between tasks is always a popular issue. There are mainly two ways to measure relatedness between tasks: common parameters sharing and common features sharing across different tasks. However, these two types of relatedness are mainly learned independently, leading to a loss of information. In this paper, we propose a new strategy to measure the relatedness that jointly learns shared parameters and shared feature representations. The objective of our proposed method is to transform the features of different tasks into a common feature space in which the tasks are closely related and the shared parameters can be better optimized. We give a detailed introduction to our proposed MTL method. Additionally, an alternating algorithm is introduced to optimize the nonconvex objection. A theoretical bound is given to demonstrate that the relatedness between tasks can be better measured by our proposed MTL algorithm. We conduct various experiments to verify the superiority of the proposed joint model and feature MTL method. [ABSTRACT FROM AUTHOR]

Conventional feature selection algorithms assign a single common feature set to all regions of the sample space. In contrast, this paper proposes a novel algorithm for localized feature selection for which each region of the sample space is characterized by its individual distinct feature subset that may vary in size and membership. This approach can therefore select an optimal feature subset that adapts to local variations of the sample space, and hence offer the potential for improved performance. Feature subsets are computed by choosing an optimal coordinate space so that, within a localized region, within-class distances and between-class distances are, respectively, minimized and maximized. Distances are measured using a logistic function metric within the corresponding region. This enables the optimization process to focus on a localized region within the sample space. A local classification approach is utilized for measuring the similarity of a new input data point to each class. The proposed logistic localized feature selection (lLFS) algorithm is invariant to the underlying probability distribution of the data; hence, it is appropriate when the data are distributed on a nonlinear or disjoint manifold. lLFS is efficiently formulated as a joint convex/increasing quasi-convex optimization problem with a unique global optimum point. The method is most applicable when the number of available training samples is small. The performance of the proposed localized method is successfully demonstrated on a large variety of data sets. We demonstrate that the number of features selected by the lLFS method saturates at the number of available discriminative features. In addition, we have shown that the Vapnik-Chervonenkis dimension of the localized classifier is finite. Both these factors suggest that the lLFS method is insensitive to the overfitting issue, relative to other methods. [ABSTRACT FROM AUTHOR]

In this paper, we consider the regularized version of the Jacobi algorithm, a block coordinate descent method for convex optimization with an objective function consisting of the sum of a differentiable function and a block-separable function. Under certain regularity assumptions on the objective function, this algorithm has been shown to satisfy the so-called sufficient decrease condition, and consequently, to converge in objective function value. In this paper, we revisit the convergence analysis of the regularized Jacobi algorithm and show that it also converges in iterates under very mild conditions on the objective function. Moreover, we establish conditions under which the algorithm achieves a linear convergence rate. [ABSTRACT FROM PUBLISHER]

In model predictive control (MPC), the control input is computed by solving a constrained finite-time optimal control (CFTOC) problem at each sample in the control loop. The main computational effort when solving the CFTOC problem using an active-set (AS) method is often spent on computing the search directions, which in MPC corresponds to solving unconstrained finite-time optimal control (UFTOC) problems. This is commonly performed using Riccati recursions or generic sparsity exploiting algorithms. In this paper, the focus is efficient search direction computations for AS type methods. The system of equations to be solved at each AS iteration is changed only by a low-rank modification of the previous one, and exploiting this structured change is important for the performance of AS-type solvers. In this paper, theory for how to exploit these low-rank changes by modifying the Riccati factorization between AS iterations in a structured way is presented. A numerical evaluation of the proposed algorithm shows that the computation time can be significantly reduced by modifying, instead of re-computing, the Riccati factorization. This speedup can be important for AS-type solvers used for linear, nonlinear, and hybrid MPC. [ABSTRACT FROM PUBLISHER]

This paper explores optimal control problems with $l_1$ objectives involving linear discrete-time systems. These problems can be efficiently solved as linear programs. They also have previously been shown to yield sparse solutions, including idle or deadbeat solutions where the input or output is respectively zero along the entire control horizon. The main contribution of this paper is to derive conditions on the problem parameters that specify when idle or deadbeat solutions occur. These results, based on analyzing the dual problem, demonstrate how different types of sparse solutions result from the choice of the problem parameters and, as a consequence, may guide the design of controllers employing $l_1$ objectives. [ABSTRACT FROM PUBLISHER]

In this paper, a novel concept factorization (CF) method, called CF with adaptive neighbors (CFANs), is proposed. The idea of CFAN is to integrate an ANs regularization constraint into the CF decomposition. The goal of CFAN is to extract the representation space that maintains geometrical neighborhood structure of the data. Similar to the existing graph-regularized CF, CFAN builds a neighbor graph weights matrix. The key difference is that the CFAN performs dimensionality reduction and finds the neighbor graph weights matrix simultaneously. An efficient algorithm is also derived to solve the proposed problem. We apply the proposed method to the problem of document clustering on the 20 Newsgroups, Reuters-21578, and TDT2 document data sets. Our experiments demonstrate the effectiveness of the method. [ABSTRACT FROM AUTHOR]

This paper addresses two-stage resource allocation in the orthogonal frequency division multiplexing access system. In the subcarrier allocation stage, hysteretic noisy chaotic neural network (HNCNN) with a newly established energy function is proposed for subcarrier allocation to improve the optimization performance and reduce the computational complexity. Activation functions with both anticlockwise and clockwise hysteretic loops are applied to the HNCNN. A new energy function is established for an objective function, which can be calculated offline, resulting in a lower computational complexity in solving subcarrier allocation than the previous energy function. In the power allocation stage, the water-filling algorithm is employed to attain optimal power allocation. Simulation results show that the energy function established in this paper can decrease the runtimes of the neural networks, and that the HNCNN with both anticlockwise and clockwise hysteretic-loop activation functions can improve probabilities of feasible and optimal solutions at higher noises. The two-stage algorithm in this paper outperforms the previous algorithms in fairness, system throughput, and resource utilization. [ABSTRACT FROM AUTHOR]

This paper aims to design variable fractional-delay (VFD) finite-impulse response (FIR) filters with controllable cutoff frequencies. Different with the existing designs, a particular ladder-shaped curve is selected as the desired transfer characteristic of the proposed design. Due to the fact that this particular transfer curve can be well approximated by the convolution windowed spectrum interpolation-based filter, the analytic formula of this filter can be further combined with the cubic spline interpolation and Taylor series expansion, thereby deriving a fast configuration procedure for all tap coefficients in the Farrow structure. Meanwhile, both the fractional delay and the cutoff frequency can be flexibly adjusted by this procedure. By means of theoretical analysis or numerical simulation, we proved that the proposed method is not only suitable to design those VFD FIR filters with low or middle cutoff frequencies, but also concurrently possesses high flexibility in performance adjustment and low complexity in coefficient configuration. Therefore, it is an appropriate supplement of the closed-form WLS design. [ABSTRACT FROM PUBLISHER]

This paper presents a complex delta–sigma modulator (CDSM) designed for integration in a digital transmitter chain targeting multi-standard coexistence with nearby receivers. The use of a DSM has the advantage of increased performance in terms of signal-to-noise-ratio in the band of interest. However, the resulting out-of-band noise becomes an issue for multi-standard coexistence, thus increasing the complexity of the subsequent filtering stage. This constraint could be relaxed in the DSM stage, by placing a complex zero near the frequency band, where a low noise level is needed. This is achieved by cross coupling the in-phase (I) and quadrature (Q) channels, thus obtaining a CDSM. A review of known design methods for CDSM revealed limitations regarding the poles/zeros optimization, and the configurability of the complex zeros placement. The proposed architecture introduces two additional cross couplings from the I and Q quantizers outputs in order to decorrelate the zeros placement and the poles optimization problem. Hence, the improved CDSM can be implemented using existing optimization tools, which reduces considerably the number of iterations and the computational effort. In addition, the resulting modulator can target different coexistence scenarios without the need of redesign, unlike other known methods. Simulation results show a noise level reduction of approximately 20–30-dB near specific frequency bands by the proposed CDSM scheme with respect to standard DSM. Finally, we show an efficient coarse/fine configurability mechanism, which is obtained when introducing additional delays in the cross-coupling paths. [ABSTRACT FROM PUBLISHER]

This paper is concerned with distributed learning and optimization in large-scale settings. The well-known fictitious play (FP) algorithm has been shown to achieve Nash equilibrium learning in certain classes of multiagent games. However, FP can be computationally difficult to implement when the number of players is large. Sampled FP (SFP) is a variant of FP that mitigates the computational difficulties arising in FP by using a Monte Carlo (i.e., sampling based) approach. Despite its computational advantages, a shortcoming of SFP is that the number of samples that must be drawn at each iteration grows without bound as the algorithm progresses. In this paper, we propose single sample FP (SSFP)—A variant of SFP in which only one sample needs to be drawn in each round of the algorithm. Convergence of SSFP to the set of Nash equilibria is proven. Simulation results show the performance of SSFP is comparable to that of SFP, despite drawing far fewer samples. [ABSTRACT FROM PUBLISHER]

This paper presents a nearest neighbor partitioning method designed to improve the performance of a neural-network classifier. For neural-network classifiers, usually the number, positions, and labels of centroids are fixed in partition space before training. However, that approach limits the search for potential neural networks during optimization; the quality of a neural network classifier is based on how clear the decision boundaries are between classes. Although attempts have been made to generate floating centroids automatically, these methods still tend to generate sphere-like partitions and cannot produce flexible decision boundaries. We propose the use of nearest neighbor classification in conjunction with a neural-network classifier. Instead of being bound by sphere-like boundaries (such as the case with centroid-based methods), the flexibility of nearest neighbors increases the chance of finding potential neural networks that have arbitrarily shaped boundaries in partition space. Experimental results demonstrate that the proposed method exhibits superior performance on accuracy and average f-measure. [ABSTRACT FROM AUTHOR]

Dropout and DropConnect are two techniques to facilitate the regularization of neural network models, having achieved the state-of-the-art results in several benchmarks. In this paper, to improve the generalization capability of spiking neural networks (SNNs), the two drop techniques are first applied to the state-of-the-art SpikeProp learning algorithm resulting in two improved learning algorithms called SPDO (SpikeProp with Dropout) and SPDC (SpikeProp with DropConnect). In view that a higher membrane potential of a biological neuron implies a higher probability of neural activation, three adaptive drop algorithms, SpikeProp with Adaptive Dropout (SPADO), SpikeProp with Adaptive DropConnect (SPADC), and SpikeProp with Group Adaptive Drop (SPGAD), are proposed by adaptively adjusting the keep probability for training SNNs. A convergence theorem for SPDC is proven under the assumptions of the bounded norm of connection weights and a finite number of equilibria. In addition, the five proposed algorithms are carried out in a collaborative neurodynamic optimization framework to improve the learning performance of SNNs. The experimental results on the four benchmark data sets demonstrate that the three adaptive algorithms converge faster than SpikeProp, SPDO, and SPDC, and the generalization errors of the five proposed algorithms are significantly smaller than that of SpikeProp. Furthermore, the experimental results also show that the five algorithms based on collaborative neurodynamic optimization can be improved in terms of several measures. [ABSTRACT FROM AUTHOR]

Finding the optimal solution to the constrained l0 -norm minimization problems in the recovery of compressive sensed signals is an NP-hard problem and it usually requires intractable combinatorial searching operations for getting the global optimal solution, unless using other objective functions (e.g., the l1 norm or lp norm) for approximate solutions or using greedy search methods for locally optimal solutions (e.g., the orthogonal matching pursuit type algorithms). In this paper, a neurodynamic optimization method is proposed to solve the l0 -norm minimization problems for obtaining the global optimum using a recurrent neural network (RNN) model. For the RNN model, a group of modified Gaussian functions are constructed and their sum is taken as the objective function for approximating the l0 norm and for optimization. The constructed objective function sets up a convexity condition under which the neurodynamic system is guaranteed to obtain the globally convergent optimal solution. An adaptive adjustment scheme is developed for improving the performance of the optimization algorithm further. Extensive experiments are conducted to test the proposed approach in this paper and the output results validate the effectiveness of the new method. [ABSTRACT FROM PUBLISHER]

This paper presents a loop gain optimization technique for integer-N digital phase-locked loops with a time-to-digital converter. Due to noise filtering properties, a phase-locked loop has an optimal loop gain which gives rise to the best jitter performance, taking into account external and internal noise sources. By using the loop gain optimization technique, the digital phase-locked loops can automatically attain this loop gain in background to minimize the jitter. Theoretical analysis is presented. The stability issue and the impact of loop latency are also discussed. Finally, the analysis is compared to behavioral simulations with good agreement. [ABSTRACT FROM AUTHOR]

This paper deals with the joint optimization problem of high-order error-feedback and realization for minimizing roundoff noise at the output of a closed-loop system with a full-order observer feedback system subject to l2-scaling constraints. It is shown that the problem can be converted into an unconstrained optimization problem by using linear-algebraic techniques. The unconstrained optimization problem at hand is then solved iteratively by employing an efficient quasi-Newton algorithm with closed-form formulas for key gradient evaluation. A case study is presented to demonstrate the validity and effectiveness of the proposed technique. [ABSTRACT FROM AUTHOR]