Search

Adversarial attacks have been extensively studied in recent years since they can identify the vulnerability of deep learning models before deployed. In this paper, we consider the black-box adversarial setting, where the adversary needs to craft adversarial examples without access to the gradients of a target model. Previous methods attempted to approximate the true gradient either by using the transfer gradient of a surrogate white-box model or based on the feedback of model queries. However, the existing methods inevitably suffer from low attack success rates or poor query efficiency since it is difficult to estimate the gradient in a high-dimensional input space with limited information. To address these problems and improve black-box attacks, we propose two prior-guided random gradient-free (PRGF) algorithms based on biased sampling and gradient averaging, respectively. Our methods can take the advantage of a transfer-based prior given by the gradient of a surrogate model and the query information simultaneously. Through theoretical analyses, the transfer-based prior is appropriately integrated with model queries by an optimal coefficient in each method. Extensive experiments demonstrate that, in comparison with the alternative state-of-the-arts, both of our methods require much fewer queries to attack black-box models with higher success rates. [ABSTRACT FROM AUTHOR]

Adversarial attacks have been extensively studied in recent years since they can identify the vulnerability of deep learning models before deployed. In this paper, we consider the black-box adversarial setting, where the adversary needs to craft adversarial examples without access to the gradients of a target model. Previous methods attempted to approximate the true gradient either by using the transfer gradient of a surrogate white-box model or based on the feedback of model queries. However, the existing methods inevitably suffer from low attack success rates or poor query efficiency since it is difficult to estimate the gradient in a high-dimensional input space with limited information. To address these problems and improve black-box attacks, we propose two prior-guided random gradient-free (PRGF) algorithms based on biased sampling and gradient averaging, respectively. Our methods can take the advantage of a transfer-based prior given by the gradient of a surrogate model and the query information simultaneously. Through theoretical analyses, the transfer-based prior is appropriately integrated with model queries by an optimal coefficient in each method. Extensive experiments demonstrate that, in comparison with the alternative state-of-the-arts, both of our methods require much fewer queries to attack black-box models with higher success rates. [ABSTRACT FROM AUTHOR]

Addressing non-convexity plays a fundamental role in solving the optimal electricity-gas flow models. In this paper, an improved spatial branch-and-bound algorithm is proposed to solve the non-convex problem, which is formulated as a mixed-integer bilinear programming, for its exact solution. The core of the algorithm is to divide the non-convex model into convex and small sub-models by branching on specific continuous variables, so that the non-convex problem can be equivalent to a rooted tree for exploration. The exactness of the algorithm is guaranteed by the same criterion as the classical branch-and-bound algorithm. To alleviate the computational burden, a novel two-stage spatial branching strategy is developed to improve the effectiveness and efficiency of the branching operations. The performance of the proposed algorithm is verified on two integrated electricity-gas systems with different sizes. Numerical results demonstrate that our method achieves a balance among feasibility, optimality, and efficiency. The comparison with another 6 convexification-based methods, 3 state-of-the-art non-convex optimization solvers, and 2 spatial branch-and-bound algorithms with classical branching rules further shows the superiority of our algorithm. [ABSTRACT FROM AUTHOR]

Dual Coordinate Descent (DCD) is one of the most popular optimization methods. The parallelization of DCD is difficult, as DCD is sequential in nature. As such, simultaneously running multiple DCD threads on batches of data elements causes result inaccuracy and slow convergence, due to the concurrent updates of multiple coordinates. Some parallelization methods adopt separable approximate functions that depend on the degree of parallelism. Such dependencies result in both poor scalability and slow-convergence. To address these challenges, in this paper we present a new parallel primal-dual algorithm for DCD, called PPD. In PPD, the block data distribution is utilized to obtain a new approximate function that is independent from the parallelism. Moreover, PPD is designed with a novel primal-dual acceleration scheme to approach the optimal solution closely and quickly. We demonstrate the advantages of PPD in terms of scalability and efficiency through experiments. [ABSTRACT FROM AUTHOR]

In this paper, we investigate resource allocation for IRS-assisted green multiuser multiple-input single-output (MISO) systems. To minimize the total transmit power, both the beamforming vectors at the access point (AP) and the phase shifts at multiple IRSs are jointly optimized, while taking into account the minimum required quality-of-service (QoS) of multiple users. First, two novel algorithms, namely a penalty-based alternating minimization (AltMin) algorithm and an inner approximation (IA) algorithm, are developed to tackle the non-convexity of the formulated optimization problem when perfect channel state information (CSI) is available. Existing designs employ semidefinite relaxation in AltMin-based algorithms, which, however, cannot ensure convergence. In contrast, the proposed penalty-based AltMin and IA algorithms are guaranteed to converge to a stationary point and a Karush-Kuhn-Tucker (KKT) solution of the design problem, respectively. Second, the impact of imperfect knowledge of the CSI of the channels between the AP and the users is investigated. To this end, a non-convex robust optimization problem is formulated and the penalty-based AltMin algorithm is extended to obtain a stationary solution. Simulation results reveal a key trade-off between the speed of convergence and the achievable total transmit power for the two proposed algorithms. In addition, we show that the proposed algorithms can significantly reduce the total transmit power at the AP compared to various baseline schemes and that the optimal numbers of transmit antennas and IRS reflecting elements, which maximize the system energy efficiency of the considered system, are finite. [ABSTRACT FROM AUTHOR]

Coupled Matrix Tensor Factorization (CMTF) facilitates the integration and analysis of multiple data sources and helps discover meaningful information. Nonnegative CMTF (N-CMTF) has been employed in many applications for identifying latent patterns, prediction, and recommendation. However, due to the added complexity with coupling between tensor and matrix data, existing N-CMTF algorithms exhibit poor computation efficiency. In this paper, a computationally efficient N-CMTF factorization algorithm is presented based on the column-wise element selection, preventing frequent gradient updates. Theoretical and empirical analyses show that the proposed N-CMTF factorization algorithm is not only more accurate but also more computationally efficient than existing algorithms in approximating the tensor as well as in identifying the underlying nature of factors. [ABSTRACT FROM AUTHOR]

In this paper, we describe a new algorithm to build a few sparse principal components from a given data matrix. Our approach does not explicitly create the covariance matrix of the data and can be viewed as an extension of the Kogbetliantz algorithm to build an approximate singular value decomposition for a few principal components. We show the performance of the proposed algorithm to recover sparse principal components on various datasets from the literature and perform dimensionality reduction for classification applications. [ABSTRACT FROM AUTHOR]

Nonlinear optimization problems with dynamical parameters are widely arising in many practical scientific and engineering applications, and various computational models are presented for solving them under the hypothesis of short-time invariance. To eliminate the large lagging error in the solution of the inherently dynamic nonlinear optimization problem, the only way is to estimate the future unknown information by using the present and previous data during the solving process, which is termed the future dynamic nonlinear optimization (FDNO) problem. In this paper, to suppress noises and improve the accuracy in solving FDNO problems, a novel noise-tolerant neural (NTN) algorithm based on zeroing neural dynamics is proposed and investigated. In addition, for reducing algorithm complexity, the quasi-Newton Broyden–Fletcher–Goldfarb–Shanno (BFGS) method is employed to eliminate the intensively computational burden for matrix inversion, termed NTN-BFGS algorithm. Moreover, theoretical analyses are conducted, which show that the proposed algorithms are able to globally converge to a tiny error bound with or without the pollution of noises. Finally, numerical experiments are conducted to validate the superiority of the proposed NTN and NTN-BFGS algorithms for the online solution of FDNO problems. [ABSTRACT FROM AUTHOR]

In this paper, we consider the policy evaluation problem in reinforcement learning with agents on a decentralized and directed network. In order to evaluate the quality of a fixed policy in this decentralized setting, one option is for agents to run decentralized temporal-difference (TD) collaboratively. To account for the practical scenarios where the state and action spaces are large and malicious attacks emerge, we focus on the decentralized TD learning with linear function approximation in the presence of malicious agents (often termed as Byzantine agents). We propose a trimmed mean-based Byzantine-resilient decentralized TD algorithm to perform policy evaluation in this setting. We establish the finite-time convergence rate, as well as the asymptotic learning error that depends on the number of Byzantine agents. Numerical experiments corroborate the robustness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

The first moment and second central moments of the portfolio return, a.k.a. mean and variance, have been widely employed to assess the expected profit and risk of the portfolio. Investors pursue higher mean and lower variance when designing the portfolios. The two moments can well describe the distribution of the portfolio return when it follows the Gaussian distribution. However, the real world distribution of assets return is usually asymmetric and heavy-tailed, which is far from being a Gaussian distribution. The asymmetry and the heavy-tailedness are characterized by the third and fourth central moments, i.e., skewness and kurtosis, respectively. Higher skewness and lower kurtosis are preferred to reduce the probability of extreme losses. However, incorporating high-order moments in the portfolio design is very difficult due to their non-convexity and rapidly increasing computational cost with the dimension. In this paper, we propose a very efficient and convergence-provable algorithm framework based on the successive convex approximation (SCA) algorithm to solve high-order portfolios. The efficiency of the proposed algorithm framework is demonstrated by the numerical experiments. [ABSTRACT FROM AUTHOR]

Robust optimization over time is a new way to tackle dynamic optimization problems where the goal is to find solutions that remain acceptable over an extended period of time. The state-of-the-art methods in this domain try to identify robust solutions based on their future predicted fitness values. However, predicting future fitness values is difficult and error prone. In this paper, we propose a new framework based on a multipopulation method in which subpopulations are responsible for tracking peaks and also gathering characteristic information about them. When the quality of the current robust solution falls below the acceptance threshold, the algorithm chooses the next robust solution based on the collected information. We propose four different strategies to select the next solution. The experimental results on benchmark problems show that our newly proposed methods perform significantly better than existing algorithms. [ABSTRACT FROM AUTHOR]

Periodic vehicle routing problem with time windows (PVRPTWs) is an important combinatorial optimization problem that can be applied in different fields. It is essentially a multiobjective optimization problem due to the problem nature. In this paper, a typical multiobjective PVRPTW with five objectives is first defined and new nonsymmetric real-world multiobjective PVRPTW instances are generated. Then, a hybrid multiobjective memetic algorithm is proposed for solving multiobjective PVRPTW. In the proposed algorithm, a two-phase strategy is devised to improve the comprehensive performance in terms of the convergence and diversity. In this strategy, several extreme solutions near an approximate Pareto front (PF) are identified at Phase I, and then the approximate PF is extended at Phase II. The proposed algorithm is extensively tested on both real-world instances and traditional instances. Experiment results show that the proposed algorithm outperforms two representative competitor algorithms on most of the instances. The effectiveness of the two-phase strategy is also confirmed. [ABSTRACT FROM AUTHOR]

The nonlinear programming (NLP) problem to solve distribution-level optimal power flow (D-OPF) poses convergence issues and does not scale well for unbalanced distribution systems. The existing scalable D-OPF algorithms either use approximations that are not valid for an unbalanced power distribution system, or apply relaxation techniques to the nonlinear power flow equations that do not guarantee a feasible power flow solution. In this paper, we propose scalable D-OPF algorithms that simultaneously achieve optimal and feasible solutions by solving multiple iterations of approximate, or relaxed, D-OPF subproblems of low complexity. The first algorithm is based on a successive linear approximation of the nonlinear power flow equations around the current operating point, where the D-OPF solution is obtained by solving multiple iterations of a linear programming (LP) problem. The second algorithm is based on the relaxation of the nonlinear power flow equations as conic constraints together with directional constraints, which achieves optimal and feasible solutions over multiple iterations of a second-order cone programming (SOCP) problem. It is demonstrated that the proposed algorithms are able to reach an optimal and feasible solution while significantly reducing the computation time as compared to an equivalent NLP D-OPF model for the same distribution system. [ABSTRACT FROM AUTHOR]

Jobs in modern computing clusters have highly diverse processing duration and heterogeneous resource requirements. In this paper, we consider the problem of job scheduling for a computing cluster comprised of multiple servers with heterogeneous computation resources, while taking the different resource demands of the jobs into account. Our focus is to achieve a low overall job response time for the system (which is also referred to as the job flowtime) while providing fairness between small and large jobs. Since the job flowtime minimization problem under multiple (even homogeneous) servers are known to be NP-hard, we propose an approximation algorithm to tackle the original online scheduling problem by adopting the recently-proposed notion of fractional job flowtime as a surrogate objective for minimization. For the general online job arrival case with multi-dimensional resource requirements, we apply Online Convex Optimization (OCO) techniques to design the corresponding scheduling algorithm with performance guarantees. In the single-dimensional resource setting, we show that the dynamic fit of the online version of our approximate algorithm grows only sublinearly with respect to time and derive a bound for its dynamic regret when comparing to its offline counterpart. While the baseline version of our proposed scheduling algorithm assumes the possibilities of job preemption and job migration across different servers, we show that the extent of job preemption and migration can be well controlled by augmenting the objective function with the corresponding switching costs. [ABSTRACT FROM AUTHOR]

It is recognized that existing multi-scale exposure fusion algorithms can be improved using edge-preserving smoothing techniques. However, the complexity of edge-preserving smoothing-based multi-scale exposure fusion is an issue for mobile devices. In this paper, a simpler multi-scale exposure fusion algorithm is designed in YUV color space. The proposed algorithm can preserve details in the brightest and darkest regions of a high dynamic range (HDR) scene and the edge-preserving smoothing-based multi-scale exposure fusion algorithm while avoiding color distortion from appearing in the fused image. The complexity of the proposed algorithm is about half of the edge-preserving smoothing-based multi-scale exposure fusion algorithm. The proposed algorithm is thus friendlier to the smartphones than the edge-preserving smoothing-based multi-scale exposure fusion algorithm. In addition, a simple detail-enhancement component is proposed to enhance fine details of fused images. The experimental results show that the proposed component can be adopted to produce an enhanced image with visibly enhanced fine details and a higher MEF-SSIM value. This is impossible for existing detail enhancement components. Clearly, the component is attractive for PC-based applications. [ABSTRACT FROM AUTHOR]

Subset selection, aiming to select the best subset from a ground set with respect to some objective function, is a fundamental problem with applications in many areas, such as combinatorial optimization, machine learning, data mining, computer vision, information retrieval, etc. Along with the development of data collection and storage, the size of the ground set grows larger. Furthermore, in many subset selection applications, the objective function evaluation is subject to noise. We thus study the large-scale noisy subset selection problem in this paper. The recently proposed DPOSS algorithm based on multiobjective evolutionary optimization is a powerful distributed solver for large-scale subset selection. Its performance, however, has been only validated in the noise-free environment. In this paper, we first prove its approximation guarantee under two common noise models, i.e., multiplicative noise and additive noise, disclosing that the presence of noise degrades the performance of DPOSS largely. Next, we propose a new distributed multiobjective evolutionary algorithm called DPONSS for large-scale noisy subset selection. We prove that the approximation guarantee of DPONSS under noise is significantly better than that of DPOSS. We also conduct experiments on the application of sparse regression, where the objective evaluation is often estimated using a sample data, bringing noise. The results on various real-world data sets, whose size can reach millions, clearly show the excellent performance of DPONSS. [ABSTRACT FROM AUTHOR]

We consider the energy-efficient mobility-aware content caching problem with clustered small cells (SCs) in ultra-dense networks (UDNs). The UDN is regarded as one of the key solutions to overcome the explosive growth of mobile data traffic and consequential energy consumption. In addition, caching the popular contents at the edge of UDNs can further mitigate the challenges of traffic and energy. In this paper, we group the SCs into disjoint clusters and model the mobility of users between clusters as a Markov chain. We formulate the caching problem aimed at minimizing the overall energy consumption for content delivery to the moving users. Then we decompose the complicated task of energy consumption optimization in cache-enabled UDNs into two sub-problems. In the first sub-problem, we formulate the inter-cluster caching problem as minimizing the energy consumption for serving a user request by the macro base station. We show that the first sub-problem is NP-hard, and develop a polynomial-heuristic algorithm. Further, we define the mobility-oblivious version of the sub-problem, and provide a ${\textstyle {1 \over {2k }}}$ -approximation solution for the mobility-aware based on this mobility-oblivious version. In the second sub-problem, we consider the heterogeneity of small cells and define the intra-cluster caching problem. The aim of this sub-problem is the placement of content in different SCs within the cluster so that the energy consumption for the content delivery in that cluster is minimized. In particular, the second sub-problem fits the general framework of the generalized assignment problem, which allows us to exploit its rich literature, to determine the optimal solution for our problem. Through extensive simulations based on real wireless data and human mobility patterns, we demonstrate the benefits of our approach that is even up to 56% better compared to the conventional caching schemes. [ABSTRACT FROM AUTHOR]

This paper is concerned with the network selection for heterogeneous networks. Although many network selection approaches have been developed for heterogeneous networks, few of them explore the network selection problem for integrated cellular and drone-cell networks. Furthermore, most of the existing designs may be unsuitable for the integrated cellular and drone-cell networks owing to the high dynamic drone-cell-user association and the relatively limited drone-cell network capacity. In this paper, we investigate a network selection problem for the integrated cellular and drone-cell networks with a goal of maximizing a proportional fairness function of time average utilities across users under coarse correlated equilibrium constraints and minimum time average utility constraints. To mitigate this challenging problem, we first convert it into a non-linear integer programming (NLIP) problem based on our derived theoretical results. Next, we propose a repeated-stochastic-game-based efficient and fair network selection (RSG-EF) algorithm to alleviate the NLIP problem by leveraging a linear approximation mechanism. Simulation results show that the RSG-EF algorithm can achieve the highest total utility and a high level of fairness across users compared with three benchmark algorithms. [ABSTRACT FROM AUTHOR]

In this paper, the joint spectrum sensing and resource allocation problem is investigated in a multi-band-multi-user cognitive radio (CR) network. Assuming imperfect spectrum sensing information, our goal is to jointly optimize the sensing threshold and power allocation strategy such that the average total throughput of secondary users (SUs) is maximized. In Addition, the power of SUs is constrained to keep the interference introduced to primary users under certain limit, which gives rise to a nonconvex mixed integer non-linear programming (MINLP) optimization problem. Our contribution in this paper is threefold. First, it is illustrated that the dimension of the nonconvex MINLP problem can be significantly reduced, which helps to re-formulate the optimization problem without resorting to integer variables. Second, it is demonstrated that the simplified formulation admits the canonical form of a monotonic optimization, and an $\epsilon $ -optimal solution can be achieved using the polyblock outer approximation algorithm. Third, a practical low-complexity spectrum sensing and resource allocation algorithm is developed to reduce the computational cost. Finally, the effectiveness of proposed algorithms is verified by simulations. [ABSTRACT FROM AUTHOR]

The emergence of big data has caused a dramatic shift in the operating regime for optimization algorithms. The performance bottleneck, which used to be computations, is now often communications. Several gradient compression techniques have been proposed to reduce the communication load at the price of a loss in solution accuracy. Recently, it has been shown how compression errors can be compensated for in the optimization algorithm to improve the solution accuracy. Even though convergence guarantees for error-compensated algorithms have been established, there is very limited theoretical support for quantifying the observed improvements in solution accuracy. In this paper, we show that Hessian-aided error compensation, unlike other existing schemes, avoids accumulation of compression errors on quadratic problems. We also present strong convergence guarantees of Hessian-based error compensation for stochastic gradient descent. Our numerical experiments highlight the benefits of Hessian-based error compensation, and demonstrate that similar convergence improvements are attained when only a diagonal Hessian approximation is used. [ABSTRACT FROM AUTHOR]

Device-to-Device (D2D) communications incorporated with content caching have been regarded as a promising way to offload the cellular traffic data. In this paper, the caching strategy is investigated to maximize the D2D offloading gain with the comprehensive consideration of user collaborative characteristics as well as the physical transmission conditions. Specifically, for a given content, the number of interested users in different groups is different, and users always ask the most trustworthy user in proximity for D2D transmissions. An analytical expression of the D2D success probability is first derived, which represents the probability that the received signal to interference ratio is no less than a given threshold. As the formulated problem is nonconvex, the optimal caching strategy for the special unbiased case is derived in a closed form, and a numerical searching algorithm is proposed to obtain the globally optimal solution for the general case. To reduce the computational complexity, an iterative algorithm based on the asymptotic approximation of the D2D success probability is proposed to obtain the solution that satisfies the Karush-Kuhn-Tucker conditions. The simulation results verify the effectiveness of the analytical results and show that the proposed algorithm outperforms the existing schemes in terms of offloading gain. [ABSTRACT FROM AUTHOR]

Networks are prevalent and often collected from multiple sources in many high-impact domains, which facilitate many emerging applications that require the connections across multiple networks. Network alignment (i.e., to find the node correspondence between different networks) has become the very first step in many applications and thus has been studied in decades. Although some existing works can use the attribute information as part of the alignment process, they still have certain limitations. For example, some existing network alignment methods can use node attribute similarities as part of the prior alignment information, whereas most of them solely explore the topology consistency without the consistency among attributes of the underlying networks. On the other hand, traditional graph matching methods encode both the node and edge attributes (and possibly the topology) into an affinity matrix and formulate it as a constrained nonconvex quadratic maximization problem. However, these methods cannot scale well to the large-scale networks. In this paper, we propose a family of network alignment algorithms (FINAL) to efficiently align the attributed networks. The key idea is to leverage the node/edge attribute information to guide the (topology-based) alignment process. We formulate this problem as a convex quadratic optimization problem, and develop effective and efficient algorithms to solve it. Moreover, we derive FINAL On-Query, an online variant of FINAL that can find similar nodes for the query nodes across networks. We perform extensive evaluations on real networks to substantiate the superiority of our proposed approaches. [ABSTRACT FROM AUTHOR]

Recent theory of mapping an image into a structured low-rank Toeplitz or Hankel matrix has become an effective method to restore images. In this paper, we introduce a generalized structured low-rank algorithm to recover images from their undersampled Fourier coefficients using infimal convolution regularizations. The image is modeled as the superposition of a piecewise constant component and a piecewise linear component. The Fourier coefficients of each component satisfy an annihilation relation, which results in a structured Toeplitz matrix. We exploit the low-rank property of the matrices to formulate a combined regularized optimization problem. In order to solve the problem efficiently and to avoid the high-memory demand resulting from the large-scale Toeplitz matrices, we introduce a fast and a memory-efficient algorithm based on the half-circulant approximation of the Toeplitz matrix. We demonstrate our algorithm in the context of single and multi-channel MR images recovery. Numerical experiments indicate that the proposed algorithm provides improved recovery performance over the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

In this paper, we study the cost-aware target viral marketing (CTVM) problem, a generalization of influence maximization. CTVM asks for the most cost-effective users to influence the most relevant users. In contrast to the vast literature, we attempt to offer exact solutions. As the problem is NP-hard, thus, exact solutions are intractable, we propose TipTop, a $(1-\epsilon)$ -optimal solution for arbitrary $\epsilon >0$ that scales to very large networks, such as Twitter. At the heart of TipTop lies an innovative technique that reduces the number of samples as much as possible. This allows us to exactly solve CTVM on a much smaller space of generated samples using integer programming. Furthermore, TipTop lends a tool for researchers to benchmark their solutions against the optimal one in large-scale networks, which is currently not available. [ABSTRACT FROM AUTHOR]

In quantitative magnetic resonance ${T}_{\textsf {1}}$ mapping, the variable flip angle (VFA) steady state spoiled gradient recalled echo (SPGR) imaging technique is popular as it provides a series of high resolution ${T}_{\textsf {1}}$ weighted images in a clinically feasible time. Fast, linear methods that estimate ${T}_{\textsf {1}}$ maps from these weighted images have been proposed, such as DESPOT1 and iterative re-weighted linear least squares. More accurate, non-linear least squares (NLLS) estimators are in play, but these are generally much slower and require careful initialization. In this paper, we present NOVIFAST, a novel NLLS-based algorithm specifically tailored to VFA SPGR ${T}_{\textsf {1}}$ mapping. By exploiting the particular structure of the SPGR model, a computationally efficient, yet accurate and precise ${T}_{\textsf {1}}$ map estimator is derived. Simulation and in vivo human brain experiments demonstrate a twenty-fold speed gain of NOVIFAST compared with conventional gradient-based NLLS estimators while maintaining a high precision and accuracy. Moreover, NOVIFAST is eight times faster than the efficient implementations of the variable projection (VARPRO) method. Furthermore, NOVIFAST is shown to be robust against initialization. [ABSTRACT FROM AUTHOR]

Rate splitting (RS) is a potentially powerful and flexible technique for multi-antenna downlink transmission. In this paper, we address several technical challenges towards its practical implementation for beyond 5G systems. To this end, we focus on a single-cell system with a multi-antenna base station (BS) and $K$ single-antenna receivers. We consider RS in its most general form with $2^{K}-1$ streams, and joint decoding to fully exploit the potential of RS. First, we investigate the achievable rates under joint decoding and formulate the precoder design problems to maximize a general utility function, or to minimize the transmit power under pre-defined rate targets. Building upon the concave-convex procedure (CCCP), we propose precoder design algorithms for an arbitrary number of users. Our proposed algorithms approximate the intractable non-convex problems with a number of successively refined convex problems, and provably converge to stationary points of the original problems. Then, to reduce the decoding complexity, we consider the optimization of the precoder and the decoding order under successive decoding. Further, we propose a stream selection algorithm to reduce the number of precoded signals. With a reduced number of streams and successive decoding at the receivers, our proposed algorithm can even be implemented when the number of users is relatively large, whereas the complexity was previously considered as prohibitively high in the same setting. Finally, we propose a simple adaptation of our algorithms to account for the imperfection of the channel state information at the transmitter. Numerical results demonstrate that the general RS scheme provides a substantial performance gain as compared to state-of-the-art linear precoding schemes, especially with a moderately large number of users. [ABSTRACT FROM AUTHOR]

The matrix decomposing into a sum of low-rank and sparse components has found extensive applications in many areas including video surveillance, computer vision, and medical imaging. In this paper, we propose a new algorithm for recovery of low rank and sparse components of a given matrix. We have also proved the convergence of the proposed algorithm. The simulation results with synthetic and real signals such as image and video signals indicate that the proposed algorithm has a better performance with lower run-time than the conventional methods. [ABSTRACT FROM AUTHOR]

This paper investigates UAV-to-UAV (unmanned aerial vehicle) communications with a turning angle constraint, in which the information transmission time is minimized by joint UAV flight path planning and transmit power control. Considering the communication throughput requirement, peak transmit power, energy consumption, UAV flight speed, collision avoidance, and turning angle constraints, the joint UAV flight path planning and transmit power control problem is formulated as a non-convex optimization. Different from majority of prior studies that separately optimize the UAV flight path and transmit power, we develop a successive convex approximation based time minimization algorithm that simultaneously updates the UAV path and transmit power. We further prove that the proposed algorithm is guaranteed to converge. In addition, an initialization scheme is designed to start the proposed time minimization algorithm by solving an energy minimization problem. Simulation results verify that the proposed time minimization algorithm can shorten the completion time length and reduce the computational complexity compared with state of the art methods. [ABSTRACT FROM AUTHOR]

In this paper, we investigate distributed inference schemes, over binary-valued Markov random fields, which are realized by the belief propagation (BP) algorithm. We first show that a decision variable obtained by the BP algorithm in a network of distributed agents can be approximated by a linear fusion of all the local log-likelihood ratios. The proposed approach clarifies how the BP algorithm works, simplifies the statistical analysis of its behavior, and enables us to develop a performance optimization framework for the BP-based distributed inference systems. Next, we propose a blind learning-adaptation scheme to optimize the system performance when there is no information available a priori describing the statistical behavior of the wireless environment concerned. In addition, we propose a blind threshold adaptation method to guarantee a certain performance level in a BP-based distributed detection system. To clarify the points discussed, we design a novel linear-BP-based distributed spectrum sensing scheme for cognitive radio networks and illustrate the performance improvement obtained, over an existing BP-based detection method, via computer simulations. [ABSTRACT FROM AUTHOR]

This work addresses the robust principal component analysis (PCA) problem using generalized nonoconvex regularization for low-rank and sparsity promotion. While the popular nuclear and $\ell _1$ -norm penalties have a bias problem, nonconvex regularization can alleviate the bias problem and can be expected to achieve better performance. In this paper, a proximal block coordinate descent (BCD) algorithm is used to efficiently solve the nonconvex regularized robust PCA problem. It is globally convergent under weak conditions. Further, for a popular class of penalties having discontinuous threshoding functions, we establish the convergence to a restricted strictly local minimizer and, also, a local linear convergence rate for the proximal BCD algorithm. Moreover, convergence to a local minimizer has been derived for hard-thresholding. Our result is the first on nonconvex robust PCA with established convergence to strictly local minimizer with local linear convergence rate. Numerical experiments have been provided to demonstrate the performance of the new algorithm. [ABSTRACT FROM AUTHOR]

This paper focuses on precoding design for sum-rate maximization while considering the effects of residual self-interference for multiuser multiple-input–multiple-output (MU-MIMO) full-duplex (FD) systems. The problem formulation leads to a nonconvex matrix-variable optimization problem, where we develop two efficient sum-rate maximization algorithms using sequential convex programming (SCP), namely, the difference of convex functions (DC)-based and the sequential convex approximations for matrix-variable programming (SCAMP) algorithms. In addition, we derive the achievable sum rate under the effect of residual self-interference. Simulation results show that, even in cases of high self-interference and high estimation error, the SCAMP algorithm provides approximately 20%–30% sum-rate improvements over both conventional optimized half-duplex (HD) transmission and the existing state-of-the-art FD algorithm in a wide range of scenarios. Finally, the convergence results indicate that the DC-based algorithm tends to initially give the best performance; however, at convergence, the SCAMP algorithm tends to significantly outperform the other algorithms. [ABSTRACT FROM AUTHOR]

Owing to its good compression ability, discrete cosine transform (DCT) is widely used in signal processing, including high-efficiency video coding (HEVC). For efficient implementation, approximate DCTs and integer DCTs were proposed, but the challenge in the tradeoff between coding performance and implementation cost always exists. To solve this problem, this article proposes a new algorithm to derive Int-DCT with good coding performance and low hardware cost. To address the multiobjective optimization problem, we extract information from the initially shortlisted transform matrices to derive the weighting factors for the different matrices properties. Those properties having a higher impact to the performance are assigned with higher weights and vice versa. Subsequently, an efficient search algorithm is proposed to shortlist those candidates which can lead to better coding performance evaluated by the proposed measure. Last but not the least, a splitting method is proposed to search the efficient extended double-base number system (EDBNS) representation of the coefficients for hardware implementation and select the solution which encounters the lowest cost. The experimental results in ASIC demonstrate that the implementation area and power cost of designs by the new algorithm are reduced by at least 11.1% and 17.5% for 32-point transform, respectively, over other competing algorithms. Meanwhile, the transforms generated by the proposed algorithm causes negligible impact to coding performance compared to the original transforms in HEVC. [ABSTRACT FROM AUTHOR]

In this article, we discuss a sequential algorithm for the computation of a minimum-time speed profile over a given path, under velocity, acceleration, and jerk constraints. Such a problem arises in industrial contexts, such as automated warehouses, where LGVs need to perform assigned tasks as fast as possible in order to increase productivity. It can be reformulated as an optimization problem with a convex objective function, linear velocity and acceleration constraints, and nonconvex jerk constraints, which, thus, represent the main source of the difficulty. While existing nonlinear programming (NLP) solvers can be employed for the solution of this problem, it turns out that the performance and robustness of such solvers can be enhanced by the sequential line-search algorithm proposed in this article. At each iteration, a feasible direction, with respect to the current feasible solution, is computed, and a step along such direction is taken in order to compute the next iterate. The computation of the feasible direction is based on the solution of a linearized version of the problem, and the solution of the linearized problem, through an approach that strongly exploits its special structure, represents the main contribution of this work. The efficiency of the proposed approach with respect to existing NLP solvers is proven through different computational experiments. Note to Practitioners—This article was motivated by the needs of LGV manufacturers. In particular, it presents an algorithm for computing the minimum-time speed law for an LGV along a preassigned path, respecting assigned velocity, acceleration, and jerk constraints. The solution algorithm should be: 1) fast, since speed planning is made continuously throughout the workday, not only when an LGV receives a new task but also during the execution of the task itself, since conditions may change, e.g., if the LGV has to be halted for security reasons and 2) reliable, i.e., it should return solutions of high quality, because a better speed profile allows to save time and even small percentage improvements, say a 5% improvement, has a considerable impact on the productivity of the warehouse, and, thus, determines a significant economic gain. The algorithm that we propose meets these two requirements, and we believe that it can be a useful tool for LGV manufacturers and users. It is obvious that the proposed method also applies to the speed planning problem for vehicles other than LGVs, e.g., road vehicles. [ABSTRACT FROM AUTHOR]

Small cells are introduced to cellular systems to enhance coverage and improve capacity. Densely deploying small cells can not only offload the traffic of macrocells but also provide an energy- and cost-efficient way to meet the sharp increase in traffic demands in mobile networks. However, such a cell deployment paradigm also leads to heterogeneous network (HetNet) infrastructure and raises new challenges for cell planning. In this paper, we study the cell planning issue in the HetNet. Our optimization task is to select a subset of candidate sites to deploy macro or small cells to minimize the total cost of ownership (TCO) or the energy consumption of the cellular system while satisfying practical constraints. We introduce approximation algorithms to cope with two different cell-planning cases, which are both NP-hard. First, we discuss the macrocell-only case. Our proposed algorithm achieves an approximation ratio of O(\log R) in this scenario, where R is the maximum achievable capacity of macrocells. Then, we introduce an O(\log \widetildeR)-approximation algorithm to the small-cell scenario, where \widetildeR is the maximum achievable capacity of a macrocell with small cells overlaid on it. Numerical results indicate that the HetNet can significantly reduce the TCO and the energy consumption of the cellular system. [ABSTRACT FROM PUBLISHER]

This paper considers multirobot task allocation problems where the estimated costs for performing tasks are interrelated, and the overall team objective need not be a standard sum-of-costs (or utilities) model, enabling straightforward treatment of the additional costs incurred by resource contention. In the model we introduce, a team may choose one of a set of shared resources to perform a task (e.g., several routes to reach a destination), and interference is modeled when multiple robots use the same resource. We show that the general problem is NP-hard, and investigate specialized subinstances with particular cost structures. For the general problem, we describe an exact algorithm which finds an optimal assignment in a reasonable time on small instances. Aiming at larger problems, we turn two particular subinstances, introducing an two algorithms that find assignments quickly even for problems of considerable size, the first being optimal, the second being an approximation algorithm but also producing high-quality solutions with bounded suboptimality. [ABSTRACT FROM PUBLISHER]

In this paper, four iterative algorithms for learning analysis operators are presented. They are built upon the same optimization principle underlying both Analysis K-SVD and Analysis SimCO. The forward and sequential analysis operator learning (AOL) algorithms are based on projected gradient descent with optimally chosen step size. The implicit AOL algorithm is inspired by the implicit Euler scheme for solving ordinary differential equations and does not require to choose a step size. The fourth algorithm, singular value AOL, uses a similar strategy as Analysis K-SVD while avoiding its high computational cost. All algorithms are proven to decrease or preserve the target function in each step and a characterization of their stationary points is provided. Further they are tested on synthetic and image data, compared to Analysis SimCO and found to give better recovery rates and faster decay of the objective function, respectively. In a final denoising experiment the presented algorithms are again shown to perform similar to or better than the state-of-the-art algorithm ASimCO. [ABSTRACT FROM AUTHOR]

This paper considers the reconstruction of a discrete-valued random vector from possibly underdetermined linear measurements using sum-of-absolute-value (SOAV) optimization. The proposed algorithm, referred to as discreteness-aware approximate message passing (DAMP), is based on the idea of approximate message passing (AMP), which has been originally proposed for compressed sensing. The DAMP algorithm has low computational complexity and its performance in the large system limit can be predicted analytically via state evolution framework, where we provide a condition for the exact reconstruction with DAMP in the noise-free case. From the analysis, we also propose a method to determine the parameters of the SOAV optimization. Moreover, based on the state evolution, we provide Bayes optimal DAMP, which has the minimum mean-square-error at each iteration of the algorithm. Simulation results show that the DAMP algorithms can reconstruct the discrete-valued vector from underdetermined linear measurements and the empirical performance agrees with our theoretical results in large-scale systems. When the problem size is not large enough, the SOAV optimization with the proposed parameters can achieve better performance than the DAMP algorithms for high signal-to-noise ratio. [ABSTRACT FROM AUTHOR]

Metaheuristics have recently become the forefront of the current research as a powerful way to deal with many electromagnetic optimization problems. Based on the simulation of the herding behavior of krill individuals, a krill herd (KH) algorithm was recently proposed to solve optimization problems. In order to extend the classical mono-objective KH algorithm approach, this paper proposes a new multiobjective KH (MOKH) algorithm and a modified MOKH approach using the beta distribution in the inertia weight tuning. Numerical results on a multiobjective constrained brushless direct current motor design problem show that the evaluated MOKH algorithms present a promising performance. [ABSTRACT FROM AUTHOR]

The classic approach for estimating parameters of a model using historical data is to solve a nonlinear least squares optimization problem using numerical methods. We develop an I-frame methodology to solve the nonlinear least squares problem quickly which can be applied to both offline and online (where data is streamed in real time) parameter estimation. Using the concept of I-frames from imaging and animation, we approximate a solution to the nonlinear least squares problem via a two-step process, an I-frame optimization, and an incremental optimization. The I-frame optimization solves for the parameters using a subset of data points and the incremental optimization adjusts the parameters in between the I-frames. We show that the criterion of generating I-frames can affect the average squared error of the final solution. Our methodology benefits from being scalable as the number of parameters and amount of data increases with an appropriate I-frame generation criterion. [ABSTRACT FROM AUTHOR]

In this paper, we tackle the problem of multi-cell resource scheduling, where the objective is to maximize the weighted sum-rate through inter-cell interference coordination (ICIC). The blanking method is used to mitigate the inter-cell interference, where a resource is either used with a predetermined transmit power or not used at all, i.e., blanked. This problem is known to be strongly NP-hard, which means that it is not only hard to solve in polynomial time, but it is also hard to find an approximation algorithm with guaranteed optimality gap. In this work, we identify special scenarios where a polynomial-time algorithm can be constructed to solve this problem with theoretical guarantees. In particular, we define a dominant interference environment, in which for each user the received power from each interferer is significantly greater than the aggregate received power from all other weaker interferers. We show that the originally strongly NP-hard problem can be tightly relaxed to a linear programming problem in a dominant interference environment. Consequently, we propose a polynomial time distributed algorithm that is not only guaranteed to be tight in a dominant interference environment, but which also computes an upper bound on the optimality gap without additional computational complexity. The proposed scheme is based on the primal-decomposition method, where the problem is divided into a master-problem and multiple subproblems. We solve the master-problem iteratively using the projected-subgradient method. We also show that each subproblem has a special network flow structure. By exploiting this network structure, each subproblem is solved using the network-based optimization methods, which significantly reduces the complexity in comparison to the general-purpose convex or linear optimization methods. In comparison with baseline schemes, simulation results of the International Mobile Telecommunications-Advanced (IMT-Advanced) scenarios show that the proposed scheme achieves higher gains in aggregate throughput, cell-edge throughput, and outage probability. [ABSTRACT FROM PUBLISHER]

We propose a fast and scalable polyatomic Frank-Wolfe (P-FW) algorithm for the resolution of high-dimensional LASSO regression problems. This algorithm improves upon traditional Frank-Wolfe methods by considering generalized greedy steps with polyatomic (i.e. linear combinations of multiple atoms) update directions, hence allowing for a more efficient exploration of the search space. To preserve sparsity of the intermediate iterates, we re-optimize the LASSO problem over the set of selected atoms at each iteration. For efficiency reasons, the accuracy of this re-optimization step is relatively low for early iterations and gradually increases with the iteration count. We provide convergence guarantees for our algorithm and validate it in simulated compressed sensing setups. Our experiments reveal that P-FW outperforms state-of-the-art methods in terms of runtime, both for FW methods and optimal first-order proximal gradient methods such as the Fast Iterative Soft-Thresholding Algorithm (FISTA). [ABSTRACT FROM AUTHOR]

Random projection has been widely used in data classification. It maps high-dimensional data into a low-dimensional subspace in order to reduce the computational cost in solving the related optimization problem. While previous studies are focused on analyzing the classification performance in the low-dimensional space, in this paper, we consider the recovery problem, i.e., how to accurately recover the optimal solution to the original high-dimensional optimization problem based on the solution learned after random projection. We present a simple algorithm, termed dual random projection, which uses the dual solution of the low-dimensional optimization problem to recover the optimal solution to the original problem. Our theoretical analysis shows that with a high probability, the proposed algorithm is able to accurately recover the optimal solution to the original problem, provided that the data matrix is (approximately) low-rank and/or optimal solution is (approximately) sparse. We further show that the proposed algorithm can be applied iteratively to reducing the recovery error exponentially. [ABSTRACT FROM PUBLISHER]

In heterogeneous networks (HetNets), the load between macrocell base stations (MBSs) and small-cell base stations (SBSs) is imbalanced due to their different transmission powers and locations. This load imbalance significantly impacts system performance and affects the experience of mobile users (MUs) with different priorities. In this paper, we aim to distributively optimize the user association in HetNets with various user priorities to solve the load balancing problem. Since the user association is a binary matching problem, which is NP-hard, we propose a distributed belief propagation (BP) algorithm to approach the optimal solution. We first develop a factor graph model, using the network topology, to represent this user association problem. With this factor graph, we propose a novel distributed BP algorithm by adopting the proportional fairness as the objective. Next, we theoretically prove the existence of the fixed point in our BP algorithm. To be more practical, we develop an approximation method to significantly reduce the computational and communication complexity of the BP algorithm. Furthermore, we analyze some properties of the factor graph relevant to the performance of the BP algorithm using the stochastic geometry. Simulation results show that 1) the proposed BP algorithm well approaches the optimal system performance and achieves a much better performance compared with other association schemes and that 2) the analytical results on the average degree distribution and sparsity of the factor graph match with those obtained from the Monte Carlo simulations. [ABSTRACT FROM PUBLISHER]

We consider stochastic optimization of a smooth non-convex loss function with a convex non-smooth regularizer. In the online setting, where a single sample of the stochastic gradient of the loss is available at every iteration, the problem can be solved using the proximal stochastic gradient descent (SGD) algorithm and its variants. However in many problems, especially those arising in communications and signal processing, information beyond the stochastic gradient may be available thanks to the structure of the loss function. By constructing a strongly convex surrogate of the loss function at every iteration, the stochastic SCA algorithms can exploit the structural properties of the loss function. In this work, we take a closer look at the stochastic SCA algorithm and develop its asynchronous variant which can be used for resource allocation in wireless networks. While the stochastic SCA algorithm is known to converge asymptotically, its iteration complexity has not been well-studied, and is the focus of the current work. The insights obtained from the non-asymptotic analysis allow us to develop a more practical asynchronous variant of the stochastic SCA algorithm which allows the use of surrogate calculated in earlier iterations. We characterize precise bound on the maximum delay the algorithm can tolerate, while still achieving the same convergence rate. We apply the algorithm to the problem of linear precoding in wireless sensor networks, where it can be implemented at low complexity but is shown to perform well in practice. [ABSTRACT FROM AUTHOR]