Search

In this paper, we study the lower iteration complexity bounds for finding the saddle point of a strongly convex and strongly concave saddle point problem: min x max y F (x , y) . We restrict the classes of algorithms in our investigation to be either pure first-order methods or methods using proximal mappings. For problems with gradient Lipschitz constants ( L x , L y and L xy ) and strong convexity/concavity constants ( μ x and μ y ), the class of pure first-order algorithms with the linear span assumption is shown to have a lower iteration complexity bound of Ω L x μ x + L xy 2 μ x μ y + L y μ y · ln 1 ϵ , where the term L xy 2 μ x μ y explains how the coupling influences the iteration complexity. Under several special parameter regimes, this lower bound has been achieved by corresponding optimal algorithms. However, whether or not the bound under the general parameter regime is optimal remains open. Additionally, for the special case of bilinear coupling problems, given the availability of certain proximal operators, a lower bound of Ω L xy 2 μ x μ y · ln (1 ϵ) is established under the linear span assumption, and optimal algorithms have already been developed in the literature. By exploiting the orthogonal invariance technique, we extend both lower bounds to the general pure first-order algorithm class and the proximal algorithm class without the linear span assumption. As an application, we apply proper scaling to the worst-case instances, and we derive the lower bounds for the general convex concave problems with μ x = μ y = 0 . Several existing results in this case can be deduced from our results as special cases. [ABSTRACT FROM AUTHOR]

We consider a class of popular distributed non-convex optimization problems, in which agents connected by a network $\mathcal {G}$ collectively optimize a sum of smooth (possibly non-convex) local objective functions. We address the following question: if the agents can only access the gradients of local functions, what are the fastest rates that any distributed algorithms can achieve, and how to achieve those rates. First, we show that there exist difficult problem instances, such that it takes a class of distributed first-order methods at least $\mathcal {O}(1/\sqrt{\xi ({ \mathcal {G} })} \times \bar{L} /{\epsilon })$ communication rounds to achieve certain $\epsilon$ -solution [where $\xi ({ \mathcal {G} })$ denotes the spectral gap of the graph Laplacian matrix, and $\bar{L}$ is some Lipschitz constant]. Second, we propose (near) optimal methods whose rates match the developed lower rate bound (up to a ploylog factor). The key in the algorithm design is to properly embed the classical polynomial filtering techniques into modern first-order algorithms. To the best of our knowledge, this is the first time that lower rate bounds and optimal methods have been developed for distributed non-convex optimization problems. [ABSTRACT FROM AUTHOR]

Consider the classical problem of solving a general linear system of equations A x = b . It is well known that the (successively over relaxed) Gauss–Seidel scheme and many of its variants may not converge when A is neither diagonally dominant nor symmetric positive definite. Can we have a linearly convergent G–S type algorithm that works for anyA? In this paper we answer this question affirmatively by proposing a doubly stochastic G–S algorithm that is provably linearly convergent (in the mean square error sense) for any feasible linear system of equations. The key in the algorithm design is to introduce a nonuniform double stochastic scheme for picking the equation and the variable in each update step as well as a stepsize rule. These techniques also generalize to certain iterative alternating projection algorithms for solving the linear feasibility problem A x ≤ b with an arbitrary A, as well as high-dimensional minimization problems for training over-parameterized models in machine learning. Our results demonstrate that a carefully designed randomization scheme can make an otherwise divergent G–S algorithm converge. [ABSTRACT FROM AUTHOR]

In this paper, we propose a perturbed proximal primal–dual algorithm (PProx-PDA) for an important class of linearly constrained optimization problems, whose objective is the sum of smooth (possibly nonconvex) and convex (possibly nonsmooth) functions. This family of problems can be used to model many statistical and engineering applications, such as high-dimensional subspace estimation and the distributed machine learning. The proposed method is of the Uzawa type, in which a primal gradient descent step is performed followed by an (approximate) dual gradient ascent step. One distinctive feature of the proposed algorithm is that the primal and dual steps are both perturbed appropriately using past iterates so that a number of asymptotic convergence and rate of convergence results (to first-order stationary solutions) can be obtained. Finally, we conduct extensive numerical experiments to validate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

We consider solving a convex optimization problem with possibly stochastic gradient, and over a randomly time-varying multiagent network. Each agent has access to some local objective function, and it only has unbiased estimates of the gradients of the smooth component. We develop a dynamic stochastic proximal-gradient consensus algorithm, with the following key features: 1) it works for both the static and certain randomly time-varying networks; 2) it allows the agents to utilize either the exact or stochastic gradient information; 3) it is convergent with provable rate. In particular, the proposed algorithm converges to a global optimal solution, with a rate of \mathcal O(1/r) [resp. \mathcal O(1/\sqrtr)] when the exact (resp. stochastic) gradient is available, where $r$ is the iteration counter. Interestingly, the developed algorithm establishes a close connection among a number of (seemingly unrelated) distributed algorithms, such as the EXTRA, the PG-EXTRA, the IC/IDC-ADMM, the DLM, and the classical distributed subgradient method. [ABSTRACT FROM PUBLISHER]

In this paper, we provide a unified iteration complexity analysis for a family of general block coordinate descent methods, covering popular methods such as the block coordinate gradient descent and the block coordinate proximal gradient, under various different coordinate update rules. We unify these algorithms under the so-called block successive upper-bound minimization (BSUM) framework, and show that for a broad class of multi-block nonsmooth convex problems, all algorithms covered by the BSUM framework achieve a global sublinear iteration complexity of $$\mathcal{{O}}(1/r)$$ , where r is the iteration index. Moreover, for the case of block coordinate minimization where each block is minimized exactly, we establish the sublinear convergence rate of O(1/ r) without per block strong convexity assumption. [ABSTRACT FROM AUTHOR]

We analyze the convergence rate of the alternating direction method of multipliers (ADMM) for minimizing the sum of two or more nonsmooth convex separable functions subject to linear constraints. Previous analysis of the ADMM typically assumes that the objective function is the sum of only two convex functions defined on two separable blocks of variables even though the algorithm works well in numerical experiments for three or more blocks. Moreover, there has been no rate of convergence analysis for the ADMM without strong convexity in the objective function. In this paper we establish the global R-linear convergence of the ADMM for minimizing the sum of any number of convex separable functions, assuming that a certain error bound condition holds true and the dual stepsize is sufficiently small. Such an error bound condition is satisfied for example when the feasible set is a compact polyhedron and the objective function consists of a smooth strictly convex function composed with a linear mapping, and a nonsmooth $$\ell _1$$ regularizer. This result implies the linear convergence of the ADMM for contemporary applications such as LASSO without assuming strong convexity of the objective function. [ABSTRACT FROM AUTHOR]

The performance of full-duplex (FD) relay systems can be greatly impacted by the self-interference (SI) at relays. By exploiting multiple antennas, the spectral efficiency of FD relay systems can be enhanced through spatial SI mitigation. This paper studies joint source transmit beamforming and relay processing to achieve rate maximization for FD multiple-input–multiple-output (MIMO) amplify-and-forward (AF) relay systems with consideration of relay processing delay. The problem is difficult to solve mainly due to the SI constraint induced by the relay processing delay. In this paper, we first present a sufficient condition under which the relay amplification matrix has rank-one structure. Then, for the case of rank-one amplification matrix, the rate maximization problem is equivalently simplified into an unconstrained problem that can be locally solved using the gradient ascent method. Next, we propose a penalty-based algorithmic framework, named P-BSUM, for a class of constrained optimization problems that have difficult equality constraints in addition to some convex constraints. By rewriting the rate maximization problem with a set of auxiliary variables, we apply the P-BSUM algorithm to the rate maximization problem in the general case. Finally, numerical results validate the efficiency of the proposed algorithms and show that the joint source-relay design approach under the rank-one assumption could be strictly suboptimal as compared to the P-BSUM-based joint source-relay design approach. [ABSTRACT FROM AUTHOR]

We consider the downlink coordinated beamforming problem in a cellular network in which the base stations (BSs) are equipped with multiple antennas and each user is equipped with a single antenna. The BSs cooperate in sharing their local interference information, and they aim to maximize the sum-rate of the users in the network. A decentralized interference pricing beamforming (IPBF) algorithm is proposed to identify the coordinated beamformer, where a BS is penalized according to the interference it creates to its peers. We show that the decentralized pricing mechanism converges to an interference equilibrium, which is a KKT point of the sum-rate maximization problem. The proofs rely on the identification of rank-1 solutions of each BSs' interference-penalized rate maximization problem. Numerical results show that the proposed iterative mechanism reduces significantly the exchanged information with respect to other state-of-the-art beamforming algorithms with very little sum-rate loss. The version of the algorithm that limits the coordination to a cluster of base stations (IPBF-L) is shown to have very small sum-rate loss with respect to the full coordinated algorithm with much less backhaul information exchange. [ABSTRACT FROM AUTHOR]

Abstract: We consider the problem of linear transceiver design to achieve max–min fairness in a downlink MIMO multicell network. This problem can be formulated as maximizing the minimum rate among all the users in an interfering broadcast channel (IBC). In this paper we show that when the number of antennas is at least two at each of the transmitters and the receivers, the min rate maximization problem is NP-hard in the number of users. Moreover, we develop a low-complexity algorithm for this problem by iteratively solving a sequence of convex subproblems. We theoretically establish the global convergence of the proposed algorithm to the set of stationary points, which may be suboptimal due to the non-convexity of the original minimum rate maximization problem. Numerical simulations show that this algorithm is efficient in achieving fairness among all the users. [Copyright &y& Elsevier]

We consider the distributed uplink resource allocation problem in a multi-carrier wireless network with multiple access points (APs). Each mobile user can optimize its own transmission rate by selecting a suitable AP and by controlling its transmit power. Our objective is to devise suitable algorithms by which mobile users can jointly perform these tasks in a distributed manner. Our approach relies on a game theoretic formulation of the joint power control and AP selection problem. In the proposed game, each user is a player with an associated strategy containing a discrete variable (the AP selection decision) and a continuous vector (the power allocation among multiple channels). We provide characterizations of the Nash Equilibrium of the proposed game. We present a novel algorithm named Joint Access Point Selection and Power Allocation (JASPA) and its various extensions (with different update schedules) that allow the users to efficiently optimize their rates. Finally, we study the properties of the proposed algorithms as well as their performance via extensive simulations. [ABSTRACT FROM PUBLISHER]

Dynamically allocating the available spectrum in the cognitive radio networks (CRN) where licensed users allow unlicensed users to make use of part of their allocated spectrum is a complex task. It requires, among others, that the interference generated by the CRN is well controlled. In this paper we address this problem for a particular type of interference constraints, in which the aggregated powers of the interference generated by the unlicensed network are required not to exceed certain thresholds. Traditional distributed “water filling” based spectrum allocation schemes are not well equipped to enforce such interference constraints. A natural extension to these algorithms involves penalizing each unlicensed user with a set of time-varying prices based upon its contribution to the total interference. In this context, the network is in equilibrium if: i) all interference constraints are satisfied; and ii) no unlicensed user has an incentive to alter its own transmission power levels. In this paper, we propose a distributed algorithm to compute such equilibrium in two distinctive network configurations: 1) the unlicensed users are mobile devices that communicate with a common access point; 2) the unlicensed users are tranceiver pairs. We show that our algorithm converges to a set of equilibria in the first scenario without restriction on the channel gains, and it converges to an equilibrium point in the second scenario under a relatively mild condition related to the channel gains. [ABSTRACT FROM AUTHOR]

The convergence properties of the iterative water-filling (IWF) based algorithms have been derived in the ideal situation where the transmitters in the network are able to obtain the exact value of the interference plus noise (IPN) experienced at the corresponding receivers in each iteration of the algorithm. However, these algorithms are not robust because they diverge when there is time-varying estimation error of the IPN, a situation that arises in real communication system. In this correspondence, we propose an algorithm that possesses convergence guarantees in the presence of various forms of such time-varying error. Moreover, we also show by simulation that in scenarios where the interference is strong, the conventional IWF diverges while our proposed algorithm still converges. [ABSTRACT FROM AUTHOR]

The spectrum cartography (SC) technique constructs multi-domain (e.g., frequency, space, and time) radio frequency (RF) maps from limited measurements, which can be viewed as an ill-posed tensor completion problem. Model-based cartography techniques often rely on handcrafted priors (e.g., sparsity, smoothness and low-rank structures) for the completion task. Such priors may be inadequate to capture the essence of complex wireless environments—especially when severe shadowing happens. To circumvent such challenges, offline-trained deep neural models of radio maps were considered for SC, as deep neural networks (DNNs) are able to “learn” intricate underlying structures from data. However, such deep learning (DL)-based SC approaches encounter serious challenges in both off-line model learning (training) and completion (generalization), possibly because the latent state space for generating the radio maps is prohibitively large. In this work, an emitter radio map disaggregation-based approach is proposed, under which only individual emitters’ radio maps are modeled by DNNs. This way, the learning and generalization challenges can both be substantially alleviated. Using the learned DNNs, a fast nonnegative matrix factorization-based two-stage SC method and a performance-enhanced iterative optimization algorithm are proposed. Theoretical aspects—such as recoverability of the radio tensor, sample complexity, and noise robustness—under the proposed framework are characterized, and such theoretical properties have been elusive in the context of DL-based radio tensor completion. Experiments using synthetic and real-data from indoor and heavily shadowed environments are employed to showcase the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

Unmanned aerial vehicle (UAV) swarm has emerged as a promising novel paradigm to achieve better coverage and higher capacity for future wireless network by exploiting the more favorable line-of-sight (LoS) propagation. To reap the potential gains of UAV swarm, the remote control signal sent by ground control unit (GCU) is essential, whereas the control signal quality are susceptible in practice due to the effect of the adjacent channel interference (ACI) and the external interference (EI) from radiation sources distributed across the region. To tackle these challenges, this paper considers priority-aware resource coordination in a multi-UAV communication system, where multiple UAVs are controlled by a GCU to perform certain tasks with a pre-defined trajectory. Specifically, we maximize the minimum signal-to-interference-plus-noise ratio (SINR) among all the UAVs by jointly optimizing channel assignment and power allocation strategy under stringent resource availability constraints. According to the intensity of ACI, we consider the corresponding problem in two scenarios, i.e., Null-ACI and ACI systems. By virtue of the particular problem structure in Null-ACI case, we first recast the formulation into an equivalent yet more tractable form and obtain the global optimal solution via Hungarian algorithm. For general ACI systems, we develop an efficient iterative algorithm for its solution based on the alternating optimization methods. Extensive simulation results demonstrate that the proposed algorithms can significantly enhance the minimum SINR among all the UAVs and adapt the allocation of communication resources to diverse mission priority. [ABSTRACT FROM AUTHOR]

Motivated by a growing list of nontraditional statistical estimation problems of the piecewise kind, this paper provides a survey of known results supplemented with new results for the class of piecewise linear-quadratic programs. These are linearly constrained optimization problems with piecewise linear-quadratic objective functions. We first summarize some local properties of a piecewise linear-quadratic function in terms of their first- and second-order directional derivatives. We next extend some well-known necessary and sufficient second-order conditions for local optimality of a quadratic program to a piecewise linear-quadratic program and provide a dozen such equivalent conditions for strong, strict, and isolated local optimality, showing in particular that a piecewise linear-quadratic program has the same characterizations for local minimality as a standard quadratic program. As a consequence of one such condition, we show that the number of strong, strict, or isolated local minima of a piecewise linear-quadratic program is finite; this result supplements a recent result about the finite number of directional stationary objective values. We also consider a special class of unconstrained composite programs involving a non-differentiable norm function, for which we show that the task of verifying the second-order stationary condition can be converted to the problem of checking the copositivity of certain Schur complement on the nonnegative orthant. [ABSTRACT FROM AUTHOR]

Symmetric nonnegative matrix factorization (SymNMF) has important applications in data analytics problems such as document clustering, community detection, and image segmentation. In this paper, we propose a novel nonconvex variable splitting method for solving SymNMF. The proposed algorithm is guaranteed to converge to the set of Karush–Kuhn–Tucker (KKT) points of the nonconvex SymNMF problem. Furthermore, it achieves a global sublinear convergence rate. We also show that the algorithm can be efficiently implemented in parallel. Further, sufficient conditions are provided that guarantee the global and local optimality of the obtained solutions. Extensive numerical results performed on both synthetic and real datasets suggest that the proposed algorithm converges quickly to a local minimum solution. [ABSTRACT FROM PUBLISHER]

Consider a multiple-input multiple-output (MIMO) downlink multi-user channel. A well-studied problem in such a system is the design of linear beamformers for power minimization with the quality of service (QoS) constraints. The most representative algorithms for solving this class of problems are the so-called minimum mean square error (MMSE)–second-order cone programming (SOCP) algorithm [Visotksy and Madhow, “Optimum Beamforming Using Transmit Antenna Arrays,” Proc. IEEE Veh. Technol. Conf., May 1999, vol. 1, pp. 851–856] , [Wong, Zheng, and Ng, “Convergence Analysis of Downlink MIMO Antenna System Using Second-Order Cone Programming,” Proc. 62nd IEEE Veh. Technol. Conf., Sep. 2005, pp. 492–496] and the uplink-downlink duality (UDD) algorithm [Codreanu, Tolli, Juntti, and Latva-Aho, “Joint Design of Tx-Rx Beamformers in MIMO Downlink Channel,” IEEE Trans. Signal Process., vol. 55, no. 9, pp. 4639–4655, Sep. 2007]. The former is based on alternating optimization of the transmit and receive beamformers; while the latter is based on the well-known uplink-dowlink duality theory. Despite their wide applicability, the convergence to Karush–Kuhn–Tucker (KKT) solutions of both algorithms is still open in the literature. In this paper, we rigorously establish the convergence of these algorithms for QoS-constrained power minimization (QCPM) problem with both single stream and multiple streams per user cases. Key to our analysis is the development and analysis of a new MMSE-DUAL algorithm, which connects the MMSE-SOCP and the UDD algorithm. Our numerical experiments show that 1) all these algorithms can almost always reach points with the same objective value irrespective of initialization and 2) the MMSE-SOCP/MMSE-DUAL algorithm works well while the UDD algorithm may fail with an infeasible initialization. [ABSTRACT FROM AUTHOR]

This work considers low-rank canonical polyadic decomposition (CPD) under a class of non-Euclidean loss functions that frequently arise in statistical machine learning and signal processing. These loss functions are often used for certain types of tensor data, e.g., count and binary tensors, where the least squares loss is considered unnatural. Compared to the least squares loss, the non-Euclidean losses are generally more challenging to handle. Non-Euclidean CPD has attracted considerable interests and a number of prior works exist. However, pressing computational and theoretical challenges, such as scalability and convergence issues, still remain. This work offers a unified stochastic algorithmic framework for large-scale CPD decomposition under a variety of non-Euclidean loss functions. Our key contribution lies in a tensor fiber sampling strategy-based flexible stochastic mirror descent framework. Leveraging the sampling scheme and the multilinear algebraic structure of low-rank tensors, the proposed lightweight algorithm ensures global convergence to a stationary point under reasonable conditions. Numerical results show that our framework attains promising non-Euclidean CPD performance. The proposed framework also exhibits substantial computational savings compared to state-of-the-art methods. [ABSTRACT FROM AUTHOR]

There has been a growing interest in developing data-driven, and in particular deep neural network (DNN) based methods for modern communication tasks. These methods achieve state-of-the-art performance for a few popular wireless resource allocation problems, while requiring less computational efforts, less resources for acquiring channel state information (CSI), etc. However, it is often challenging for these approaches to learn in a dynamic environment. This work develops a new approach that enables data-driven methods to continuously learn and optimize wireless resource allocation in a dynamic environment. Specifically, we consider an “episodically dynamic” setting where the environment statistics change in “episodes,” and in each episode the environment is stationary. We propose to build the notion of continual learning (CL) into wireless system design, so that the learning model can incrementally adapt to the new episodes, without forgetting knowledge learned from the previous episodes. We demonstrate the effectiveness of the CL approach by integrating it with three popular DNN based models for power control, beamforming and multi-user MIMO, respectively, and testing using both synthetic and ray-tracing based data sets. These numerical results show that the proposed CL approach is not only able to adapt to the new scenarios quickly and seamlessly, but importantly, it also maintains high performance over the previously encountered scenarios as well. [ABSTRACT FROM AUTHOR]

In this work, a gradient-based primal-dual method of multipliers is proposed for solving a class of linearly constrained non-convex problems. We show that with random initialization of the primal and dual variables, the algorithm is able to compute second-order stationary points (SOSPs) with probability one. Further, we present applications of the proposed method in popular signal processing and machine learning problems such as decentralized matrix factorization and decentralized training of overparameterized neural networks. One of the key steps in the analysis is to construct a new loss function for these problems such that the required convergence conditions (especially the gradient Lipschitz conditions) can be satisfied without changing the global optimal points. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the adaptive video delivery for multiple users over time-varying and mutually interfering multi-cell wireless networks. The key research challenge is to jointly design the physical-layer resource allocation scheme and application-layer rate adaptation logic, such that the users’ long-term fair quality of experience (QoE) can be maximized. Due to the timescale mismatch between these two layers and the asynchrony of user requests, however, it is difficult to directly model the cross-layer stochastic control problem by using a reinforcement learning framework. To address this difficulty, we propose a novel two-level decision framework where an optimization-based beamforming scheme (performed at the base stations) and a deep reinforcement learning (DRL)-based rate adaptation scheme (performed at the user terminals) are, respectively, developed, such that a highly complex long-term multi-user QoE fairness problem is decomposed into some relatively simple problems and solved effectively. Our strategy represents a significant departure from the existing schemes with consideration of either a short-term multi-user QoE maximization or a long-term single-user point-to-point QoE maximization. Extensive simulations demonstrate that the proposed cross-layer design is effective and promising. [ABSTRACT FROM AUTHOR]

In topic modeling, identifiability of the topics is an essential issue. Many topic modeling approaches have been developed under the premise that each topic has a characteristic anchor word that only appears in that topic. The anchor-word assumption is fragile in practice, because words and terms have multiple uses; yet it is commonly adopted because it enables identifiability guarantees. Remedies in the literature include using three- or higher-order word co-occurence statistics to come up with tensor factorization models, but such statistics need many more samples to obtain reliable estimates, and identifiability still hinges on additional assumptions, such as consecutive words being persistently drawn from the same topic. In this work, we propose a new topic identification criterion using second order statistics of the words. The criterion is theoretically guaranteed to identify the underlying topics even when the anchor-word assumption is grossly violated. An algorithm based on alternating optimization, and an efficient primal-dual algorithm are proposed to handle the resulting identification problem. The former exhibits high performance and is completely parameter-free; the latter affords up to 200 times speedup relative to the former, but requires step-size tuning and a slight sacrifice in accuracy. A variety of real text copora are employed to showcase the effectiveness of the approach, where the proposed anchor-free method demonstrates substantial improvements compared to a number of anchor-word based approaches under various evaluation metrics. [ABSTRACT FROM AUTHOR]

Numerical optimization has played a central role in addressing key signal processing (SP) problems. Highly effective methods have been developed for a large variety of SP applications such as communications, radar, filter design, and speech and image analytics, just to name a few. However, optimization algorithms often entail considerable complexity, which creates a serious gap between theoretical design/analysis and real-time processing. In this paper, we aim at providing a new learning-based perspective to address this challenging issue. The key idea is to treat the input and output of an SP algorithm as an unknown nonlinear mapping and use a deep neural network (DNN) to approximate it. If the nonlinear mapping can be learned accurately by a DNN of moderate size, then SP tasks can be performed effectively—since passing the input through a DNN only requires a small number of simple operations. In our paper, we first identify a class of optimization algorithms that can be accurately approximated by a fully connected DNN. Second, to demonstrate the effectiveness of the proposed approach, we apply it to approximate a popular interference management algorithm, namely, the WMMSE algorithm. Extensive experiments using both synthetically generated wireless channel data and real DSL channel data have been conducted. It is shown that, in practice, only a small network is sufficient to obtain high approximation accuracy, and DNNs can achieve orders of magnitude speedup in computational time compared to the state-of-the-art interference management algorithm. [ABSTRACT FROM AUTHOR]

Symmetric nonnegative matrix factorization (SNMF) is equivalent to computing a symmetric nonnegative low rank approximation of a data similarity matrix. It inherits the good data interpretability of the well-known nonnegative matrix factorization technique and has better ability of clustering nonlinearly separable data. In this paper, we focus on the algorithmic aspect of the SNMF problem and propose simple inexact block coordinate decent methods to address the problem, leading to both serial and parallel algorithms. The proposed algorithms have guaranteed convergence to stationary solutions and can efficiently handle large-scale and/or sparse SNMF problems. Extensive simulations verify the effectiveness of the proposed algorithms compared to recent state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Generalized canonical correlation analysis (GCCA) aims at finding latent low-dimensional common structure from multiple views (feature vectors in different domains) of the same entities. Unlike principal component analysis that handles a single view, (G)CCA is able to integrate information from different feature spaces. Here we focus on MAX-VAR GCCA, a popular formulation that has recently gained renewed interest in multilingual processing and speech modeling. The classic MAX-VAR GCCA problem can be solved optimally via eigen-decomposition of a matrix that compounds the (whitened) correlation matrices of the views; but this solution has serious scalability issues, and is not directly amenable to incorporating pertinent structural constraints such as nonnegativity and sparsity on the canonical components. We posit regularized MAX-VAR GCCA as a nonconvex optimization problem and propose an alternating optimization-based algorithm to handle it. Our algorithm alternates between inexact solutions of a regularized least squares subproblem and a manifold-constrained nonconvex subproblem, thereby achieving substantial memory and computational savings. An important benefit of our design is that it can easily handle structure-promoting regularization. We show that the algorithm globally converges to a critical point at a sublinear rate, and approaches a global optimal solution at a linear rate when no regularization is considered. Judiciously designed simulations and large-scale word embedding tasks are employed to showcase the effectiveness of the proposed algorithm. [ABSTRACT FROM PUBLISHER]

In this paper, we propose joint transceiver design algorithms for the full-duplex $K$ -pair multiple-input multiple-output interference channel with simultaneous wireless information and power transfer. To mitigate and exploit the complex interference, we consider two important utility optimization problems, i.e., the sum power minimization problem and the sum-rate maximization problem. In the first problem, our aim is to minimize the total transmission power under both transmission rate and energy harvesting (EH) constraints. An iterative algorithm based on alternating optimization (AO) and with guaranteed monotonic convergence is proposed to successively optimize the transceiver coefficients. The algorithm consists of three main steps, where the concave–convex procedure (CCCP), the minimum mean-square error (MMSE) criterion, and the semidefinite relaxation technique are, respectively, employed to compute the vectors of power splitting ratios, the receiving matrices, and the transmitting beamforming vectors. Two simplified algorithms based on fixed beamformers, namely, the maximum ratio transmission and the maximum signal-to-interference-leakage beamformers are also proposed. In the second problem, our aim is to maximize the sum-rate under additional power and EH constraints. Due to the highly non-convex nature of this problem, we first reformulate it into an equivalent-weighted MMSE problem by introducing suitable weight factors, such that the global optima of the two problems are identical. Then, by utilizing the concept of AO and CCCP, we show that the equivalent problem can be efficiently solved. Again, with the aid of the fixed beamformers, two simplified algorithms are provided to reduce the computational complexity. Simulation results are presented to validate the effectiveness of the proposed algorithms. [ABSTRACT FROM PUBLISHER]

This paper considers a power splitting based multiuser multiple-input-single-output (MISO) downlink system with simultaneous wireless information and power transfer, where each single antenna receiver splits the received signal into two streams of different power for decoding information and harvesting energy separately. Assuming that the most common zero-forcing (ZF) beamforming scheme is employed by the base station, we aim to maximize the system energy efficiency in bits per Joule by joint beamforming and power splitting under both the signal-to-interference-plus-noise ratio constraints and energy harvesting constraints. The energy efficiency optimization (EEO) problem is nonconvex and very hard to solve. In this paper, by exploiting the problem structure, we first simplify the EEO problem to a joint transmit power allocation and receive power splitting problem. Then, with a tactful reformulation, we propose a Lagrangian relaxation (LR) method coupled with Dinkelbach method to address the simplified EEO problem, whilst devising a nearly closed-form solution for the subproblems involved in the Dinkelbach method. It is proven that the proposed LR method is optimum under some condition and can guarantee at least a feasible solution, which is a notable advantage over the existing methods. Besides, we develop a low complexity EEO algorithm by proportionally distributing the total power to users. Finally, numerical results validate the excellent efficiency of the proposed algorithms. [ABSTRACT FROM AUTHOR]