Search

Online model selection involves selecting a model from a set of candidate models 'on the fly' to perform prediction on a stream of data. The choice of candidate models henceforth has a crucial impact on the performance. Although employing a larger set of candidate models naturally leads to more flexibility in model selection, this may be infeasible in cases where prediction tasks are performed on edge devices with limited memory. Faced with this challenge, the present paper proposes an online federated model selection framework where a group of learners (clients) interacts with a server with sufficient memory such that the server stores all candidate models. However, each client only chooses to store a subset of models that can be fit into its memory and performs its own prediction task using one of the stored models. Furthermore, employing the proposed algorithm, clients and the server collaborate to fine-tune models to adapt them to a non-stationary environment. Theoretical analysis proves that the proposed algorithm enjoys sub-linear regret with respect to the best model in hindsight. Experiments on real datasets demonstrate the effectiveness of the proposed algorithm., Comment: Accepted by Transactions on Machine Learning Research (TMLR)

Multi-kernel learning (MKL) exhibits well-documented performance in online non-linear function approximation. Federated learning enables a group of learners (called clients) to train an MKL model on the data distributed among clients to perform online non-linear function approximation. There are some challenges in online federated MKL that need to be addressed: i) Communication efficiency especially when a large number of kernels are considered ii) Heterogeneous data distribution among clients. The present paper develops an algorithmic framework to enable clients to communicate with the server to send their updates with affordable communication cost while clients employ a large dictionary of kernels. Utilizing random feature (RF) approximation, the present paper proposes scalable online federated MKL algorithm. We prove that using the proposed online federated MKL algorithm, each client enjoys sub-linear regret with respect to the RF approximation of its best kernel in hindsight, which indicates that the proposed algorithm can effectively deal with heterogeneity of the data distributed among clients. Experimental results on real datasets showcase the advantages of the proposed algorithm compared with other online federated kernel learning ones., Comment: Published at NeurIPS 2022

Communication efficiency arises as a necessity in federated learning due to limited communication bandwidth. To this end, the present paper develops an algorithmic framework where an ensemble of pre-trained models is learned. At each learning round, the server selects a subset of pre-trained models to construct the ensemble model based on the structure of a graph, which characterizes the server's confidence in the models. Then only the selected models are transmitted to the clients, such that certain budget constraints are not violated. Upon receiving updates from the clients, the server refines the structure of the graph accordingly. The proposed algorithm is proved to enjoy sub-linear regret bound. Experiments on real datasets demonstrate the effectiveness of our novel approach.

Online learning with expert advice is widely used in various machine learning tasks. It considers the problem where a learner chooses one from a set of experts to take advice and make a decision. In many learning problems, experts may be related, henceforth the learner can observe the losses associated with a subset of experts that are related to the chosen one. In this context, the relationship among experts can be captured by a feedback graph, which can be used to assist the learner's decision making. However, in practice, the nominal feedback graph often entails uncertainties, which renders it impossible to reveal the actual relationship among experts. To cope with this challenge, the present work studies various cases of potential uncertainties, and develops novel online learning algorithms to deal with uncertainties while making use of the uncertain feedback graph. The proposed algorithms are proved to enjoy sublinear regret under mild conditions. Experiments on real datasets are presented to demonstrate the effectiveness of the novel algorithms.

Multi-kernel learning (MKL) has been widely used in function approximation tasks. The key problem of MKL is to combine kernels in a prescribed dictionary. Inclusion of irrelevant kernels in the dictionary can deteriorate accuracy of MKL, and increase the computational complexity. To improve the accuracy of function approximation and reduce the computational complexity, the present paper studies data-driven selection of kernels from the dictionary that provide satisfactory function approximations. Specifically, based on the similarities among kernels, the novel framework constructs and refines a graph to assist choosing a subset of kernels. In addition, random feature approximation is utilized to enable online implementation for sequentially obtained data. Theoretical analysis shows that our proposed algorithms enjoy tighter sub-linear regret bound compared with state-of-art graph-based online MKL alternatives. Experiments on a number of real datasets also showcase the advantages of our novel graph-aided framework., Comment: Preliminary results of this work have been presented in "Online Multi-Kernel Learning with Graph-Structured Feedback." P. M. Ghari, and Y. Shen, International Conference on Machine Learning (ICML), pp. 3474-3483. PMLR, July 2020

Nowadays, providing higher data rate is a momentous goal for wireless communications systems. Interference is one of the important obstacles to reach this purpose. Interference alignment is a management technique that align interference from other transmitters in the least possible dimension subspace at each receiver and as a result, provide the remaining dimensions for free interference signal. An uncoordinated interference is an example of interference which cannot be aligned coordinately with interference from coordinated part and consequently, the performance of interference alignment approaches is degraded. In this paper, we propose two rank minimization methods to enhance the performance of interference alignment in the presence of uncoordinated interference sources. Firstly, a new objective function is chosen then, a new class of convex relaxation is proposed with respect to the uncoordinated interference which leads to decrease the optimal value of our optimization problem. Moreover, we use schatten-p-norm as surrogate of rank function and we implement iteratively reweighted algorithm to solve optimization problem. In addition, we apply our proposed methods to mitigate interference in relay-aided MIMO interference channel, and propose a weighted-sum method to improve the performance of interference alignment in the amplify-and-forward relay-aided MIMO system based on the rank minimization approach. Finally, our simulation results show that our proposed methods can obtain considerably higher multiplexing gain and sum rate than other approaches in the interference alignment framework and the performance of interference alignment is improved.

The localization problem in a wireless sensor network is to determine the coordination of sensor nodes using the known positions of some nodes (called anchors) and corresponding noisy distance measurements. There is a variety of different approaches to solve this problem such as semi-definite programming (SDP) based, sum of squares and second order cone programming, and between them, SDP-based approaches have shown good performance. In recent years, the primary SDP approach has been investigated and a variety of approaches are proposed in order to enhance its performance. In SDP approaches, errors in approximating the given distances are minimized as an objective function. It is desirable that the distribution of error in these problems would be a delta distribution, which is practically impossible. Therefore, we may approximate delta distribution by Gaussian distribution with very small variance. In this paper, we define a new objective function which makes the error distribution as similar as possible to a Gaussian distribution with a very small variance. Simulation results show that our proposed method has higher accuracy compared to the traditional SDP approach and other prevalent objective functions which are used such as least squares. Our method is also faster than other popular approaches which try to improve the accuracy of the primary SDP approach., Comment: 11 pages, 9 figures