Search

Bi-Level Optimization (BLO) is originated from the area of economic game theory and then introduced into the optimization community. BLO is able to handle problems with a hierarchical structure, involving two levels of optimization tasks, where one task is nested inside the other. In machine learning and computer vision fields, despite the different motivations and mechanisms, a lot of complex problems, such as hyper-parameter optimization, multi-task and meta learning, neural architecture search, adversarial learning and deep reinforcement learning, actually all contain a series of closely related subproblms. In this paper, we first uniformly express these complex learning and vision problems from the perspective of BLO. Then we construct a best-response-based single-level reformulation and establish a unified algorithmic framework to understand and formulate mainstream gradient-based BLO methodologies, covering aspects ranging from fundamental automatic differentiation schemes to various accelerations, simplifications, extensions and their convergence and complexity properties. Last but not least, we discuss the potentials of our unified BLO framework for designing new algorithms and point out some promising directions for future research. A list of important papers discussed in this survey, corresponding codes, and additional resources on BLOs are publicly available at: https://github.com/vis-opt-group/BLO. [ABSTRACT FROM AUTHOR]

With the abundance of data, machine learning applications engaged increased attention in the last decade. An attractive approach to robustify the statistical analysis is to preprocess the data through clustering. This paper develops a low-complexity Riemannian optimization framework for solving optimization problems on the set of positive semidefinite stochastic matrices. The low-complexity feature of the proposed algorithms stems from the factorization of the optimization variable $\mathbf {X}=\mathbf {Y}\mathbf {Y}^{\mathrm{T}}$ X = Y Y T and deriving conditions on the number of columns of $\mathbf {Y}$ Y under which the factorization yields a satisfactory solution. The paper further investigates the embedded and quotient geometries of the resulting Riemannian manifolds. In particular, the paper explicitly derives the tangent space, Riemannian gradients and Hessians, and a retraction operator allowing the design of efficient first and second-order optimization methods for the graph-based clustering applications of interest. The numerical results reveal that the resulting algorithms present a clear complexity advantage as compared with state-of-the-art euclidean and Riemannian approaches for graph clustering applications. [ABSTRACT FROM AUTHOR]

Traffic control optimization is a challenging task for various traffic centers around the world and the majority of existing approaches focus only on developing adaptive methods for normal (recurrent) traffic conditions. Optimizing the control plans when severe incidents occur still remains an open problem, especially when a high number of lanes or entire intersections are affected. This paper presents a novel methodology for optimizing the traffic signal timings in signalized urban intersections, under non-recurrent traffic incidents. With the purpose of producing fast and reliable decisions, we combine the fast running Machine Learning (ML) algorithms and the reliable Genetic Algorithms (GA) into a single optimization framework. Firstly, we deploy a typical GA algorithm by considering the phase duration as the decision variable and the objective function as the total travel time in the network. We fine tune the GA for crossover, mutation, fitness calculation and obtain the optimal parameters. Secondly, we train several regression models to predict the total travel time in the studied traffic network, and select the best performing model which we further hyper-tune. Lastly, we propose a new algorithm BGA-ML combining the GA algorithm and the extreme-gradient decision-tree (XGBT), which is the best performing regression model, together in a single optimization framework. Comparison and results are generated by two experiments (one synthetic and one from real urban traffic network) and show that the new BGA-ML is much faster than the original GA algorithm and can reduce the total travel time by almost half when used under incident conditions. [ABSTRACT FROM AUTHOR]

As the semiconductor manufacturing process is moving towards the 3 nm node, there is a crucial need to reduce the edge placement error (EPE) to ensure proper functioning of the integrated circuit (IC) devices. EPE is the most important metric that quantify the fidelity of fabricated patterns in multi-patterning processes, and it is the combination of overlay errors and critical dimension (CD) errors. Recent advances in machine learning have enabled many new possibilities to improve the performance and efficiency of EPE optimization techniques. In this paper, we conducted a survey of recent research work that applied machine learning/ deep learning techniques for the purposes of enhancing virtual overlay metrology, reducing overlay error, and improving mask optimization methods for EPE reduction. Thorough discussions about the objectives, datasets, input features, models, key findings, and limitations are provided. In general, the results of the review work show a great potential of machine learning techniques in aiding the improvement of EPE in the field of semiconductor manufacturing. [ABSTRACT FROM AUTHOR]

Correspondence analysis (CA) is a multivariate statistical tool used to visualize and interpret data dependencies by finding maximally correlated embeddings of pairs of random variables. CA has found applications in fields ranging from epidemiology to social sciences. However, current methods for CA do not scale to large, high-dimensional datasets. In this paper, we provide a novel interpretation of CA in terms of an information-theoretic quantity called the principal inertia components. We show that estimating the principal inertia components, which consists in solving a functional optimization problem over the space of finite variance functions of two random variable, is equivalent to performing CA. We then leverage this insight to design algorithms to perform CA at scale. Specifically, we demonstrate how the principal inertia components can be reliably approximated from data using deep neural networks. Finally, we show how the maximally correlated embeddings of pairs of random variables in CA further play a central role in several learning problems including multi-view and multi-modal learning methods and visualization of classification boundaries. [ABSTRACT FROM AUTHOR]

This paper revisits the temporal difference (TD) learning algorithm for the policy evaluation tasks in reinforcement learning. Typically, the performance of TD(0) and TD($\lambda$ λ ) is very sensitive to the choice of stepsizes. Oftentimes, TD(0) suffers from slow convergence. Motivated by the tight link between the TD(0) learning algorithm and the stochastic gradient methods, we develop a provably convergent adaptive projected variant of the TD(0) learning algorithm with linear function approximation that we term AdaTD(0). In contrast to the TD(0), AdaTD(0) is robust or less sensitive to the choice of stepsizes. Analytically, we establish that to reach an $\epsilon$ ε accuracy, the number of iterations needed is $\tilde{O}(\epsilon ^{-2}\ln ^4\frac{1}{\epsilon }/\ln ^4\frac{1}{\rho })$ O ˜ (ε - 2 ln 4 1 ε / ln 4 1 ρ) in the general case, where $\rho$ ρ represents the speed of the underlying Markov chain converges to the stationary distribution. This implies that the iteration complexity of AdaTD(0) is no worse than that of TD(0) in the worst case. When the stochastic semi-gradients are sparse, we provide theoretical acceleration of AdaTD(0). Going beyond TD(0), we develop an adaptive variant of TD($\lambda$ λ ), which is referred to as AdaTD($\lambda$ λ ). Empirically, we evaluate the performance of AdaTD(0) and AdaTD($\lambda$ λ ) on several standard reinforcement learning tasks, which demonstrate the effectiveness of our new approaches. [ABSTRACT FROM AUTHOR]

Capturing the dynamics in user preference is crucial to better predict user future behaviors because user preferences often drift over time. Many existing recommendation algorithms – including both shallow and deep ones – often model such dynamics independently, i.e., user static and dynamic preferences are not modeled under the same latent space, which makes it difficult to fuse them for recommendation. This paper considers the problem of embedding a user's sequential behavior into the latent space of user preferences, namely translating sequence to preference. To this end, we formulate the sequential recommendation task as a dictionary learning problem, which learns: 1) a shared dictionary matrix, each row of which represents a partial signal of user dynamic preferences shared across users; and 2) a posterior distribution estimator using a deep autoregressive model integrated with Gated Recurrent Unit (GRU), which can select related rows of the dictionary to represent a user's dynamic preferences conditioned on his/her past behaviors. Qualitative studies on the Netflix dataset demonstrate that the proposed method can capture the user preference drifts over time and quantitative studies on multiple real-world datasets demonstrate that the proposed method can achieve higher accuracy compared with state-of-the-art factorization and neural sequential recommendation methods. [ABSTRACT FROM AUTHOR]

As an aerial base station, unmanned aerial vehicle (UAV) has been considered as a promising technology to assist future wireless communications due to its flexible, swift and low cost features, where resource allocation is the basis for ensuring energy-efficient UAV-assisted networks. This paper formulates a joint optimization problem of user association, UAV trajectory design and power control to maximize the channel capacity among all ground users at a limited power level in a downlink transmission. To tackle the mixed-integer non-linear programming problem, this paper proposes a clustering-aided reinforcement learning approach consisting of three consecutive stages. Firstly, modified expectation-maximization unsupervised learning algorithm is investigated to cluster the ground users, which reduces the dimensions and hence, the association complexity is reduced as well. Then, Kuhn-Munkres algorithm is incorporated for user association, which associates a UAV with the ground users via matching to the cluster, and assigns the UAVs to the centroid of the matching cluster for pre-placement, with the aim of speeding up the convergence of the following deep reinforcement learning algorithm. Finally, a multi-agent twin delayed deep deterministic (MATD3) policy gradient is proposed to solve the non-convex sub-problem, which determines the transmit power and designs the fine-tuned trajectory of UAVs. By incorporating low-bias value estimation technique, the reward of the proposed MATD3 algorithm is improved. Simulation results have demonstrated that our proposed approach achieves higher reward as well as converging faster than existing reinforcement algorithms. Besides, the clustering-aided reinforcement learning has lower computational complexity than the benchmark schemes. [ABSTRACT FROM AUTHOR]

Due to the diversity of power consumption patterns, the false positive rate (FPR) of data-driven electricity theft detection (ETD) methods is too high to meet practical needs, which severely restricts the engineering application of data-based methods. To reduce FPR of ETD methods based on advanced metering infrastructure (AMI), a deep neural network with low FPR (LFPR-DNN) is proposed in this paper. First, a deep model is constructed based on one-dimensional convolution and residual network, which can automatically extract features from consumption data. Then, a two-stage training scheme is used to train the network. In the first stage, the conventional gradient descent algorithm is used to update the network weights. To minimize the impact of data imbalance on detection performance, focal loss is used. Besides, grid search is used to optimize the hyper-parameters of the model. In the second stage, with FPR as the optimization objective, the particle swarm optimization (PSO) algorithm is used to train the network. Finally, the proposed LFPR-DNN is verified by using the open Irish data set. Compared to other state-of-the-art classifiers, LFPR-DNN has the lowest FPR with 0.29% and the highest AUC with 99.42%. The FPR is reduced by an order of magnitude, which verifies the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper investigates the stochastic optimization problem focusing on developing scalable parallel algorithms for deep learning tasks. Our solution involves a reformation of the objective function for stochastic optimization in neural network models, along with a novel parallel computing strategy, coined the weighted aggregating stochastic gradient descent (WASGD). Following a theoretical analysis on the characteristics of the new objective function, WASGD introduces a decentralized weighted aggregating scheme based on the performance of local workers. Without any center variable, the new method automatically gauges the importance of local workers and accepts them by their contributions. Furthermore, we have developed an enhanced version of the method, WASGD+, by (1) implementing a designed sample order and (2) upgrading the weight evaluation function. To validate the new method, we benchmark our pipeline against several popular algorithms including the state-of-the-art deep neural network classifier training techniques (e.g., elastic averaging SGD). Comprehensive validation studies have been conducted on four classic datasets: CIFAR-100, CIFAR-10, Fashion-MNIST, and MNIST. Subsequent results have firmly validated the superiority of the WASGD scheme in accelerating the training of deep architecture. Better still, the enhanced version, WASGD+, is shown to be a significant improvement over its prototype. [ABSTRACT FROM AUTHOR]

Coded distributed computing(CDC) has shown great potentials to solve the unexpected delay caused by stragglers in distributed computing. In this paper, we focus on the auction design for efficient resource allocation in CDC. Specifically, we aim to design a learning auction mechanism to handle heterogeneous user demands and also to free users from the complexity of specifying valuations for resource combinations, which increases exponentially with the resource dimensions. The user demand type is heterogeneous according to different variation trends of the value with finish time and workload, which is modeled by deep learning. The platform would allocate resources according to the user value function. Then users do not need to consider the complex relationship between uncertain finish time and resource configuration in CDC. Due to the inference error of the learning model and the complexity of calculating uncertain finish time, the considered social welfare optimization problem is a non-linear and non-convex integer problem. Even worse, the typical VCG-based payment scheme cannot guarantee truthfulness with the inference error. In response to these difficulties, we transform the social welfare optimization problem into a mixed integer programming problem which already has efficient solutions. The social welfare gap caused by the inference error is analyzed theoretically. The relationship between the utility regret of reporting truthfully and the inference error is also analyzed. We prove that our mechanism satisfies incentive alignment and individual rationality. Extensive experiments show the superiority of our mechanism compared with existing ones. [ABSTRACT FROM AUTHOR]

Objectives: Eye blink artifact detection in scalp electroencephalogram (EEG) of epilepsy patients is challenging due to its similar waveforms to epileptiform discharges. Developing an accurate detection method is urgent and critical. Methods: In this paper, we proposed a novel multi-dimensional feature optimization based eye blink artifact detection algorithm for EEGs containing rich epileptiform discharges. An unsupervised clustering algorithm based on smoothed nonlinear energy operator (SNEO) and variational mode extraction (VME) is proposed to detect epileptiform discharges in the frontal leads. Then, multi-dimensional time/frequency EEG features extracted from forehead electrodes (FP1 and FP2 channels) combining with the improved VME (IVME) threshold are derived for EEG representation. A variance filtering method is further applied for discriminative feature selection and a machine learning model is finally learned to perform detection. Results: Experiments on EEGs of 16 subjects from the Children’s Hospital of Zhejiang University School of Medicine (CHZU) show that our method achieves the highest average sensitivity, specificity and accuracy of 95.04, 89.52, and 93.01, respectively. That outperforms 5 recent and state-of-the-art (SOTA) eye blink detection algorithms. Significance: The proposed method is robust in eye blink artifact detection for EEGs containing high-frequency epileptiform discharges. It is also effective in dealing with individual differences in EEGs, which is usually ignored in conventional methods. [ABSTRACT FROM AUTHOR]

Spatial failure patterns in wafer defect maps are very useful for root cause analysis, which is essential for yield optimization in the manufacturing process. Especially, the scratch defect type is the most challenging pattern to recognize because the position, shape, size and curvature vary widely from one scratch to another. Discontinuity points within scratches also contribute to the low recognition rate, and such points are often hidden defective dies that become reliability threat. Previous studies on defect pattern identification show that the recognition rate for scratches is among the lowest in all patterns even if the overall accuracy is high. In this paper, we propose a novel scratch pattern recognition method. In this paper, image processing techniques are applied to recognize scratch patterns in wafer maps. The Hough transform is first employed to identify line segments, from which scratches can be reconstructed. In contrast to previous machine-learning based methods, there is no need to train a complicated prediction model so that the computation time is small. The method is validated by using wafer maps from six real products and public data. Experimental results show that the proposed method achieves high recall, precision and accuracy in all cases. [ABSTRACT FROM AUTHOR]

In this paper, the problem of distributed content caching in a small-cell Base Stations (sBSs) wireless network that maximizes the cache hit performance is considered. Most of the existing works consider static demands, however, here, data at each sBS is considered to be correlated across time and sBSs. Federated learning (FL) based caching strategy is proposed which is assumed to be a weighted combination of past caching strategies of the sBS as well as the neighbouring sBSs. A high probability generalization guarantees on the performance of the proposed federated caching strategy is derived. The theoretical guarantee provides following insights on obtaining the caching strategy: (i) run regret minimization at each sBS to obtain a sequence of caching strategies across time, and (ii) maximize an estimate of the bound to obtain a set of weights for the caching strategy which depends on the discrepancy. Theoretical guarantee on the performance of the least recently frequently used (LRFU) caching strategy is derived. Further, FL based heuristic caching algorithm is also proposed. Finally, it is shown through simulations using Movie Lens dataset that the proposed algorithm significantly outperforms the recent online learning algorithms. [ABSTRACT FROM AUTHOR]

We consider a class of nonconvex regularized optimization problems, which appear frequently in machine learning and data processing. Due to the structure of the problems, the iteratively reweighted algorithm was developed and applied to the consensus optimization. In this paper, we propose the acceleration of this scheme by adding an inertial term in each iteration. The proposed algorithms inherit the advantages of classical decentralized algorithms: they can be implemented over a connected network, in which the agents communicate with their neighbors and perform local computations. We also employ the diminishing stepsizes technique for the iteratively reweighted algorithm and consider its acceleration. In specific cases, our algorithms reduce to existing decentralized schemes and also indicate novel ones. Mathematically, we prove the convergence for both algorithms with several assumptions on the objective functions. With Kurdyka-Łojasiewicz property, convergence rates can be derived for constant stepsize case. Numerical results demonstrate the efficiency of the algorithms. [ABSTRACT FROM AUTHOR]

The incorporation of Artificial Intelligence (AI) and Machine Learning (ML) in mobile networks is expected to raise the degree of automation by proposing Cognitive Autonomous Networks (CAN). In CAN, learning based functions, called Cognitive Functions (CFs), adjust network control parameters to optimize specific Key Performance Indicators (KPIs). The CFs share the same resources, and this very often introduces an overlap among their target control parameter adjustment, i.e., at one point of time, multiple CFs may want to change the same control parameter albeit by different amounts depending on their respective levels of interest in that parameter. Correspondingly, a Controller is required in CAN to coordinate the sharing of the parameter among the independent CFs to meet their varying extents of interests. Although a Nash Social Welfare Function (NSWF) based Controller was introduced at first, to overcome the problems of this Controller a second Controller was introduced based on Eisenberg-Gale Solution (EGS). To use an EGS based Controller, impact of each network control parameter on each CF, called Config-Weight (CW), needs to be calculated. In this paper we propose a Shapley value based method for CW calculation, prove the optimality of the method mathematically and by simulation, provide a comparison between the Controllers in a simulation environment that resembles 5G network and find that up to 9.18% improvement can be obtained using the EGS based Controller. [ABSTRACT FROM AUTHOR]

We consider the problem of minimizing the $\ell _1$ norm of a linear map over the sphere, which arises in various machine learning applications such as orthogonal dictionary learning (ODL) and robust subspace recovery (RSR). The problem is numerically challenging due to its nonsmooth objective and nonconvex constraint, and its algorithmic aspects have not been well explored. In this paper, we show how the manifold structure of the sphere can be exploited to design fast algorithms with provable guarantees for tackling this problem. Specifically, our contribution is fourfold. First, we present a manifold proximal point algorithm (ManPPA) for the problem and show that it converges at a global sublinear rate. Furthermore, we show that ManPPA can achieve a local quadratic convergence rate when applied to sharp instances of the problem. Second, we develop a semismooth Newton-based inexact augmented Lagrangian method for computing the search direction in each iteration of ManPPA and show that it has an asymptotic superlinear convergence rate. Third, we propose a stochastic variant of ManPPA called StManPPA, which is well suited for large-scale computation, and establish its sublinear convergence rate. Both ManPPA and StManPPA have provably faster convergence rates than existing subgradient-type methods. Fourth, using ManPPA as a building block, we propose a new heuristic method for solving a matrix analog of the problem, in which the sphere is replaced by the Stiefel manifold. The results from our extensive numerical experiments on the ODL and RSR problems demonstrate the efficiency and efficacy of our proposed methods. [ABSTRACT FROM AUTHOR]

Non-orthogonal multiple access based cooperative relaying system (NOMA-CRS) has been proposed to alleviate the decay in spectral efficiency of the conventional CRS. However, existing NOMA-CRS studies assume perfect successive interference canceler at the relay and mostly investigate sum rate whereas the error performance has not been taken into consideration. In this paper, we analyze error performance of the NOMA-CRS and the closed-form bit error probability (BEP) expression is derived over Nakagami-m fading channels. Then, thanks to the high performance of machine learning (ML) in challenging optimization problems, a joint power sharing-power allocation (PS-PA) scheme is proposed to minimize the bit error rate (BER) of the NOMA-CRS. The proposed ML-assisted optimization has a very low online implementation complexity. Based on provided extensive simulations, theoretical BEP analysis is validated. Besides, the proposed ML-aided PS-PA provides minimum BER (MBER) and outperforms previous PA strategies for the NOMA-CRS notably. [ABSTRACT FROM AUTHOR]

With the increasing of the data size and the development of multi-core computers, asynchronous parallel stochastic optimization algorithms such as KroMagnon have gained significant attention. In this paper, we propose a new Sparse approximation and asynchronous parallel Stochastic Variance Reduced Gradient (SSVRG) method for sparse and high-dimensional machine learning problems. Unlike standard SVRG and its asynchronous parallel variant, KroMagnon, the snapshot point of SSVRG is set to the average of all the iterates in the previous epoch, which allows it to take much larger learning rates and also makes it more robust to the choice of learning rates. In particular, we use the sparse approximation of the popular SVRG estimator to perform completely sparse updates at all iterations. Therefore, SSVRG has a much lower per-iteration computational cost than its dense counterpart, SVRG++, and is very friendly to asynchronous parallel implementation. Moreover, we provide the convergence guarantees of SSVRG for both strongly convex and non-strongly convex problems, while existing asynchronous algorithms (e.g., KroMagnon and ASAGA) only have convergence guarantees for strongly convex problems. Finally, we extend SSVRG to non-smooth and asynchronous parallel settings. Numerical experimental results demonstrate that SSVRG converges significantly faster than the state-of-the-art asynchronous parallel methods, e.g., KroMagnon, and is usually more than three orders of magnitude faster than SVRG++. [ABSTRACT FROM AUTHOR]

In this paper, we study a new latency optimization problem for blockchain-based federated learning (BFL) in multi-server edge computing. In this system model, distributed mobile devices (MDs) communicate with a set of edge servers (ESs) to handle both machine learning (ML) model training and block mining simultaneously. To assist the ML model training for resource-constrained MDs, we develop an offloading strategy that enables MDs to transmit their data to one of the associated ESs. We then propose a new decentralized ML model aggregation solution at the edge layer based on a consensus mechanism to build a global ML model via peer-to-peer (P2P)-based blockchain communications. Blockchain builds trust among MDs and ESs to facilitate reliable ML model sharing and cooperative consensus formation, and enables rapid elimination of manipulated models caused by poisoning attacks. We formulate latency-aware BFL as an optimization aiming to minimize the system latency via joint consideration of the data offloading decisions, MDs’ transmit power, channel bandwidth allocation for MDs’ data offloading, MDs’ computational allocation, and hash power allocation. Given the mixed action space of discrete offloading and continuous allocation variables, we propose a novel deep reinforcement learning scheme with a parameterized advantage actor critic algorithm. We theoretically characterize the convergence properties of BFL in terms of the aggregation delay, mini-batch size, and number of P2P communication rounds. Our numerical evaluation demonstrates the superiority of our proposed scheme over baselines in terms of model training efficiency, convergence rate, system latency, and robustness against model poisoning attacks. [ABSTRACT FROM AUTHOR]

Parallelism and data reuse are the most critical issues for the design of hardware acceleration in a deep learning processor. Besides, abundant on-chip memories and precise data management are intrinsic design requirements because most of deep learning algorithms are data-driven and memory-bound. In this paper, we propose a compiler-architecture co-design scheme targeting a reconfigurable and algorithm-oriented array processor, named ReAAP. Given specific deep neural networks, the proposed co-design scheme is effective to perform parallelism and data reuse optimization on compute-intensive layers for guiding reconfigurable computing in hardware. Especially, the systemic optimization is performed in our proposed domain-specific compiler to deal with the intrinsic tensions between parallelism and data locality, for the purpose of automatically mapping diverse layer-level workloads onto our proposed reconfigurable array architecture. In this architecture, abundant on-chip memories are software-controlled and its massive data access is precisely handled by compiler-generated instructions. In our experiments, the ReAAP is implemented on an embedded FPGA platform. Experimental results demonstrate that our proposed co-design scheme is effective to integrate software flexibility with hardware parallelism for accelerating diverse deep learning workloads. As a whole system, ReAAP achieves a consistently high utilization of hardware resource for accelerating all the diverse compute-intensive layers in ResNet, MobileNet, and BERT. [ABSTRACT FROM AUTHOR]

Edge systems are required to autonomously make real-time decisions based on large quantities of input data under strict power, performance, area, and other constraints. Meeting these constraints is only possible by specializing systems through hardware accelerators purposefully built for machine learning and data analysis algorithms. However, data science evolves at a quick pace, and manual design of custom accelerators has high non-recurrent engineering costs: general solutions are needed to automatically and rapidly transition from the formulation of a new algorithm to the deployment of a dedicated hardware implementation. Our solution is the SOftware Defined Architectures (SODA) Synthesizer, an end-to-end, multi-level, modular, extensible compiler toolchain providing a direct path from machine learning tools to hardware. The SODA Synthesizer frontend is based on the multilevel intermediate representation (MLIR) framework; it ingests pre-trained machine learning models, identifies kernels suited for acceleration, performs high-level optimizations, and prepares them for hardware synthesis. In the backend, SODA leverages state-of-the-art high-level synthesis techniques to generate highly efficient accelerators, targeting both field programmable devices (FPGAs) and application-specific circuits (ASICs). In this paper, we describe how the SODA Synthesizer can also assemble the generated accelerators (based on the finite state machine with datapath model) in a custom system driven by a distributed controller, building a coarse-grained dataflow architecture that does not require a host processor to orchestrate parallel execution of multiple accelerators. We show the effectiveness of our approach by automatically generating ASIC accelerators for layers of popular deep neural networks (DNNs). Our high-level optimizations result in up to 74x speedup on isolated accelerators for individual DNN layers, and our dynamically scheduled architecture yields an additional 3x performance improvement when combining accelerators to handle streaming inputs. [ABSTRACT FROM AUTHOR]

GPUs are widely used to accelerate the training of machine learning workloads. As modern machine learning models become increasingly larger, they require a longer time to train, leading to higher GPU energy consumption. This paper presents GPOEO, an online GPU energy optimization framework for machine learning training workloads. GPOEO dynamically determines the optimal energy configuration by employing novel techniques for online measurement, multi-objective prediction modeling, and search optimization. To characterize the target workload behavior, GPOEO utilizes GPU performance counters. To reduce the performance counter profiling overhead, it uses an analytical model to detect the training iteration change and only collects performance counter data when an iteration shift is detected. GPOEO employs multi-objective models based on gradient boosting and a local search algorithm to find a trade-off between execution time and energy consumption. We evaluate the GPOEO by applying it to 71 machine learning workloads from two AI benchmark suites running on an NVIDIA RTX3080Ti GPU. Compared with the NVIDIA default scheduling strategy, GPOEO delivers a mean energy saving of 16.2% with a modest average execution time increase of 5.1%. [ABSTRACT FROM AUTHOR]

The focus of few shot learning research has been on the development of meta-learning recently, where a meta-learner is trained on a variety of tasks in hopes of being generalizable to new tasks. Tasks in meta training and meta test are usually assumed to be from the same domain, which would not necessarily hold in real world scenarios. In this paper, we propose variational hyperparameter inference for few-shot learning across domains. Based on an especially successful algorithm named model agnostic meta learning, the proposed variational hyperparameter inference integrates meta learning and variational inference into the optimization of hyperparameters, which enables the meta-learner with adaptivity for generalization across domains. In particular, we choose to learn adaptive hyperparameters including the learning rate and weight decay to avoid the failure in the face of few labeled examples across domain. Moreover, we model hyperparameters as distributions instead of fixed values, which will further enhance the generalization ability by capturing the uncertainty. Extensive experiments are conducted on two benchmark datasets including few shot learning dataset within-domain and across-domain. The results demonstrate that our methods outperforms previous approaches consistently, and comprehensive ablation studies further validate its effectiveness on few shot learning both within domains and across domains. [ABSTRACT FROM AUTHOR]

The transportation sector is one among the key sources of greenhouse gas emissions (GHGs) leading to climate change and global warming. Energy transition through electrified transportation is one of the solutions to tackle the issues. Electric vehicles (EVs) offer significant environmental and economic advantages against the conventional Internal Combustion Engine (ICE) vehicles. EVs are called mobility loads and their connectivity to the utility grid for charging is unpredictable. The large penetration of such unpredictable loads into the utility grid will lead to undesirable impacts on the utility service. This paper highlights the importance of managing and optimizing the charging schedules. The optimization of EV charging has diverse aspects, and the perspectives of EV charging differ among consumers, aggregators, and utility services. Proper planning and management of EV charging is essential to achieve harmony amongst these stakeholders. A comprehensive review on the objectives of electric vehicle charging optimization from various perspectives is presented and discussed in this paper. EV charging optimization techniques including mathematical programming, meta heuristics algorithms and machine learning techniques are explored. The main objectives, constraints, strength, and limitations of different charging optimization techniques are analyzed in detail. A brief discussion on the communication strategies for data exchange in EV charging framework is presented and the need for a communication security constrained EV charging scheduling is also emphasized.

Reconfiguration of a distribution network is one of the main approaches to control and enhance distribution network indices, such as voltage profile and power losses. Distribution network operators perform reconfiguration for long-term or short-term periods based on network equipment and intended objectives. Long-term or static reconfiguration is suitable for traditional and modern networks with conventional switches. On the other hand, modern distribution networks that are equipped with one or more remote control switches can perform reconfigurations within short-term periods, to maximize predefined objectives. This paper presents a comprehensive review of recent literature on network reconfiguration. Reconfiguration methodologies are classified into five groups: classical methods, heuristic methods, metaheuristic methods, hybrid methods, and methods based on machine learning. The paper provides a general definition and comparison of the categories and discusses their application in dynamic and static reconfiguration. The paper introduces dynamic reconfiguration as the future challenges in smart and modern distribution networks and for the first time categorizes various methodologies in dynamic reconfiguration. The paper serves as a guide to assist engineers and researchers in selecting the most suitable methodology based on their system equipment and objectives.

In real-world applications, it is important for machine learning algorithms to be robust against data outliers or corruptions. In this paper, we focus on improving the robustness of a large class of learning algorithms that are formulated as low-rank semi-definite programming (SDP) problems. Traditional formulations use the square loss, which is notorious for being sensitive to outliers. We propose to replace this with more robust noise models, including the $\ell _1$ ℓ 1 -loss and other nonconvex losses. However, the resultant optimization problem becomes difficult as the objective is no longer convex or smooth. To alleviate this problem, we design an efficient algorithm based on majorization-minimization. The crux is on constructing a good optimization surrogate, and we show that this surrogate can be efficiently obtained by the alternating direction method of multipliers (ADMM). By properly monitoring ADMM's convergence, the proposed algorithm is empirically efficient and also theoretically guaranteed to converge to a critical point. Extensive experiments are performed on four machine learning applications using both synthetic and real-world data sets. Results show that the proposed algorithm is not only fast but also has better performance than the state-of-the-arts. [ABSTRACT FROM AUTHOR]

Sparsity-constrained optimization problems are common in machine learning, such as sparse coding, low-rank minimization and compressive sensing. However, most of previous studies focused on constructing various hand-crafted sparse regularizers, while little work was devoted to learning adaptive sparse regularizers from given input data for specific tasks. In this paper, we propose a deep sparse regularizer learning model that learns data-driven sparse regularizers adaptively. Via the proximal gradient algorithm, we find that the sparse regularizer learning is equivalent to learning a parameterized activation function. This encourages us to learn sparse regularizers in the deep learning framework. Therefore, we build a neural network composed of multiple blocks, each being differentiable and reusable. All blocks contain learnable piecewise linear activation functions which correspond to the sparse regularizer to be learned. Furthermore, the proposed model is trained with back propagation, and all parameters in this model are learned end-to-end. We apply our framework to multi-view clustering and semi-supervised classification tasks to learn a latent compact representation. Experimental results demonstrate the superiority of the proposed framework over state-of-the-art multi-view learning models. [ABSTRACT FROM AUTHOR]

Spatial dose non-uniformity is a key variation of concern in ion implantation. These non-uniformities are often compensated for by adjusting the implantation times spent at each point on the wafer: areas with a low uncompensated dose are assigned more implantation time, while those with a greater dose are assigned reduced times. In this paper, we present a machine learning based method that rapidly learns a set of compensating implantation times in order to achieve a desired uniformity.We propose an iterative tuning approach comprised of two components. The first component is an empirical Bayesian forward model that infers the relationship between the implantation times and the implantation dose profile. The model is updated as new implantations are performed and measured, enabling progressive accuracy improvements during tuning. The second component is an optimization method that selects compensating times by solving a constrained optimization problem. When tuning a process, we alternate between these two components, repeatedly selecting compensating times, measuring the resulting dose profile and updating the model until the desired uniformity is achieved. Our proposed method is compared to the conventional non-Bayesian industry method of record, and converges to a desired uniformity in significantly fewer iterations. Finally, the solutions found by our method result in a greater total dose given the same total implantation time and uniformity, increasing the throughput of implantation with no added cost. [ABSTRACT FROM AUTHOR]

This paper proposes an integrated dispatching framework for matching drivers with riders in ride-hailing systems. The goal is to compute matching solutions that maximize social welfare and benefit both sides of the market, such that the sustainable growth of the ride-hailing system is ensured. The proposed framework integrates data-driven proactive guidance strategies with batched matching optimization to increase social welfare, improve matching rate and reduce rider wait time. Proactive guidance strategies are computed by leveraging short-term demand forecasts based on historical data. Taken the resulting guidance strategies as inputs, the batched matching algorithm computes optimal bipartite matching between drivers and riders in a batch. Using New York City taxi data from 2016 March 1st to March 31st as input, we conduct a numerical study to evaluate the performance of the proposed framework and compare it with existing approaches in the literature. Our results show that the proposed framework improves social welfare for up to 50%. It also increases the matching rate by an average of 20% and reduces the average rider wait time by over 15%. This implies a strong potential for the proposed dispatching framework to improve service quality in ride-hailing systems. [ABSTRACT FROM AUTHOR]

Non-orthogonal multiple access (NOMA) is a key technology to enable massive machine type communications (mMTC) in 5G networks and beyond. In this paper, NOMA is applied to improve the random access efficiency in high-density spatially-distributed multi-cell wireless IoT networks, where IoT devices contend for accessing the shared wireless channel using an adaptive $p$ -persistent slotted Aloha protocol. To enable a capacity-optimal network, a novel formulation of random channel access management is proposed, in which the transmission probability of each IoT device is tuned to maximize the geometric mean of users’ expected capacity. It is shown that the network optimization objective is high dimensional and mathematically intractable, yet it admits favourable mathematical properties that enable the design of efficient data-driven algorithmic solutions which do not require a priori knowledge of the channel model or network topology. A centralized model-based algorithm and a scalable distributed model-free algorithm, are proposed to optimally tune the transmission probabilities of IoT devices and attain the maximum capacity. The convergence of the proposed algorithms to the optimal solution is further established based on convex optimization and game-theoretic analysis. Extensive simulations demonstrate the merits of the novel formulation and the efficacy of the proposed algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we investigate an intelligent reflecting surface (IRS) assisted multi-user multiple-input multiple-output (MIMO) full-duplex (FD) system. We jointly optimize the active beamforming matrices at the access point (AP) and uplink users, and the passive beamforming matrix at the IRS to maximize the weighted sum-rate of the system. Since it is practically difficult to acquire the channel state information (CSI) for IRS-related links due to its passive operation and large number of elements, we conceive a mixed-timescale beamforming scheme. Specifically, the high-dimensional passive beamforming matrix at the IRS is updated based on the channel statistics while the active beamforming matrices are optimized relied on the low-dimensional real-time effective CSI at each time slot. We propose an efficient stochastic successive convex approximation (SSCA)-based algorithm for jointly designing the active and passive beamforming matrices. Moreover, due to the high computational complexity caused by the matrix inversion computation in the SSCA-based optimization algorithm, we further develop a deep-unfolding neural network (NN) to address this issue. The proposed deep-unfolding NN maintains the structure of the SSCA-based algorithm but introduces a novel non-linear activation function and some learnable parameters induced by the first-order Taylor expansion to approximate the matrix inversion. In addition, we develop a black-box NN as a benchmark. Simulation results show that the proposed mixed-timescale algorithm outperforms the existing single-timescale algorithm and the proposed deep-unfolding NN approaches the performance of the SSCA-based algorithm with much reduced computational complexity when deployed online. [ABSTRACT FROM AUTHOR]

In this paper, a new method is proposed for creating an optimized ensemble classifier. The proposed method mitigates the issue of class imbalances by partitioning the input data into its various data classes. The partitions are then clustered incrementally to generate a pool of class pure data clusters. The generated data clusters are then balanced by adding samples from all classes which are closest to the cluster centroid. In this manner all generated data clusters are balanced and classifiers trained on such a data cluster are unbiased as well. This creates a diverse input space for training of base classifiers. The pool of clusters is then utilized to train a set of diverse base classifiers to generate the base classifier pool. The pool of classifiers is then treated as a combinatorial problem of optimization and an evolutionary algorithm is incorporated. The proposed approach generates an optimized ensemble classifier that can not only achieve the highest classification accuracy but also has a lower component size as well. The proposed approach is tested on 31 benchmark datasets from UCI machine learning repository and results are compared with existing state-of-the-art ensemble classifiers as well. [ABSTRACT FROM AUTHOR]

For vehicle-to-network communications, handover (HO) management enables vehicles to maintain the connection with the network while transiting through coverage areas of different base stations (BSs). However, the high mobility of vehicles means shorter connection periods with each BS that leads to frequent HOs, hence raises the necessity for optimal HO decision making for high quality infotainment services. Machine learning is capable of capturing underlying pattern via data driven methods to find optimal solutions to complex problems, and much learning-based HO optimization research has been conducted focusing on specific network setups. However, attention still needs to be paid to the actual deployment aspect and standardized datasets or simulation environments for evaluation. This paper proposes a deep reinforcement learning-based HO algorithm using the input parameters that are configurable in the existing measurement report of cellular networks. The performance of the proposed algorithm is evaluated using the well-known network simulator ns-3 with its official LTE module. A realistic network setup in the city center of Glasgow (U.K.) is configured with vehicle trajectories generated by the routes mobility model using Google Maps Directions API. Evaluation results reveal that the proposed algorithm significantly outperforms the A3 RSRP baseline with an average of 25.72% packet loss reduction per HO, suggesting significant improvement in quality of service of phone call and video streaming, etc. The proposed algorithm also has a small implementation cost compared to some state-of-the-art and should be deployed by a software update to a local BS controller. [ABSTRACT FROM AUTHOR]

Person detection has attracted great attention in the computer vision area and is an imperative element in human-centric computer vision. Although the predictive performances of person detection networks have been improved dramatically, they are vulnerable to adversarial patch attacks. Changing the pixels in a restricted region can easily fool the person detection network in safety-critical applications such as autonomous driving and security systems. Despite the necessity of countering adversarial patch attacks, very few efforts have been dedicated to defending person detection against adversarial patch attack. In this paper, we propose a novel defense strategy that defends against an adversarial patch attack by optimizing a defensive frame for person detection. The defensive frame alleviates the effect of the adversarial patch while maintaining person detection performance with clean person. The proposed defensive frame in the person detection is generated with a competitive learning algorithm which makes an iterative competition between detection threatening module and detection shielding module in person detection. Comprehensive experimental results demonstrate that the proposed method effectively defends person detection against adversarial patch attacks. [ABSTRACT FROM AUTHOR]

This paper considers stochastic convex optimization problems with two sets of constraints: (a) deterministic constraints on the domain of the optimization variable, which are difficult to project onto; and (b) deterministic or stochastic constraints that admit efficient projection. Problems of this form arise frequently in the context of semidefinite programming as well as when various NP-hard problems are solved approximately via semidefinite relaxation. Since projection onto the first set of constraints is difficult, it becomes necessary to explore projection-free algorithms, such as the stochastic Frank-Wolfe (FW) algorithm. On the other hand, the second set of constraints cannot be handled in the same way, and must be incorporated as an indicator function within the objective function, thereby complicating the application of FW methods. Similar problems have been studied before; however, they suffer from slow convergence rates. This work, equipped with momentum based gradient tracking technique, guarantees fast convergence rates on par with the best-known rates for problems without the second set of constraints. Zeroth-order variants of the proposed algorithms are also developed and again improve upon the state-of-the-art rate results. We further propose the novel trimmed FW variants that enjoy the same convergence rates as their classical counterparts, but are empirically shown to require significantly fewer calls to the linear minimization oracle speeding up the overall algorithm. The efficacy of the proposed algorithms is tested on relevant applications of sparse matrix estimation, clustering via semidefinite relaxation, and uniform sparsest cut problem. [ABSTRACT FROM AUTHOR]

Machine learning can provide deep insights into data, allowing machines to make high-quality predictions and having been widely used in real-world applications, such as text mining, visual classification, and recommender systems. However, most sophisticated machine learning approaches suffer from huge time costs when operating on large-scale data. This issue calls for the need of Large-scale Machine Learning (LML), which aims to learn patterns from big data with comparable performance efficiently. In this paper, we offer a systematic survey on existing LML methods to provide a blueprint for the future developments of this area. We first divide these LML methods according to the ways of improving the scalability: 1) model simplification on computational complexities, 2) optimization approximation on computational efficiency, and 3) computation parallelism on computational capabilities. Then we categorize the methods in each perspective according to their targeted scenarios and introduce representative methods in line with intrinsic strategies. Lastly, we analyze their limitations and discuss potential directions as well as open issues that are promising to address in the future. [ABSTRACT FROM AUTHOR]

Many signal processing and machine learning problems can be formulated as consensus optimization problems which can be solved efficiently via a cooperative multi-agent system. However, the agents in the system can be unreliable due to a variety of reasons: noise, faults and attacks. Providing erroneous updates leads the optimization process in a wrong direction, and degrades the performance of distributed machine learning algorithms. This paper considers the problem of decentralized learning using ADMM in the presence of unreliable agents. First, we rigorously analyze the effect of erroneous updates (in ADMM learning iterations) on the convergence behavior of the multi-agent system. We show that the algorithm linearly converges to a neighborhood of the optimal solution under certain conditions and characterize the neighborhood size analytically. Next, we provide guidelines for network design to achieve a faster convergence to the neighborhood. We also provide conditions on the erroneous updates for exact convergence to the optimal solution. Finally, to mitigate the influence of unreliable agents, we propose ROAD, a robust variant of ADMM, and show its resilience to unreliable agents with an exact convergence to the optimum. [ABSTRACT FROM AUTHOR]

Actuated by the growing attention to personal healthcare and the pandemic, the popularity of E-health is proliferating. Nowadays, enhancement on medical diagnosis via machine learning models has been highly effective in many aspects of e-health analytics. Nevertheless, in the classic cloud-based/centralized e-health paradigms, all the data will be centrally stored on the server to facilitate model training, which inevitably incurs privacy concerns and high time delay. Distributed solutions like Decentralized Stochastic Gradient Descent (D-SGD) are proposed to provide safe and timely diagnostic results based on personal devices. However, methods like D-SGD are subject to the gradient vanishing issue and usually proceed slowly at the early training stage, thereby impeding the effectiveness and efficiency of training. In addition, existing methods are prone to learning models that are biased towards users with dense data, compromising the fairness when providing E-health analytics for minority groups. In this paper, we propose a Decentralized Block Coordinate Descent (D-BCD) learning framework that can better optimize deep neural network-based models distributed on decentralized devices for E-health analytics. As a gradient-free optimization method, Block Coordinate Descent (BCD) mitigates the gradient vanishing issue and converges faster at the early stage compared with the conventional gradient-based optimization. To overcome the potential data scarcity issues for users’ local data, we propose similarity-based model aggregation that allows each on-device model to leverage knowledge from similar neighbor models, so as to achieve both personalization and high accuracy for the learned models. Benchmarking experiments on three real-world datasets illustrate the effectiveness and practicality of our proposed D-BCD, where additional simulation study showcases the strong applicability of D-BCD in real-life E-health scenarios. [ABSTRACT FROM AUTHOR]

Federated learning (FL) is a popular edge learning approach that utilizes local data and computing resources of network edge devices to train machine learning (ML) models while preserving users’ privacy. Nevertheless, performing efficient learning tasks on the devices and achieving longer battery life are primary challenges faced by federated learning. In this paper, we are the first to study the application of heterogeneous computing (HC) and wireless power transfer (WPT) to federated learning to address these challenges. Especially, we propose a heterogeneous computation and resource allocation framework based on a heterogeneous mobile architecture to achieve effective implementation of FL. To minimize the energy consumption of smart devices and maximize their harvesting energy simultaneously, we formulate an optimization problem featuring multidimensional control, which jointly considers time splitting for WPT, dataset size allocation, transmit power allocation and subcarrier assignment during communications, and processor frequency of processing units (central processing unit (CPU) and graphics processing unit (GPU)). However, the major obstacle is how to design a proper algorithm to solve this optimization problem efficiently. For this purpose, we decouple the optimization variables so as to achieve high efficiency in deriving its solution. Particularly, we first compute the optimal processor frequency and dataset size allocation via employing the Lagrangian dual method, followed by finding the closed-form solution to the optimal time splitting allocation, and finally attain the optimal subcarrier assignment as well as transmit power for transmissions through an iteration algorithm. To evaluate the performance of our proposed scheme, we set up four baseline schemes as comparison, and simulation results show that the proposed scheme converges quite fast and better enhance the energy efficiency of the wireless powered FL system compared with the baseline schemes. [ABSTRACT FROM AUTHOR]

$k$ k -means method using Lloyd heuristic is a traditional clustering method which has played a key role in multiple downstream tasks of machine learning because of its simplicity. However, Lloyd heuristic always finds a bad local minimum, i.e., the bad local minimum makes objective function value not small enough, which limits the performance of $k$ k -means. In this paper, we use coordinate descent (CD) method to solve the problem. First, we show that the $k$ k -means minimization problem can be reformulated as a trace maximization problem, then a simple and efficient coordinate descent scheme is proposed to solve the maximization problem. Two interesting findings through theory are that Lloyd cannot decrease the objective function value of $k$ k -means produced by our CD further, and our proposed method CD to solve $k$ k -means problem can avoid produce empty clusters. In addition, according to the computational complexity analysis, it is verified CD has the same time complexity with original $k$ k -means method. Extensive experiments including statistical hypothesis testing, on several real-world datasets with varying number of clusters, varying number of samples and varying number of dimensions show that CD performs better compared to Lloyd, i.e., lower objective value, better local minimum and fewer iterations. And CD is more robust to initialization than Lloyd whether the initialization strategy is random or initialization of $k$ k -means++. [ABSTRACT FROM AUTHOR]

With the help of auxiliary data, tensor completion may better recover a low rank multidimensional array from limited observation entries. Most existing methods, including coupled matrix-tensor factorization and coupled tensor rank minimization, mainly focus on how to extract and incorporate subspace or directly use auxiliary data for tensor completion. They are either sensitive to a given rank or lack of physical interpretations of subspace information. In addition, the shared subspace information receives little attention in current tensor completion methods, especially there is no analysis of its impact on sample complexity. In this paper, we propose to separately explore and exploit shared subspaces for tensor completion. Specifically, dictionary learning takes the subspace from auxiliary data in the first step. Then a low rank optimization model for tensor completion is provided to incorporate the trained subspace by assuming that the recovered tensor is composed of two low rank components where one shares the subspace information with auxiliary data and the other is outside the shared space. Based on this optimization model, we make a quantitative analysis to illustrate the effect of subspace information on sample complexity, and provide theoretical insights into the usefulness of subspace information. Finally, experiments on simulated data are conducted to validate the theoretical analysis on the impact of subspace information. Experiments in two real-world applications including color image and multispectral image recovery show that the proposed method outperforms state-of-the-art ones in terms of prediction accuracy and CPU time. [ABSTRACT FROM AUTHOR]

Dual Coordinate Descent (DCD) is one of the most popular optimization methods. The parallelization of DCD is difficult, as DCD is sequential in nature. As such, simultaneously running multiple DCD threads on batches of data elements causes result inaccuracy and slow convergence, due to the concurrent updates of multiple coordinates. Some parallelization methods adopt separable approximate functions that depend on the degree of parallelism. Such dependencies result in both poor scalability and slow-convergence. To address these challenges, in this paper we present a new parallel primal-dual algorithm for DCD, called PPD. In PPD, the block data distribution is utilized to obtain a new approximate function that is independent from the parallelism. Moreover, PPD is designed with a novel primal-dual acceleration scheme to approach the optimal solution closely and quickly. We demonstrate the advantages of PPD in terms of scalability and efficiency through experiments. [ABSTRACT FROM AUTHOR]

Data representation techniques have made a substantial contribution to advancing data processing and machine learning (ML). Improving predictive power was the focus of previous representation techniques, which unfortunately perform rather poorly on the interpretability in terms of extracting underlying insights of the data. Recently, the Kolmogorov model (KM) was studied, which is an interpretable and predictable representation approach to learning the underlying probabilistic structure of a set of random variables. The existing KM learning algorithms using semi-definite relaxation with randomization (SDRwR) or discrete monotonic optimization (DMO) have, however, limited utility to big data applications because they do not scale well computationally.In this paper, we propose a computationally scalable KM learning algorithm, based on the regularized dual optimization combined with enhanced gradient descent (GD) method. To make our method more scalable to large-dimensional problems, we propose two acceleration schemes, namely, the eigenvalue decomposition (EVD) elimination strategy and an approximate EVD algorithm. Furthermore, a thresholding technique by exploiting the error bound analysis and leveraging the normalized Minkowski $\ell _1$ -norm, is provided for the selection of the number of iterations of the approximate EVD algorithm. When applied to big data applications, it is demonstrated that the proposed method can achieve compatible training/prediction performance with significantly reduced computational complexity; roughly two orders of magnitude improvement in terms of the time overhead, compared to the existing KM learning algorithms. Furthermore, it is shown that the accuracy of logical relation mining for interpretability by using the proposed KM learning algorithm exceeds 80%. [ABSTRACT FROM AUTHOR]

Coded computation techniques provide robustness against straggling workers in distributed computing. However, most of the existing schemes require exact provisioning of the straggling behavior and ignore the computations carried out by straggling workers. Moreover, these schemes are typically designed to recover the desired computation results accurately, while in many machine learning and iterative optimization algorithms, faster approximate solutions are known to result in an improvement in the overall convergence time. In this paper, we first introduce a novel coded matrix-vector multiplication scheme, called coded computation with partial recovery (CCPR), which benefits from the advantages of both coded and uncoded computation schemes, and reduces both the computation time and the decoding complexity by allowing a trade-off between the accuracy and the speed of computation. We then extend this approach to distributed implementation of more general computation tasks by proposing a coded communication scheme with partial recovery, where the results of subtasks computed by the workers are coded before being communicated. Numerical simulations on a large linear regression task confirm the benefits of the proposed scheme in terms of the trade-off between the computation accuracy and latency. [ABSTRACT FROM AUTHOR]

Due to the exponential growth of multimedia data, multi-modal hashing as a promising technique to make cross-view retrieval scalable is attracting more and more attention. However, most of the existing multi-modal hashing methods either divide the learning process unnaturally into two separate stages or treat the discrete optimization problem simplistically as a continuous one, which leads to suboptimal results. Recently, a few discrete multi-modal hashing methods that try to address such issues have emerged, but they still ignore several important discrete constraints (such as the balance and decorrelation of hash bits). In this paper, we overcome those limitations by proposing a novel method named “Enhanced Discrete Multi-modal Hashing (EDMH)” which learns binary codes and hashing functions simultaneously from the pairwise similarity matrix of data, under the aforementioned discrete constraints. Although the model of EDMH looks a lot more complex than the other models for multi-modal hashing, we are actually able to develop a fast iterative learning algorithm for it, since the subproblems of its optimization all have closed-form solutions after introducing a couple of auxiliary variables. Our experimental results on three real-world datasets have revealed the usefulness of those previously ignored discrete constraints and demonstrated that EDMH not only performs much better than state-of-the-art competitors according to several retrieval metrics but also runs much faster than most of them. [ABSTRACT FROM AUTHOR]

The primary goal of any financial investor is to maximize returns and minimize risks. The selection of the right assets and appropriate allocation of principal wealth is highly essential for the same. Investment specialists and portfolio managers have to manually analyze assets, determine potential returns and risks, and adhere to client preferences to create optimal portfolios. It is a complex and grueling process and demands significant skill and relevant experience for success. Like all other sectors in finance, machine learning models have seen tremendous success in the portfolio selection problem as well. In this article, a thorough review of several machine learning portfolio optimization techniques such as clustering based, Support Vector Machines based, genetic algorithm based, and more has been presented. This article, in its entirety, condenses and interprets the numerous approaches and the merits and limitations corresponding to their implementation. The conclusions presented by this review can be utilized to identify the advantages of these papers. This will help future researchers in their study of the domain and ensure the availability of essential data for further analysis in a systematic and comprehensive manner. With the help of this article, researchers will be well placed to identify areas that present scope for improvement and come up with novel or possibly hybrid techniques to achieve near perfection in portfolio selection. [ABSTRACT FROM AUTHOR]

Learning to improve AUC performance for imbalanced data is an important machine learning research problem. Most methods of AUC maximization assume that the model function is linear in the original feature space. However, this assumption is not suitable for nonlinear separable problems. Although there have been some nonlinear methods of AUC maximization, scaling up nonlinear AUC maximization is still an open question. To address this challenging problem, in this paper, we propose a novel large-scale nonlinear AUC maximization method (named as TSAM) based on the triply stochastic gradient descents. Specifically, we first use the random Fourier feature to approximate the kernel function. After that, we use the triply stochastic gradients w.r.t. the pairwise loss and random feature to iteratively update the solution. Finally, we prove that TSAM converges to the optimal solution with the rate of $ \mathcal {O}(1/t)$ O (1 / t) after $t$ t iterations. Experimental results on a variety of benchmark datasets not only confirm the scalability of TSAM, but also show a significant reduction of computational time compared with existing batch learning algorithms, while retaining the similar generalization performance. [ABSTRACT FROM AUTHOR]

Despite many studies have been published on non-technical loss management, there is a lack of solutions focused in the financial return of utilities’ actions. This paper aims to contribute to fill up this gap, presenting a new approach that maximizes the financial return of utilities when selecting consumers for field inspections. The predicted return of an inspection is calculated considering its operational cost versus the potential of energy recovery and the tariff value. The potential of energy recovery, in turn, is calculated considering the probability of non-technical loss existence and the forecast of unmeasured energy. The Rotation Forest algorithm is utilized to indicate the non-technical loss existence, while the eXtreme Gradient Boosting algorithm is utilized for energy recovery forecast. The proposed approach was applied in a real database of 261,489 consumers from a Brazilian utility. Then, 338 new field inspections were performed on unlabeled consumers in order to corroborate results in a real application. Achieved results showed an increase of up to 11.5 times in the financial return of field inspections when applying the proposed approach. Results could also be used to determine the optimal number of inspections in the utility, for which the financial return is maximum. [ABSTRACT FROM AUTHOR]