Search

Fighting against the pandemic diseases with unique characters requires new sophisticated approaches like the artificial intelligence. This paper develops an artificial intelligence algorithm to produce multi-dimensional policies for controlling and minimizing the pandemic casualties under the limited pharmacological resources. In this respect, a comprehensive parametric model with a priority and age-specific vaccination policy and a variety of non-pharmacological policies are introduced. This parametric model is utilized for constructing an artificial intelligence algorithm by following the exact analogy of the model-based solution. Also, this parametric model is manipulated by the artificial intelligence algorithm to seek for the best multi-dimensional non-pharmacological policies that minimize the future pandemic casualties as desired. The role of the pharmacological and non-pharmacological policies on the uncertain future casualties are extensively addressed on the real data. It is shown that the developed artificial intelligence algorithm is able to produce efficient policies which satisfy the particular optimization targets such as focusing on minimization of the death casualties more than the infected casualties or considering the curfews on the people age over 65 rather than the other non-pharmacological policies. The paper finally analyses a variety of the mutant virus cases and the corresponding non-pharmacological policies aiming to reduce the morbidity and mortality rates. [ABSTRACT FROM AUTHOR]

Fighting against the pandemic diseases with unique characters requires new sophisticated approaches like the artificial intelligence. This paper develops an artificial intelligence algorithm to produce multi-dimensional policies for controlling and minimizing the pandemic casualties under the limited pharmacological resources. In this respect, a comprehensive parametric model with a priority and age-specific vaccination policy and a variety of non-pharmacological policies are introduced. This parametric model is utilized for constructing an artificial intelligence algorithm by following the exact analogy of the model-based solution. Also, this parametric model is manipulated by the artificial intelligence algorithm to seek for the best multi-dimensional non-pharmacological policies that minimize the future pandemic casualties as desired. The role of the pharmacological and non-pharmacological policies on the uncertain future casualties are extensively addressed on the real data. It is shown that the developed artificial intelligence algorithm is able to produce efficient policies which satisfy the particular optimization targets such as focusing on minimization of the death casualties more than the infected casualties or considering the curfews on the people age over 65 rather than the other non-pharmacological policies. The paper finally analyses a variety of the mutant virus cases and the corresponding non-pharmacological policies aiming to reduce the morbidity and mortality rates. [ABSTRACT FROM AUTHOR]

The Internet of vehicles (IoV) is a large information interaction network that collects information on vehicles, roads and pedestrians. One of the important uses of vehicle networks is to meet the entertainment needs of driving users through communication between vehicles and roadside units (RSUs). Due to the limited storage space of RSUs, determining the content cached in each RSU is a key challenge. With the development of 5G and video editing technology, short video systems have become increasingly popular. Current widely used cache update methods, such as partial file precaching and content popularity- and user interest-based determination, are inefficient for such systems. To solve this problem, this paper proposes a QoE-driven edge caching method for the IoV based on deep reinforcement learning. First, a class-based user interest model is established. Compared with the traditional file popularity- and user interest distribution-based cache update methods, the proposed method is more suitable for systems with a large number of small files. Second, a quality of experience (QoE)-driven RSU cache model is established based on the proposed class-based user interest model. Third, a deep reinforcement learning method is designed to address the QoE-driven RSU cache update issue effectively. The experimental results verify the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

This paper proposes a new decentralized data-driven load restoration (DDLR) scheme for transmission and distribution (TD) systems with high penetration of wind power. Robust DDLR models are constructed in order to handle uncertainties and ensure the feasibility of decentralized schemes. The Wasserstein metric is used to describe the ambiguity sets of probability distributions in order to build the complete DDLR model and realize computationally tractable formulation. A data-driven model-nested analytical target cascading (DATC) algorithm is developed to obtain the final load restoration result by iteratively solving small-scale mathematical models. The proposed DDLR scheme provides load restoration results with adjustable robustness, and performance efficiency is independent from the amount of data. The DDLR scheme makes full use of the available data while respecting information privacy requirements of independent operated systems, and ensures the feasibility of a decentralized load restoration strategy even in the worst-case condition. The effectiveness of the proposed method is validated using a small-scale TD system and a large-scale system with the IEEE 118-bus TS and thirty IEEE-33 DSs, showing high computational efficiency and superior restoration performance. [ABSTRACT FROM AUTHOR]

Nonlinear optimization problems with dynamical parameters are widely arising in many practical scientific and engineering applications, and various computational models are presented for solving them under the hypothesis of short-time invariance. To eliminate the large lagging error in the solution of the inherently dynamic nonlinear optimization problem, the only way is to estimate the future unknown information by using the present and previous data during the solving process, which is termed the future dynamic nonlinear optimization (FDNO) problem. In this paper, to suppress noises and improve the accuracy in solving FDNO problems, a novel noise-tolerant neural (NTN) algorithm based on zeroing neural dynamics is proposed and investigated. In addition, for reducing algorithm complexity, the quasi-Newton Broyden–Fletcher–Goldfarb–Shanno (BFGS) method is employed to eliminate the intensively computational burden for matrix inversion, termed NTN-BFGS algorithm. Moreover, theoretical analyses are conducted, which show that the proposed algorithms are able to globally converge to a tiny error bound with or without the pollution of noises. Finally, numerical experiments are conducted to validate the superiority of the proposed NTN and NTN-BFGS algorithms for the online solution of FDNO problems. [ABSTRACT FROM AUTHOR]

Federated learning (FL) as a promising edge-learning framework can effectively address the latency and privacy issues by featuring distributed learning at the devices and model aggregation in the central server. In order to enable efficient wireless data aggregation, over-the-air computation (AirComp) has recently attracted great attention. However, fading of wireless channels can produce aggregate distortions in an AirComp-based FL scheme. In this paper, we propose a modified federated averaging (FedAvg) algorithm by introducing the local learning rates and present the convergence analysis. To combat the distortion, the local learning rate is optimized to adapt the fading channel, which is termed as dynamic learning rate (DLR). We begin our discussion by considering multiple-input-single-output (MISO) scenario, since the underlying optimization problem is convex and has a closed-form solution. Our studies are extended to a more general multiple-input-multiple-output (MIMO) case and an iterative method is derived. We also present the asymptotic analysis and give a near-optimal and closed-form receive beamforming solution when the number of antennas approaches infinity. Extensive simulation results demonstrate the effectiveness of the proposed DLR scheme in reducing the aggregate distortion and guaranteeing the testing accuracy on the MNIST and CIFAR10 datasets. In addition, the asymptotic analysis and the close-form solution are verified through numerical simulations. [ABSTRACT FROM AUTHOR]

In network optimization problems, from traffic engineering to network monitoring, the routing model is typically considered as something given and frozen. This paper is motivated by the fundamental question how the ability to change and optimize the routing model itself influences the efficiency at which communication networks can be operated. To this end, we identify two main dimensions of a routing model: consistency (of a single route) and coherence (of sets of routes). We present analytical results on the impact of the routing model on the achievable route diversity as well as on the runtime of solving optimization problems underlying different case studies. We also uncover that it can sometimes be beneficial to artificially restrict the routing model, to significantly reduce the computational complexity without negatively affecting the route diversity much. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of multiple graph matching (MGM) by considering both offline batch mode and online setting. We explore the concept of cycle-consistency over pairwise matchings and formulate the problem as finding optimal composition path on the supergraph, whose vertices refer to graphs and edge weights denote score function regarding consistency and affinity. By our theoretical study we show that the offline and online MGM on supergraph can be converted to finding all pairwise shortest paths and single-source shortest paths respectively. We adopt the Floyd algorithm and shortest path faster algorithm (SPFA) , to effectively find the optimal path. Extensive experimental results show our methods surpass state-of-the-art MGM methods, including CAO , MISM , IMGM , and many other recent methods in offline and online settings. Source code will be made publicly available. [ABSTRACT FROM AUTHOR]

Clustering is a widely used machine learning technique for unlabelled data. One of the recently proposed techniques is the twin support vector clustering (TWSVC) algorithm. The idea of TWSVC is to generate hyperplanes for each cluster. TWSVC utilizes the hinge loss function to penalize the misclassification. However, the hinge loss relies on shortest distance between different clusters, and is unstable for noise-corrupted datasets, and for re-sampling. In this paper, we propose a novel Sparse Pinball loss Twin Support Vector Clustering (SPTSVC). The proposed SPTSVC involves the $\epsilon$ -insensitive pinball loss function to formulate a sparse solution. Pinball loss function provides noise-insensitivity and re-sampling stability. The $\epsilon$ -insensitive zone provides sparsity to the model and improves testing time. Numerical experiments on synthetic as well as real world benchmark datasets are performed to show the efficacy of the proposed model. An analysis on the sparsity of various clustering algorithms is presented in this work. In order to show the feasibility and applicability of the proposed SPTSVC on biomedical data, experiments have been performed on epilepsy and breast cancer datasets. [ABSTRACT FROM AUTHOR]

Black-box optimization algorithms typically start a search from scratch, assuming little prior knowledge about the task at hand. In practice, this approach can be prohibitive for computationally expensive problems, as a large number of costly function evaluations are often needed before a suitable (near-optimal) solution is found. Under this observation, recent efforts have incorporated transfer learning capabilities into sequential model-based Bayesian optimization (BO) solvers, resulting in substantial performance speed-ups by leveraging information from related past problems. However, a common simplifying assumption in existing approaches is that the search spaces of a previously encountered source and the ongoing target task bear the same features and dimensionality, with the difference lying in their respective objective functions. In this article, we present a generalized transfer BO algorithm that relaxes the aforementioned assumption. Our method jointly transforms source features while training probabilistic transfer regression models for the target, thus applying to practical use-cases where (in addition to the difference in objective functions) the number of features could change across the source and target tasks; for example, features can be added and/or removed. The theoretical basis of our proposal is analyzed, and its empirical performance is demonstrated on synthetic benchmark functions as well as in realistic examples spanning engineering design and the automated configuration of a machine learning model. Note to Practitioners—Problems of industrial interest have a tendency of being repetitive in nature. For this reason, domain experts are always in high demand, as they are able to harness their experience of similar problems to come up with fast solutions in difficult situations. However, domain experts are not easy to find. Given this fact, the present paper puts forth a method for automating the process of knowledge extraction (through experiential learning) and transfer across problems in the domain of computationally expensive black-box optimization. The key novelty and motivation of this work lies in enabling the adaptive transfer of knowledge even when the number of features changes across the source and target problems. Our proposed approach is verified experimentally on a range of benchmarks as well as real-world problems of a computationally expensive nature, highlighting the utility of an optimization engine that is able to learn from experience without the need for constant human intervention. [ABSTRACT FROM AUTHOR]

We propose Graph-Cut RANSAC, GC-RANSAC in short, a new robust geometric model estimation method where the local optimization step is formulated as energy minimization with binary labeling, applying the graph-cut algorithm to select inliers. The minimized energy reflects the assumption that geometric data often form spatially coherent structures – it includes both a unary component representing point-to-model residuals and a binary term promoting spatially coherent inlier-outlier labelling of neighboring points. The proposed local optimization step is conceptually simple, easy to implement, efficient with a globally optimal inlier selection given the model parameters. Graph-Cut RANSAC, equipped with “the bells and whistles” of USAC and MAGSAC++, was tested on a range of problems using a number of publicly available datasets for homography, 6D object pose, fundamental and essential matrix estimation. It is more geometrically accurate than state-of-the-art robust estimators, fails less often and runs faster or with speed similar to less accurate alternatives. The source code is available at https://github.com/danini/graph-cut-ransac. [ABSTRACT FROM AUTHOR]

Connected vehicles will change the modes of future transportation management and organization, especially at an intersection without traffic light. Centralized coordination methods globally coordinate vehicles approaching the intersection from all sections by considering their states altogether. However, they need substantial computation resources since they own a centralized controller to optimize the trajectories for all approaching vehicles in real-time. In this paper, we propose a centralized coordination scheme of automated vehicles at an intersection without traffic signals using reinforcement learning (RL) to address low computation efficiency suffered by current centralized coordination methods. We first propose an RL training algorithm, model accelerated proximal policy optimization (MA-PPO), which incorporates a prior model into proximal policy optimization (PPO) algorithm to accelerate the learning process in terms of sample efficiency. Then we present the design of state, action and reward to formulate centralized coordination as an RL problem. Finally, we train a coordinate policy in a simulation setting and compare computing time and traffic efficiency with a coordination scheme based on model predictive control (MPC) method. Results show that our method spends only 1/400 of the computing time of MPC and increase the efficiency of the intersection by 4.5 times. [ABSTRACT FROM AUTHOR]

This paper investigates resource provisioning in cyber-physical systems (CPSs) by developing a new definition of anelasticity. A flat semi-dormant multicontroller (FSDMC) model is established on a special type of CPS platform named arbitrated networked control system with dual communication channels. A novel, quantitative, and formal definition of anelasticity for the FSDMC is proposed. A new finite capacity M/M/c queuing system with ${N}$ -policy and asynchronous multiple working vacations of partial servers is established, and the FSDMC is modeled as a quasi-birth-and-death process to obtain the stationary probability distribution of the system. Based on the queueing model, we quantify various performance indices of the system to build a nonlinear cost-performance ratio (CPR) function. An optimization model is presented to minimize the CPR. A particle swarm optimization (PSO) algorithm is used to find the optimum solution of the optimization model and obtain the optimal configuration values of the system parameters under stability condition. By changing the system parameters, the sensitivity of the system performance indices and the CPR are analyzed, respectively. The unexpected workload varies randomly over time. Thus, an M/M/1/K queue is constructed in a Markovian environment by employing a three-state, irreducible Markov process. In this queue, the conditional average queue length and the probabilities of the three-state process are calculated. Then, the anelasticity value of the system is precisely determined. When the average arrival rate exceeds the average service rate in the queueing system, an optimal CPR unchanged adaptive algorithm based on PSO is designed to dynamically adjust the controller service rate. Extensive numerical results show the usefulness and effectiveness of the proposed techniques and exhibit that the system can maintain elastic invariance in adaptive adjustment parameters. [ABSTRACT FROM AUTHOR]

In recent years there has been a noticeable increase in the number of available Big Data infrastructures. This fact has promoted the adaptation of traditional machine learning techniques to be capable of addressing large scale problems in distributed environments. Kernel methods like support vector machines (SVMs) suffer from scalability problems due to their nonparametric nature and the complexity of their training procedures. In this paper, we propose a new and efficient distributed implementation of a training procedure for nonlinear semiparametric (budgeted) SVMs called distributed iterative reweighted least squares (IRWLS). This algorithm uses $ {k}$ -means to select the centroids of the semiparametric model and a new distributed algorithmic implementation of the IRWLS optimization procedure to find the weights of the model. We have implemented the proposed algorithm in Apache Spark and we have benchmarked it against other state-of-the-art methods, either full SVM ($ {p}$ -pack SVM) or budgeted (budgeted stochastic gradient descent). Experimental results show that the proposed algorithm achieves higher accuracy while controlling the size of the final model, and also offers high performance in terms of run time and efficiency, when processing very large datasets (the computation time grows linear with the number of training patterns). [ABSTRACT FROM AUTHOR]

The nonlinear programming (NLP) problem to solve distribution-level optimal power flow (D-OPF) poses convergence issues and does not scale well for unbalanced distribution systems. The existing scalable D-OPF algorithms either use approximations that are not valid for an unbalanced power distribution system, or apply relaxation techniques to the nonlinear power flow equations that do not guarantee a feasible power flow solution. In this paper, we propose scalable D-OPF algorithms that simultaneously achieve optimal and feasible solutions by solving multiple iterations of approximate, or relaxed, D-OPF subproblems of low complexity. The first algorithm is based on a successive linear approximation of the nonlinear power flow equations around the current operating point, where the D-OPF solution is obtained by solving multiple iterations of a linear programming (LP) problem. The second algorithm is based on the relaxation of the nonlinear power flow equations as conic constraints together with directional constraints, which achieves optimal and feasible solutions over multiple iterations of a second-order cone programming (SOCP) problem. It is demonstrated that the proposed algorithms are able to reach an optimal and feasible solution while significantly reducing the computation time as compared to an equivalent NLP D-OPF model for the same distribution system. [ABSTRACT FROM AUTHOR]

Modern mixed-integer quadratic solvers generally handle binary variables more efficiently than nonlinear mixed-integer solvers. This is relevant to the power system operation models as the unit commitment formulations typically contain a large number of binary variables. This paper investigates how to achieve the accuracy level close to the one of the exact nonlinear models, but by utilising convex models and solvers. The presented unit commitment model is based on a Taylor-series expansion where both the voltage magnitude and angle are quadratically constrained. To achieve high accuracy, the model takes advantage of the meshed transmission network structure that enables replacement of the quadratic inequality constraints that cause constraint relaxation errors with the linear equality constraints. Quadratic constraints to be replaced as well as the operating point parameters are determined based on the presolve. The first presented case study validates the model's accuracy and the convergence of the iterative algorithm, while the second is a non-iterative full unit commitment problem. Unit commitment results show superior accuracy and similar computation times to the existing quadratic formulations on one hand and faster computation times than the exact nonlinear polar formulation on the other. [ABSTRACT FROM AUTHOR]

The distribution network restoration problem is by nature a mixed integer and non-linear optimization problem due to the switching decisions and Optimal Power Flow (OPF) constraints, respectively. The link between these two parts involves logical implications modelled through big-M coefficients. The presence of these coefficients makes the relaxation of the mixed-integer problem using branch-and-bound method very poor in terms of computation burden. Moreover, this link inhibits the use of classical Benders algorithm in decomposing the problem because the resulting cuts will still depend on the big-M coefficients. In this paper, a novel decomposition approach is proposed for the restoration problem named Modified Combinatorial Benders (MCB). In this regard, the reconfiguration problem and the OPF problem are decomposed into master and sub problems, which are solved through successive iterations. In the case of a large outage area, the numerical results show that the MCB provides, within a short time (after a few iterations), a restoration solution with a quality that is close to the proven optimality when it can be exhibited. [ABSTRACT FROM AUTHOR]

The cloud radio access network (C-RAN) is a promising paradigm to meet the stringent requirements of the fifth generation (5G) wireless systems. Meanwhile, the wireless traffic prediction is a key enabler for C-RANs to improve both the spectrum efficiency and energy efficiency through load-aware network managements. This paper proposes a scalable Gaussian process (GP) framework as a promising solution to achieve large-scale wireless traffic prediction in a cost-efficient manner. Our contribution is three-fold. First, to the best of our knowledge, this paper is the first to empower GP regression with the alternating direction method of multipliers (ADMM) for parallel hyper-parameter optimization in the training phase, where such a scalable training framework well balances the local estimation in baseband units (BBUs) and information consensus among BBUs in a principled way for large-scale executions. Second, in the prediction phase, we fuse local predictions obtained from the BBUs via a cross-validation-based optimal strategy, which demonstrates itself to be reliable and robust for general regression tasks. Moreover, such a cross-validation-based optimal fusion strategy is built upon a well acknowledged probabilistic model to retain the valuable closed-form GP inference properties. Third, we propose a C-RAN-based scalable wireless prediction architecture, where the prediction accuracy and the time consumption can be balanced by tuning the number of the BBUs according to the real-time system demands. The experimental results show that our proposed scalable GP model can outperform the state-of-the-art approaches considerably, in terms of wireless traffic prediction performance. [ABSTRACT FROM AUTHOR]

Electromagnetic devices containing permanent magnet (PM) are common and typical electromagnetic devices that have advantages of low-power consumption, small size, high sensitivity, and its optimization problem has aroused widespread concern. Calculation accuracy of electromagnetic properties and efficiency of optimization processes are two important factors that affect the optimization effect of electromagnetic devices. A fast calculation method was proposed in this paper that aimed at electromagnetic devices with PM characteristics of strong non-linearity and low solving precision. The response surface method was used to derive the basis function, and the error between actual measured values and calculated values was modified by the Kriging method. On the basis of the calculation model, the distributed collaborative strategy was adopted to fulfill the parallel execution of the fast computation. Simultaneously, the updating strategy of particles was improved, and the robustness of the multi-objective particle swarm optimization algorithm in solving the optimization problem of electromagnetic devices with PM was enhanced. Effectiveness of the method proposed in this paper was verified by the case study of a specific type of electromagnetic relay with PM. [ABSTRACT FROM AUTHOR]

Approximation jobs that allow partial execution of their many tasks to achieve valuable results have played an important role in today’s large-scale data analytics. This fact can be utilized to maximize the system utility of a big data computing cluster by choosing proper tasks in scheduling for each approximation job. A fundamental challenge herein, however, is that the machine service capacity may fluctuate substantially during a job’s lifetime, which makes it difficult to assign valuable tasks to well-performing machines. In addition, the cluster scheduler needs to make online scheduling decisions without knowing future job arrivals according to machine availabilities. In this paper, we tackle this online resource allocation problem for approximation jobs in parallel computing clusters. In particular, we model a cluster with heterogeneous machines as a multi-armed bandit where each machine is treated as an arm. By making estimations on machine service rates while balancing the exploration-exploitation trade-off, we design an efficient online resource allocation algorithm from a bandit perspective. The proposed algorithm extends existing online convex optimization techniques and yields a sublinear regret bound. Moreover, we also examine the performance of the proposed algorithm via extensive trace-driven simulations and demonstrate that it outperforms the baselines substantially. [ABSTRACT FROM AUTHOR]

In most real-world settings, designs are often gradually adapted and improved over time. Consequently, there exists knowledge from distinct (but possibly related) design exercises, which have either been previously completed or are currently in-progress, that may be leveraged to enhance the optimization performance of a particular target optimization task of interest. Further, it is observed that modern day design cycles are typically distributed in nature, and consist of multiple teams working on associated ideas in tandem. In such environments, vast amounts of related information can become available at various stages of the search process corresponding to some ongoing target optimization exercise. Successfully exploiting this knowledge is expected to be of significant value in many practical settings, where solving an optimization problem from scratch may be exorbitantly costly or time consuming. Accordingly, in this paper, we propose an adaptive knowledge reuse framework for surrogate-assisted multiobjective optimization of computationally expensive problems, based on the novel idea of multiproblem surrogates. This idea provides the capability to acquire and spontaneously transfer learned models across problems, facilitating efficient global optimization. The efficacy of our proposition is demonstrated on a series of synthetic benchmark functions, as well as two practical case studies. [ABSTRACT FROM AUTHOR]

In quantitative magnetic resonance ${T}_{\textsf {1}}$ mapping, the variable flip angle (VFA) steady state spoiled gradient recalled echo (SPGR) imaging technique is popular as it provides a series of high resolution ${T}_{\textsf {1}}$ weighted images in a clinically feasible time. Fast, linear methods that estimate ${T}_{\textsf {1}}$ maps from these weighted images have been proposed, such as DESPOT1 and iterative re-weighted linear least squares. More accurate, non-linear least squares (NLLS) estimators are in play, but these are generally much slower and require careful initialization. In this paper, we present NOVIFAST, a novel NLLS-based algorithm specifically tailored to VFA SPGR ${T}_{\textsf {1}}$ mapping. By exploiting the particular structure of the SPGR model, a computationally efficient, yet accurate and precise ${T}_{\textsf {1}}$ map estimator is derived. Simulation and in vivo human brain experiments demonstrate a twenty-fold speed gain of NOVIFAST compared with conventional gradient-based NLLS estimators while maintaining a high precision and accuracy. Moreover, NOVIFAST is eight times faster than the efficient implementations of the variable projection (VARPRO) method. Furthermore, NOVIFAST is shown to be robust against initialization. [ABSTRACT FROM AUTHOR]

A new method for robust estimation, MAGSAC++, is proposed. It introduces a new model quality (scoring) function that does not make inlier-outlier decisions, and a novel marginalization procedure formulated as an M-estimation with a novel class of M-estimators (a robust kernel) solved by an iteratively re-weighted least squares procedure. Instead of the inlier-outlier threshold, it requires only its loose upper bound which can be chosen from a significantly wider range. Also, we propose a new termination criterion and a technique for selecting a set of inliers in a data-driven manner as a post-processing step after the robust estimation finishes. On a number of publicly available real-world datasets for homography, fundamental matrix fitting and relative pose, MAGSAC++ produces results superior to the state-of-the-art robust methods. It is more geometrically accurate, fails fewer times, and it is often faster. It is shown that MAGSAC++ is significantly less sensitive to the setting of the threshold upper bound than the other state-of-the-art algorithms to the inlier-outlier threshold. Therefore, it is easier to be applied to unseen problems and scenes without acquiring information by hand about the setting of the inlier-outlier threshold. The source code and examples both in C++ and Python are available at https://github.com/danini/magsac. [ABSTRACT FROM AUTHOR]

Despite the success of stochastic variance-reduced gradient (SVRG) algorithms in solving large-scale problems, their stochastic gradient complexity often scales linearly with data size and is expensive for huge data. Accordingly, we propose a hybrid stochastic-deterministic minibatch proximal gradient (HSDMPG) algorithm for strongly convex problems with linear prediction structure, e.g., least squares and logistic/softmax regression. HSDMPG enjoys improved computational complexity that is data-size-independent for large-scale problems. It iteratively samples an evolving minibatch of individual losses to estimate the original problem, and can efficiently minimize the sampled subproblems. For strongly convex loss of $n$ n components, HSDMPG attains an $\epsilon$ ε -optimization-error within $\mathcal {O} \left(\kappa \log ^{\zeta +1}\left(\frac{1}{\epsilon }\right)\frac{1}{\epsilon }\bigwedge n\log ^{\zeta }\left(\frac{1}{\epsilon }\right)\right)$ O κ log ζ + 1 1 ε 1 ε ⋀ n log ζ 1 ε stochastic gradient evaluations, where $\kappa$ κ is condition number, $\zeta =1$ ζ = 1 for quadratic loss and $\zeta =2$ ζ = 2 for generic loss. For large-scale problems, our complexity outperforms those of SVRG-type algorithms with/without dependence on data size. Particularly, when $\epsilon =\mathcal {O}(1/\sqrt{n})$ ε = O (1 / n) which matches the intrinsic excess error of a learning model and is sufficient for generalization, our complexity for quadratic and generic losses is respectively $\mathcal {O} (n^{0.5}\log ^{2}(n))$ O (n 0. 5 log 2 (n)) and $\mathcal {O} (n^{0.5}\log ^{3}(n))$ O (n 0. 5 log 3 (n)) , which for the first time achieves optimal generalization in less than a single pass over data. Besides, we extend HSDMPG to online strongly convex problems and prove its higher efficiency over the prior algorithms. Numerical results demonstrate the computational advantages of HSDMPG. [ABSTRACT FROM AUTHOR]

This paper proposes an efficient reliability-based optimization algorithm based on adaptive dynamic Taylor Kriging (ADTK). In the ADTK model, the basis functions are optimally selected by the binary particle swarm optimization algorithm and the minimal number of sampling data with a desired fitting error is decided. To evaluate performance of reliability calculation, the Monte Carlo simulation method is employed, where the performance constraint functions are constructed by the ADTK surrogate model. Overall, an electromagnetic application-superconducting magnetic energy storage device (TEAM problem 22) is used to investigate performances of the proposed method. [ABSTRACT FROM AUTHOR]

Knowledge-based neural network modeling techniques using space-mapping concept have been demonstrated in the existing literature as efficient methods to overcome the accuracy limitations of empirical/equivalent circuit models when matching new electromagnetic data. For different modeling problems, the mapping structures can be different. In this paper, we propose a unified automated model generation algorithm that uses \boldsymbol l\mathbf {1} optimization to automatically determine the type and the topology of the mapping structure in a knowledge-based neural network model. By encompassing various types of mappings of the knowledge-based neural network model in the existing literature, we present a new unified model structure and derive new sensitivity formulas for the training of the unified model. The proposed \boldsymbol l\mathbf {1} formulation of modeling can force some weights of the mapping neural networks to zeros while leaving other weights as nonzeros. We utilize this feature to allow \boldsymbol l\mathbf {1} optimization to automatically determine which mapping is necessary and which mapping is unnecessary. Using the proposed \boldsymbol l\mathbf {1} optimization method, the mapping structure can be determined to address different needs of different modeling problems. The structure of the final knowledge-based model can be flexible combinations of some or all of linear mapping, nonlinear mapping, input mapping, frequency mapping, and output mapping. In this way, the proposed algorithm is more systematic and can further speed up the knowledge-based modeling process than existing knowledge-based modeling algorithms. The proposed method is illustrated by three microwave filter modeling examples. [ABSTRACT FROM PUBLISHER]

Residual demand curves (RDCs) can be used to represent the strategic interaction of participants in electricity markets. RDCs relate the energy that an agent can buy or sell in one hour with the clearing market price that would be obtained in such hour, assuming the market is organized as simple bid independent auctions. Despite the fact that they have been widely used in the literature, the existence of time and/or spatial constraints in the market clearing algorithm makes the RDCs not directly applicable. This paper tries to overcome these difficulties by extending the concept of RDCs to zonal pricing markets where complex offering conditions and transmission constraints are taken into account. Therefore, the RDCs are redefined in order to capture such effects, which are usually neglected or oversimplified. A new method for computing the redefined RDCs is established and its application to the Iberian electricity market is presented. The results show that modeling complex conditions and transmission constraints in RDCs can have a significant effect when compared to the standard approach found in the literature. Therefore, the method presented in this paper modeling the effect of firm's decisions on market prices in a more accurate way. [ABSTRACT FROM AUTHOR]

This paper considers multirobot task allocation problems where the estimated costs for performing tasks are interrelated, and the overall team objective need not be a standard sum-of-costs (or utilities) model, enabling straightforward treatment of the additional costs incurred by resource contention. In the model we introduce, a team may choose one of a set of shared resources to perform a task (e.g., several routes to reach a destination), and interference is modeled when multiple robots use the same resource. We show that the general problem is NP-hard, and investigate specialized subinstances with particular cost structures. For the general problem, we describe an exact algorithm which finds an optimal assignment in a reasonable time on small instances. Aiming at larger problems, we turn two particular subinstances, introducing an two algorithms that find assignments quickly even for problems of considerable size, the first being optimal, the second being an approximation algorithm but also producing high-quality solutions with bounded suboptimality. [ABSTRACT FROM PUBLISHER]

Classification with a sparsity constraint on the solution plays a central role in many high dimensional signal processing applications. In some cases, the features can be grouped together, so that entire subsets of features can be selected or discarded. In many applications, however, this can be too restrictive. In this paper, we are interested in a less restrictive form of structured sparse feature selection: We assume that while features can be grouped according to some notion of similarity, not all features in a group need be selected for the task at hand. The Sparse Group Lasso (SGL) was proposed to solve problems of this form. The main contributions of this paper are a new procedure called Sparse Overlapping Group (SOG) lasso, an extension to the SGL to overlapping groups and theoretical sample complexity bounds for the same. We establish model selection error bounds that specializes to many other cases. We experimentally validate our proposed method on both real and toy datasets. [ABSTRACT FROM AUTHOR]

Neural networks with rectified linear unit (ReLU) activation functions (a.k.a. ReLU networks) have achieved great empirical success in various domains. Nonetheless, existing results for learning ReLU networks either pose assumptions on the underlying data distribution being, e.g., Gaussian, or require the network size and/or training size to be sufficiently large. In this context, the problem of learning a two-layer ReLU network is approached in a binary classification setting, where the data are linearly separable and a hinge loss criterion is adopted. Leveraging the power of random noise perturbation, this paper presents a novel stochastic gradient descent (SGD) algorithm, which can provably train any single-hidden-layer ReLU network to attain global optimality, despite the presence of infinitely many bad local minima, maxima, and saddle points in general. This result is the first of its kind, requiring no assumptions on the data distribution, training/network size, or initialization. Convergence of the resultant iterative algorithm to a global minimum is analyzed by establishing both an upper bound and a lower bound on the number of non-zero updates to be performed. Moreover, generalization guarantees are developed for ReLU networks trained with the novel SGD leveraging classic compression bounds. These guarantees highlight a key difference (at least in the worst case) between reliably learning a ReLU network as well as a leaky ReLU network in terms of sample complexity. Numerical tests using both synthetic data and real images validate the effectiveness of the algorithm and the practical merits of the theory. [ABSTRACT FROM AUTHOR]

Surrogate-assisted evolutionary algorithms (EAs) have been intensively used to solve computationally expensive problems with some success. However, traditional EAs are not suitable to deal with high-dimensional expensive problems (HEPs) with high-dimensional search space even if their fitness evaluations are assisted by surrogate models. The recently proposed autoencoder-embedded evolutionary optimization (AEO) framework is highly appropriate to deal with high-dimensional problems. This work aims to incorporate surrogate models into it to further boost its performance, thus resulting in surrogate-assisted AEO (SAEO). It proposes a novel model management strategy that can guarantee reasonable amounts of re-evaluations; hence, the accuracy of surrogate models can be enhanced via being updated with new evaluated samples. Moreover, to ensure enough data samples before constructing surrogates, a problem-dimensionality-dependent activation condition is developed for incorporating surrogates into the SAEO framework. SAEO is tested on seven commonly used benchmark functions and compared with state-of-the-art algorithms for HEPs. The experimental results show that SAEO can further enhance the performance of AEO on most cases and SAEO performs significantly better than other algorithms. Therefore, SAEO has great potential to deal with HEPs. [ABSTRACT FROM AUTHOR]

Fuzzy clustering is one of the most popular clustering approaches and has attracted considerable attention in many fields. However, high computational cost has become a bottleneck which limits its applications in large-scale problems. Moreover, most fuzzy clustering algorithms are sensitive to noise. To address these issues, a novel fuzzy clustering algorithm, called fast fuzzy clustering based on anchor graph (FFCAG), is proposed. The FFCAG algorithm integrates anchor-based similarity graph construction and membership matrix learning into a unified framework, such that the prior knowledge of anchors can be further utilized to improve clustering performance. Specifically, FFCAG first constructs an anchor-based similarity graph with a parameter-free neighbor assignment strategy. Then, it designs a quadratic programming model to learn the membership matrix of anchors, which is very different from traditional fuzzy clustering algorithms. More importantly, a novel balanced regularization term is introduced into the objective function to produce more accurate clustering results. Finally, we adopt an alternating optimization algorithm with guaranteed convergence to solve the proposed method. Experimental results performed on synthetic and real-world datasets demonstrate the proposed FFCAG can significantly reduce the computational time with comparable, even superior, clustering performance, compared with state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

As a promising solution to boost the performance of distance-related algorithms (e.g., $K$ -means and KNN), FPGA-based acceleration attracts lots of attention, but also comes with numerous challenges. In this work, we propose, STPAcc, an optimization framework based on structural triangle-inequality (TI)-based pruning (STP) for accelerating distance-related algorithms on CPU-FPGA platforms. STPAcc provides a domain-specific language to unify distance-related algorithms effectively, a structural TI-based pruning strategy to remove unnecessary distance computations, a coarse-grained workload partitioning and mapping strategy to fully exploit the potentials of the CPU-FPGA platform, and fine-grained hardware optimizations to further improve performance on the FPGA. Intensive experiments show that STPAcc designs achieve $31.42\times $ speedup and $99.63\times $ better energy efficiency on average over standard CPU-based implementations. [ABSTRACT FROM AUTHOR]

The classic approach for estimating parameters of a model using historical data is to solve a nonlinear least squares optimization problem using numerical methods. We develop an I-frame methodology to solve the nonlinear least squares problem quickly which can be applied to both offline and online (where data is streamed in real time) parameter estimation. Using the concept of I-frames from imaging and animation, we approximate a solution to the nonlinear least squares problem via a two-step process, an I-frame optimization, and an incremental optimization. The I-frame optimization solves for the parameters using a subset of data points and the incremental optimization adjusts the parameters in between the I-frames. We show that the criterion of generating I-frames can affect the average squared error of the final solution. Our methodology benefits from being scalable as the number of parameters and amount of data increases with an appropriate I-frame generation criterion. [ABSTRACT FROM AUTHOR]

The optimization of electrical machines and energy-harvesting devices has been one of the hot topics of research in recent years. In this context, advanced computational intelligence algorithms are suitable design tools, since they are able to optimize complex systems with a large number of constraints and free design parameters. This paper presents the social network optimization (SNO): a new population-based algorithm developed to guarantee an effective and faster exploration of the solution domain with respect to traditional optimization techniques. The proposed technique will be described and validated with a benchmark case study. Finally, the optimization problem of a tubular permanent magnet linear generator will be formulated in the context of vehicular energy-harvesting systems. [ABSTRACT FROM PUBLISHER]

This paper proposes a new formulation for the corrective security-constrained optimal power flow (SCOPF) problem with DC power flow constraints. The goal is to produce a generation schedule which has a minimal number of post-contingency corrections as well as a minimal amount of total MW rescheduled. In other words, the new SCOPF model effectively clears contingencies with corrective actions that have a minimal impact on system operations. The proposed SCOPF model utilizes sparse optimization techniques to achieve computational tractability for large-scale power systems. We also propose two efficient decomposition algorithms. Extensive computational experiments show the advantage of the proposed model and algorithms on several standard IEEE test systems and large-scale real-world power systems. [ABSTRACT FROM PUBLISHER]

In hyperspectral unmixing applications, one typically assumes that a single spectrum exists for every endmember. In many scenarios, this is not the case, and one requires a set or a distribution of spectra to represent an endmember or class. This inherent spectral variability can pose severe difficulties in classical unmixing approaches. In this paper, we present a new algorithm for dealing with endmember variability in spectral unmixing, based on the geometrical interpretation of the resulting unmixing problem, and an alternating optimization approach. This alternating-angle-minimization algorithm uses sets of spectra to represent the variability present in each class and attempts to identify the subset of endmembers which produce the smallest reconstruction error. The algorithm is analogous to the popular multiple endmember spectral mixture analysis technique but has a much more favorable computational complexity while producing similar results. We illustrate the algorithm on several artificial and real data sets and compare with several other recent techniques for dealing with endmember variability. [ABSTRACT FROM PUBLISHER]

Increasing penetration of variable loads and renewable generation will cause the conventional operation strategy of the distribution system to become less certain and effective. This paper presents a method of determining the minimum loss network configuration of a distribution system with uncertain load and renewable generation. A robust optimization model is proposed wherein the optimal decisions on network configurations and losses are derived in two subsequent steps. The first step enforces the radiality constraint before knowing the actual system loads and output level of renewable generation. Power flows are computed next to achieve minimum network losses considering the worst operating conditions over the uncertainty sets. A mixed-integer two-stage robust optimization formulation and a decomposition algorithm in a master-slave structure are proposed to solve the problem. Results of an illustrative example and two test systems are presented to demonstrate the proposed method. [ABSTRACT FROM PUBLISHER]

IC3 is one of the most successful algorithms for hardware model checking. Since its invention in 2010, several variants of the original algorithm have been published, proposing optimizations and/or alternative procedures for many different steps of the algorithm. In this paper, we present a thorough empirical comparison of a large set of optimizations and procedures for the steps of IC3, considering “high-level” variants/extensions to the basic algorithm, as well as “low-level” optimizations/configuration settings. We implemented each of them in the same tool, optimizing the implementations to the best of our knowledge. This enabled for a flexible experimentation in a controlled environment, and to gain new insights about their most important differences and commonalities, as well as about their performance characteristics. We conducted the experiments using as benchmarks the problems used in the last four editions of the hardware model checking competition. The analysis helped us to identify several settings leading to significant improvements with respect to a basic implementation of IC3. [ABSTRACT FROM AUTHOR]

The exploding complexity and computation efficiency requirements of applications are stimulating a strong demand for hardware acceleration with heterogeneous platforms such as FPGAs. However, a high-quality FPGA design is very hard to create and optimize as it requires FPGA expertise and a long design iteration time. In contrast, software applications are typically developed in a short development cycle, with high-level languages like Python, which have much higher levels of abstraction than all existing hardware design flows. To close this gap between hardware design flows and software applications, and simplify FPGA programming, we create PyLog, a high-level, algorithm-centric Python-based programming and synthesis flow for FPGA. PyLog is powered by a set of compiler optimization passes and a type inference system to generate high-quality hardware design. It abstracts away the implementation details, and allows designers to focus on algorithm specification. PyLog takes in Python functions, generates PyLog intermediate representation (PyLog IR), performs several optimization passes, including pragma insertion, design space exploration, and memory customization, etc., and creates complete FPGA system designs. PyLog also has a runtime that allows users to run the PyLog code directly on the target FPGA platform without any extra code development. The whole design flow is automated. Evaluation shows that PyLog significantly improves FPGA design productivity and generates highly efficient FPGA designs that outperform highly optimized CPU implementation and state-of-the-art FPGA implementation by $3.17\times$ 3. 17 × and $1.24\times$ 1. 24 × on average. [ABSTRACT FROM AUTHOR]

Constrained optimization problems (COPs) are frequently encountered in real-world design applications. For some COPs, the evaluation of the objective(s) and/or constraint(s) may involve significant computational/temporal/financial cost. Such problems are referred to as expensive COPs (ECOPs). Surrogate modeling has been widely used in conjunction with optimization methods for such problems, wherein the search is partially driven by an approximate function instead of true expensive evaluations. However, for any true evaluation, nearly all existing methods compute all objective and constraint values together as one batch. Such full evaluation approaches may be inefficient for cases where the objective/constraint(s) can be evaluated independently of each other. In this article, we propose and study a constraint handling strategy for ECOPs using partial evaluations. The constraints are evaluated in a sequence determined based on their likelihood of being violated; and the evaluation is aborted if a constraint violation is encountered. Modified ranking strategies are introduced to effectively rank the solutions using the limited information thus obtained, while saving on significant function evaluations. The proposed algorithm is compared with a number of its variants to establish the utility of its key components systematically. Numerical experiments and benchmarking are conducted on a range of mathematical and engineering design problems to demonstrate the efficacy of the approach compared to state-of-the-art evolutionary optimization approaches. [ABSTRACT FROM AUTHOR]

We focus on over-the-air (OTA) Federated Learning (FL), which has been suggested recently to reduce the communication overhead of FL due to the repeated transmissions of the model updates by a large number of users over the wireless channel. In OTA FL, all users simultaneously transmit their updates as analog signals over a multiple access channel, and the server receives a superposition of the analog transmitted signals. However, this approach results in the channel noise directly affecting the optimization procedure, which may degrade the accuracy of the trained model. We develop a Convergent OTA FL (COTAF) algorithm which enhances the common local stochastic gradient descent (SGD) FL algorithm, introducing precoding at the users and scaling at the server, which gradually mitigates the effect of noise. We analyze the convergence of COTAF to the loss minimizing model and quantify the effect of a statistically heterogeneous setup, i.e. when the training data of each user obeys a different distribution. Our analysis reveals the ability of COTAF to achieve a convergence rate similar to that achievable over error-free channels. Our simulations demonstrate the improved convergence of COTAF over vanilla OTA local SGD for training using non-synthetic datasets. Furthermore, we numerically show that the precoding induced by COTAF notably improves the convergence rate and the accuracy of models trained via OTA FL. [ABSTRACT FROM AUTHOR]

The cross-entropy (CE) method can accelerate power system reliability evaluation effectively. An importance sampling (IS) based iterative parameter updating algorithm is usually used in CE optimization. However, the efficiency in capturing the concerned samples for parameter updating may be unsatisfactory, especially for a system with intrinsic nature of rare failure events. Besides, the stopping criterion of this iterative algorithm is usually determined subjectively, incurring insufficient or excessive parameter updating and a resultant higher computational burden of the entire simulation. To address the above two problems, a CE method based on subset simulation and minimum computational burden criterion is proposed. In CE optimization, the subset simulation combined with M-H sampling is adopted as an alternative to IS to effectively improve the efficiency of capturing desired samples for parameter updating at each iteration. Moreover, a novel quantitative stopping criterion is presented in such a way that the computational burden for the entire simulation is estimated and compared after each parameter updating iteration, and the optimal parameters corresponding to the minimum computational burden can be determined. The computational performance of the proposed method is validated by comparing the proposed approach with existing methods under several numerical tests including a realistic power system. [ABSTRACT FROM AUTHOR]

Kriging models, also known as Gaussian process models, are widely used in surrogate-assisted evolutionary algorithms (SAEAs). However, the cubic time complexity of the standard Kriging models limits their usage in high-dimensional optimization. To tackle this problem, we propose an incremental Kriging model for high-dimensional surrogate-assisted evolutionary computation. The main idea is to update the Kriging model incrementally based on the equations of the previously trained model instead of building the model from scratch when new samples arrive, so that the time complexity of updating the Kriging models can be reduced to quadratic. The proposed incremental learning scheme is very suitable for online SAEAs since they evaluate new samples in each one or several generations. The proposed algorithm is able to achieve competitive optimization results on the test problems compared with the standard Kriging-assisted evolutionary algorithm and is significantly faster than the standard Kriging approach. The proposed algorithm also shows competitive or better performances compared with four fast Kriging-assisted evolutionary algorithms and four state-of-the-art SAEAs. This work provides a fast way of employing Kriging models in high-dimensional surrogate-assisted evolutionary computation. [ABSTRACT FROM AUTHOR]

Security-constrained unit commitment (SCUC), as one of key components in power system operation, is being widely applied in vertically integrated utilities and restructured power systems. The efficient solution framework is to implement iterations between a master problem (unit commitment) and subproblems (network security evaluations). In industrial applications, both Lagrangian relaxation and mixed-integer programming are commonly applied for the unit commitment problem, and both linear sensitivity factor and Benders cut methods are used to generate additional constraints in the phase of network security evaluations. This paper evaluates capabilities and performances of each algorithm through technical discussion and numerical testing. Special topics on the large-scale SCUC engine development are also discussed in this paper, such as input data screening, inactive constrains elimination, contingency management, infeasibility handling, parallel computing, and model simplification. This paper will benefit academic researchers, software developers, and system operators when they design, develop and assess effective models and algorithms for solving large-scale SCUC problems. [ABSTRACT FROM PUBLISHER]

This paper models 1-out-of-N standby computing systems with a dynamic checkpointing policy. The system performs a real-time mission task that has to be accomplished within an allowed mission time. During the mission, to facilitate an effective failure recovery the system undergoes checkpointing procedures according to a policy that dynamically determines a checkpointing frequency based on the activated element and remaining work for completing the mission. System elements are heterogeneous; they can follow different, arbitrary types of time-to-failure distributions, have different performance and wait in different standby modes before their activation. A new numerical algorithm based on state space event transitions is first proposed to evaluate mission success probability of the real-time standby systems considered in this work. Additional new contributions are made by formulating and solving optimal dynamic checkpointing policy problems, as well as an integrated optimization problem that finds the optimal combination of checkpointing policy and element activation sequence maximizing mission success probability. Advantages of using the dynamic checkpointing policy over fixed even checkpoints are demonstrated through examples. Examples and results are also provided to illustrate effects of different mission and element parameters on mission success probability as well as on the optimal dynamic checkpointing policy. [ABSTRACT FROM AUTHOR]

This article proposes an evolutionary algorithm using multiple penalties and multiple local surrogates (MPMLS) for expensive constrained optimization. In each generation, MPMLS defines and optimizes a number of subproblems. Each subproblem penalizes the constraints in the original problem using a different penalty coefficient and has its own search subregion. A local surrogate is built for optimizing each subproblem. Two major advantages of MPMLS are: 1) it can maintain good population diversity so that the search can approach the optimal solution of the original problem from different directions and 2) it only needs to build local surrogates so that the computational overhead of the model building can be reduced. Numerical experiments demonstrate that our proposed algorithm performs much better than some other state-of-the-art evolutionary algorithms. [ABSTRACT FROM AUTHOR]

Once defined, the access control policies and regulations would never be changed in a running and state transition process. However, it will give attackers the possibility of discovering vulnerabilities in the system, and the control systems lack the ability of dynamic perception of security state and risk, causing the systems to be exposed to risks. In this article, a dynamic Bell–LaPadula (BLP) model is proposed. The conditional random field (CRF) is introduced into the BLP model to optimize the rules. First, the model formalizes the security attributes, states of system, transition rules, and constraint models on the basis of the state transition of CRFs. After the historical system access logs are processed as the original dataset, a feature selection method is proposed to extract the requests and current states as feature vectors. Second, this article presents a rules training algorithm based on L-BFGS to implement the study and training of datasets, and then marks the logs in the test set through Viterbi algorithm automatically. On the base of these, a rule generation algorithm is proposed to dynamically adjust the access control rules based on the current security status and events of the system. Third, the security of CRFs-BLP is proved by theoretical analysis. Finally, the validity and accuracy of the model are verified by estimating the value of the precision, recall, and $F1$ -score. As the system threats are shown to be decreased obviously from these experiments, this dynamic model can decrease the vulnerabilities and risk effectively. [ABSTRACT FROM AUTHOR]