Search

Multi-Task Learning (MTL) is a learning paradigm in machine learning and its aim is to leverage useful information contained in multiple related tasks to help improve the generalization performance of all the tasks. In this paper, we give a survey for MTL from the perspective of algorithmic modeling, applications and theoretical analyses. For algorithmic modeling, we give a definition of MTL and then classify different MTL algorithms into five categories, including feature learning approach, low-rank approach, task clustering approach, task relation learning approach and decomposition approach as well as discussing the characteristics of each approach. In order to improve the performance of learning tasks further, MTL can be combined with other learning paradigms including semi-supervised learning, active learning, unsupervised learning, reinforcement learning, multi-view learning and graphical models. When the number of tasks is large or the data dimensionality is high, we review online, parallel and distributed MTL models as well as dimensionality reduction and feature hashing to reveal their computational and storage advantages. Many real-world applications use MTL to boost their performance and we review representative works in this paper. Finally, we present theoretical analyses and discuss several future directions for MTL. [ABSTRACT FROM AUTHOR]

Traffic control optimization is a challenging task for various traffic centers around the world and the majority of existing approaches focus only on developing adaptive methods for normal (recurrent) traffic conditions. Optimizing the control plans when severe incidents occur still remains an open problem, especially when a high number of lanes or entire intersections are affected. This paper presents a novel methodology for optimizing the traffic signal timings in signalized urban intersections, under non-recurrent traffic incidents. With the purpose of producing fast and reliable decisions, we combine the fast running Machine Learning (ML) algorithms and the reliable Genetic Algorithms (GA) into a single optimization framework. Firstly, we deploy a typical GA algorithm by considering the phase duration as the decision variable and the objective function as the total travel time in the network. We fine tune the GA for crossover, mutation, fitness calculation and obtain the optimal parameters. Secondly, we train several regression models to predict the total travel time in the studied traffic network, and select the best performing model which we further hyper-tune. Lastly, we propose a new algorithm BGA-ML combining the GA algorithm and the extreme-gradient decision-tree (XGBT), which is the best performing regression model, together in a single optimization framework. Comparison and results are generated by two experiments (one synthetic and one from real urban traffic network) and show that the new BGA-ML is much faster than the original GA algorithm and can reduce the total travel time by almost half when used under incident conditions. [ABSTRACT FROM AUTHOR]

In this paper, we study an automatic hypothesis generation (HG) problem, which refers to the discovery of meaningful implicit connections between scientific terms, including but not limited to diseases, chemicals, drugs, and genes extracted from databases of biomedical publications. Most prior studies of this problem focused on the use of static information of terms and largely ignored the temporal dynamics of scientific term relations. Even when the dynamics were considered in a few recent studies, they learned the representations for the scientific terms, rather than focusing on the term-pair relations. Since the HG problem is to predict term-pair connections, it is not enough to know with whom the terms are connected, it is more important to know how the connections have been formed (in a dynamic process). We formulate this HG problem as a future connectivity prediction in a dynamic attributed graph. The key is to capture the temporal evolution of node-pair (term-pair) relations. We propose an inductive edge (node-pair) embedding method named T-PAIR, utilizing both the graphical structure and node attribute to encode the temporal node-pair relationship. We demonstrate the efficiency of the proposed model on three real-world datasets, which are three graphs constructed from Pubmed papers published until 2019 in Neurology, Immunotherapy, and Virology, respectively. Evaluations were conducted on predicting future term-pair relations between millions of seen terms (in the transductive setting), as well as on the relations involving unseen terms (in the inductive setting). Experiment results and case study analyses show the effectiveness of the proposed model. [ABSTRACT FROM AUTHOR]

Academic research in the field of autonomous vehicles has reached high popularity in recent years related to several topics as sensor technologies, V2X communications, safety, security, decision making, control, and even legal and standardization rules. Besides classic control design approaches, Artificial Intelligence and Machine Learning methods are present in almost all of these fields. Another part of research focuses on different layers of Motion Planning, such as strategic decisions, trajectory planning, and control. A wide range of techniques in Machine Learning itself have been developed, and this article describes one of these fields, Deep Reinforcement Learning (DRL). The paper provides insight into the hierarchical motion planning problem and describes the basics of DRL. The main elements of designing such a system are the modeling of the environment, the modeling abstractions, the description of the state and the perception models, the appropriate rewarding, and the realization of the underlying neural network. The paper describes vehicle models, simulation possibilities and computational requirements. Strategic decisions on different layers and the observation models, e.g., continuous and discrete state representations, grid-based, and camera-based solutions are presented. The paper surveys the state-of-art solutions systematized by the different tasks and levels of autonomous driving, such as car-following, lane-keeping, trajectory following, merging, or driving in dense traffic. Finally, open questions and future challenges are discussed. [ABSTRACT FROM AUTHOR]

Time series classification models have been garnering significant importance in the research community. However, not much research has been done on generating adversarial samples for these models. These adversarial samples can become a security concern. In this paper, we propose utilizing an adversarial transformation network (ATN) on a distilled model to attack various time series classification models. The proposed attack on the classification model utilizes a distilled model as a surrogate that mimics the behavior of the attacked classical time series classification models. Our proposed methodology is applied onto 1-nearest neighbor dynamic time warping (1-NN DTW) and a fully convolutional network (FCN), all of which are trained on 42 University of California Riverside (UCR) datasets. In this paper, we show both models were susceptible to attacks on all 42 datasets. When compared to Fast Gradient Sign Method, the proposed attack generates a larger faction of successful adversarial black-box attacks. A simple defense mechanism is successfully devised to reduce the fraction of successful adversarial samples. Finally, we recommend future researchers that develop time series classification models to incorporating adversarial data samples into their training data sets to improve resilience on adversarial samples. [ABSTRACT FROM AUTHOR]

It is widely acknowledged that the success of deep learning is built upon large-scale training data and tremendous computing power. However, the data and computing power are not always available for many real-world applications. In this paper, we address the machine learning problem where it lacks training data and limits computing power. Specifically, we investigate domain adaptation which is able to transfer knowledge from one labeled source domain to an unlabeled target domain, so that we do not need much training data from the target domain. At the same time, we consider the situation that the running environment is confined, e.g., in edge computing the end device has very limited running resources. Technically, we present the Faster Domain Adaptation (FDA) protocol and further report two paradigms of FDA: early stopping and amid skipping. The former accelerates domain adaptation by multiple early exit points. The latter speeds up the adaptation by wisely skip several amid neural network blocks. Extensive experiments on standard benchmarks verify that our method is able to achieve the comparable and even better accuracy but employ much less computing resources. To the best of our knowledge, there are very few works which investigated accelerating knowledge adaptation in the community. This work is expected to inspire the topic for more discussion. [ABSTRACT FROM AUTHOR]

The problem of single-group multicast beamforming (SMBF) is well-known NP-hard. It motivates the pursuit of computationally efficient near-optimal solutions. Due to multicasting, the multicast group is bottlenecked by the user(s) with the minimum received signal-to-noise ratio (SNR). This paper provides an in-depth interpretation of the SMBF problem from the multicasting point of view and proposes to solve it in two steps: i) select the bottlenecking users by a machine learning approach based on determinantal point process (DPP), and ii) design the beamformer for the selected users. The DPP model jointly considers the magnitudes and directions of users’ channel vectors, and thus enables an efficient selection of the bottlenecking users. Moreover, for a specific channel model, the DPP model is only associated with network size and each takes a one-off training cost, thus can be used as a codebook. The proposed DPP-based subset selection is incorporated adaptively into two fast beamforming algorithms, i.e., the QR decomposition algorithm and the successive beamforming (SB) algorithm. They specifically design the beamformers for the selected users by leveraging channel orthogonalization therein. Numerical results demonstrate the superiority of the proposed QR-DPP and SB-DPP algorithms in terms of the performance-complexity compromise and their robustness to different scenarios. [ABSTRACT FROM AUTHOR]

This paper attacks the challenging problem of video retrieval by text. In such a retrieval paradigm, an end user searches for unlabeled videos by ad-hoc queries described exclusively in the form of a natural-language sentence, with no visual example provided. Given videos as sequences of frames and queries as sequences of words, an effective sequence-to-sequence cross-modal matching is crucial. To that end, the two modalities need to be first encoded into real-valued vectors and then projected into a common space. In this paper we achieve this by proposing a dual deep encoding network that encodes videos and queries into powerful dense representations of their own. Our novelty is two-fold. First, different from prior art that resorts to a specific single-level encoder, the proposed network performs multi-level encoding that represents the rich content of both modalities in a coarse-to-fine fashion. Second, different from a conventional common space learning algorithm which is either concept based or latent space based, we introduce hybrid space learning which combines the high performance of the latent space and the good interpretability of the concept space. Dual encoding is conceptually simple, practically effective and end-to-end trained with hybrid space learning. Extensive experiments on four challenging video datasets show the viability of the new method. Code and data are available at https://github.com/danieljf24/hybrid_space. [ABSTRACT FROM AUTHOR]

Coded distributed computing(CDC) has shown great potentials to solve the unexpected delay caused by stragglers in distributed computing. In this paper, we focus on the auction design for efficient resource allocation in CDC. Specifically, we aim to design a learning auction mechanism to handle heterogeneous user demands and also to free users from the complexity of specifying valuations for resource combinations, which increases exponentially with the resource dimensions. The user demand type is heterogeneous according to different variation trends of the value with finish time and workload, which is modeled by deep learning. The platform would allocate resources according to the user value function. Then users do not need to consider the complex relationship between uncertain finish time and resource configuration in CDC. Due to the inference error of the learning model and the complexity of calculating uncertain finish time, the considered social welfare optimization problem is a non-linear and non-convex integer problem. Even worse, the typical VCG-based payment scheme cannot guarantee truthfulness with the inference error. In response to these difficulties, we transform the social welfare optimization problem into a mixed integer programming problem which already has efficient solutions. The social welfare gap caused by the inference error is analyzed theoretically. The relationship between the utility regret of reporting truthfully and the inference error is also analyzed. We prove that our mechanism satisfies incentive alignment and individual rationality. Extensive experiments show the superiority of our mechanism compared with existing ones. [ABSTRACT FROM AUTHOR]

The wide deployment of advanced robots with industrial IoT (IIoT) technologies in smart factories generates a large volume of data during production and a wide variety of data processing workloads are launched to maintain productivity and safety of smart manufacture. The emerging fog computing paradigm offers a promising solution to enhancing data processing performance in a smart factory environment while on the other hand brings in new challenges to resource management, which call for a more effective approach for recommending resource configurations to heterogeneous workloads. In this paper, we propose an Optimized Recommendations of Heterogeneous Resource Configurations (ORHRC) model that employs machine learning techniques to provide resource configuration recommendations for the heterogeneous workloads in a fog computing-based smart factory environment. ORHRC learns a recommendation model by leveraging the operating characteristics and execution time of workloads on fog servers with different configurations. We also design a decision model in ORHRC to further improve prediction accuracy and reduce operational overheads. Experiment results show that ORHRC outperforms the state of art configuration recommendation methods in terms of average prediction accuracy. Note to Practitioners—The various data processing workloads in a smart factory environment need to be processed by the computational resources with optimal configurations for meeting their performance requirements. In this paper, we employ machine learning technologies for enabling automatic recommendation of resource configurations to heterogeneous workloads. Specifically, we develop an Optimized Recommendations of Heterogeneous Resource Configurations (ORHRC) model that can identify the optimal resource configurations for various workloads. We also conducted extensive experiments that verify the effectiveness of the proposed ORHRC model. [ABSTRACT FROM AUTHOR]

We give an overview of recent developments in the modeling of radiowave propagation, based on machine learning (ML) algorithms. We identify the input and output specification and the architecture of the model as the main challenges associated with ML-driven propagation models. Relevant papers are discussed and categorized based on their approach to each of these challenges. Emphasis is given on presenting the prospects and open problems in this promising and rapidly evolving area. [ABSTRACT FROM AUTHOR]

Capturing long-range dependencies during feature extraction is crucial for video-based person re-identification (re-id) since it would help to tackle many challenging problems such as occlusion and dramatic pose variation. Moreover, capturing subtle differences, such as bags and glasses, is indispensable to distinguish similar pedestrians. In this paper, we propose a novel and efficacious Statistic Attention (SA) block which can capture both the long-range dependencies and subtle differences. SA block leverages high-order statistics of feature maps, which contain both long-range and high-order information. By modeling relations with these statistics, SA block can explicitly capture long-range dependencies with less time complexity. In addition, high-order statistics usually concentrate on details of feature maps and can perceive the subtle differences between pedestrians. In this way, SA block is capable of discriminating pedestrians with subtle differences. Furthermore, this lightweight block can be conveniently inserted into existing deep neural networks at any depth to form Statistic Attention Network (SANet). To evaluate its performance, we conduct extensive experiments on two challenging video re-id datasets, showing that our SANet outperforms the state-of-the-art methods. Furthermore, to show the generalizability of SANet, we evaluate it on three image re-id datasets and two more general image classification datasets, including ImageNet. The source code is available at http://vipl.ict.ac.cn/resources/codes/code/SANet_code.zip. [ABSTRACT FROM AUTHOR]

In this paper, we propose a circuit yield prediction method to improve the efficiency of circuit yield prediction. Both traditional Monte Carlo simulation and the current popular machine learning for yield prediction require a large number of circuit simulation results, which increases production costs. This method finds the data boundary points through high-dimensional kernel density estimation, and performs simulation calculations on these boundary values instead of the overall simulation. Experiments show that, compared with machine learning and Monte Carlo methods, the proposed algorithm can efficiently predict the yield of circuits. [ABSTRACT FROM AUTHOR]

To tackle the increasingly larger training data and models, researchers and engineers resort to multiple servers in a data center for distributed machine learning (DML). On one hand, DML enables us to leverage the computation power of multiple servers, which can effectively accelerate those computation-intensive tasks. On the other hand, DML also incurs significant communication cost due to parameter synchronization among these servers. In this paper, we want to explore the impact of synchronization topology, including both logical topology and physical topology, on the DML performance. First, we revisit the existing logical topologies, e.g., parameter server and ring allreduce, for parameter synchronization, and we find that these flat synchronization topologies is inefficient when running a large-scale DML training. Therefore, we propose a hierarchical parameter synchronization topology, called HiPS, which can achieve efficient parameter synchronization even on a large scale. Then, we compare two representative physical network topologies, namely, Fat-Tree and BCube. Based on our analyses, BCube has many advantages over Fat-Tree, e.g., higher bandwidth, better load balance, and lower hardware cost. The simulation results also show that BCube is more friendly to RDMA. Relying on the advantages of HiPS and BCube, the GST of “HiPS+BCube” is 12% ~ 70% lower than other combinations. Moreover, when the cluster size increases from 16 to 1024, the performance of “HiPS+BCube” only drops by 6.5%, while the performance of “Ring+BCube” drops by 44.6%. Hence, we believe “HiPS+BCube” is the optimal solution to benefit DML in large scale. [ABSTRACT FROM AUTHOR]

This paper analyzes regularization terms proposed recently for improving the adversarial robustness of deep neural networks (DNNs), from a theoretical point of view. Specifically, we study possible connections between several effective methods, including input-gradient regularization, Jacobian regularization, curvature regularization, and a cross-Lipschitz functional. We investigate them on DNNs with general rectified linear activations, which constitute one of the most prevalent families of models for image classification and a host of other machine learning applications. We shed light on essential ingredients of these regularizations and re-interpret their functionality. Through the lens of our study, more principled and efficient regularizations can possibly be invented in the near future. [ABSTRACT FROM AUTHOR]

Stochastic unit commitment is an efficient method for grid operation in the presence of significant uncertainties. An example is an operation during a predicted hurricane with uncertain line out-ages. However, the solution quality comes at the cost of substantial computational burden, which makes its adoption challenging. This paper evaluates some possible ways that machine learning can be used to reduce this computational burden. First, a set of feasibility studies is conducted. Results suggest that using machine learning as an assistant to the stochastic unit commitment solver is more advantageous than using it as a standalone solver. In particular, the machine learning model is trained to facilitate solving the problem by determining the unnecessary constraints that can be removed from the original problem without affecting the final accuracy. The variables that can be used as input features/predictors or outputs for the machine learning model are determined through feasibility studies. Then, an algorithm to train and utilize a machine learning model is proposed. The method is tested on a 500-bus synthetic South Carolina system. Various test cases show an average reduction in solution time by more than 90% by using the trained machine learning model to assist the stochastic unit commitment solver. [ABSTRACT FROM AUTHOR]

Dependency-aware jobs, such as the big data analytic workflows, are commonly executed on the cloud. They are compiled to directed acyclic graphs, with tasks linked in regarding the dependency. The cloud scheduler, which maintains a large number of resources, is responsible to execute tasks in parallel. To resolve the complex dependencies, Deep Reinforcement Learning (DRL) based schedulers are widely applied. However, we find that the DRL-based schedulers are vulnerable to the perturbations in the input jobs and may generate falsified decisions to benefit a particular job while delaying the others. By perturbation, we mean a slight adjustment to the job's node features or dependencies, while not changing its functionality. In this paper, we first explore the vulnerability of DRL-based schedulers to job perturbations without accessing the information of the DRL models used in the scheduler. We devise the black-box perturbation system, in which, a proxy model is trained to mimic the DRL-based scheduling policy. We show that the high-faith proxy model can help to craft effective perturbations. The DRL-based schedulers can be as high as 60% likely to be badly affected by the perturbations. Then, we investigate the solution to improve the robustness of DRL-based schedulers to such perturbations. We propose an adversarial training framework to force the neural model to adapt to the perturbation patterns during training so as to eliminate the potential damage during applications. Experiments show that the adversarial-trained scheduler is more robust, reducing the chance of being affected to 3-fold less and the potential bad effects halved. [ABSTRACT FROM AUTHOR]

Clinically oriented deep learning algorithms, combined with large-scale medical datasets, have significantly promoted computer-aided diagnosis. To address increasing ethical and privacy issues, Federated Learning (FL) adopts a distributed paradigm to collaboratively train models, rather than collecting samples from multiple institutions for centralized training. Despite intensive research on FL, two major challenges are still existing when applying FL in the real-world medical scenarios, including the performance degradation (i.e., retrogress) after each communication and the intractable class imbalance. Thus, in this paper, we propose a novel personalized FL framework to tackle these two problems. For the retrogress problem, we first devise a Progressive Fourier Aggregation (PFA) at the server side to gradually integrate parameters of client models in the frequency domain. Then, at the client side, we design a Deputy-Enhanced Transfer (DET) to smoothly transfer global knowledge to the personalized local model. For the class imbalance problem, we propose the Conjoint Prototype-Aligned (CPA) loss to facilitate the balanced optimization of the FL framework. Considering the inaccessibility of private local data to other participants in FL, the CPA loss calculates the global conjoint objective based on global imbalance, and then adjusts the client-side local training through the prototype-aligned refinement to eliminate the imbalance gap with such a balanced goal. Extensive experiments are performed on real-world dermoscopic and prostate MRI FL datasets. The experimental results demonstrate the advantages of our FL framework in real-world medical scenarios, by outperforming state-of-the-art FL methods with a large margin. The source code is available at https://github.com/CityU-AIM-Group/PRR-Imbalance https://github.com/CityU-AIM-Group/PRR-Imbalance. [ABSTRACT FROM AUTHOR]

In this paper, we study a new latency optimization problem for blockchain-based federated learning (BFL) in multi-server edge computing. In this system model, distributed mobile devices (MDs) communicate with a set of edge servers (ESs) to handle both machine learning (ML) model training and block mining simultaneously. To assist the ML model training for resource-constrained MDs, we develop an offloading strategy that enables MDs to transmit their data to one of the associated ESs. We then propose a new decentralized ML model aggregation solution at the edge layer based on a consensus mechanism to build a global ML model via peer-to-peer (P2P)-based blockchain communications. Blockchain builds trust among MDs and ESs to facilitate reliable ML model sharing and cooperative consensus formation, and enables rapid elimination of manipulated models caused by poisoning attacks. We formulate latency-aware BFL as an optimization aiming to minimize the system latency via joint consideration of the data offloading decisions, MDs’ transmit power, channel bandwidth allocation for MDs’ data offloading, MDs’ computational allocation, and hash power allocation. Given the mixed action space of discrete offloading and continuous allocation variables, we propose a novel deep reinforcement learning scheme with a parameterized advantage actor critic algorithm. We theoretically characterize the convergence properties of BFL in terms of the aggregation delay, mini-batch size, and number of P2P communication rounds. Our numerical evaluation demonstrates the superiority of our proposed scheme over baselines in terms of model training efficiency, convergence rate, system latency, and robustness against model poisoning attacks. [ABSTRACT FROM AUTHOR]

Click-through rate (CTR) prediction, whose goal is to predict the probability of the user to click on an item, has become increasingly significant in the recommender systems. Recently, some deep learning models with the ability to automatically extract the user interest from his/her behaviors have achieved great success. In these work, the attention mechanism is used to select the user interested items in historical behaviors, improving the performance of the CTR predictor. Normally, these attentive modules can be jointly trained with the base predictor by using gradient descents. In this paper, we regard user interest modeling as a feature selection problem, which we call user interest selection. For such a problem, we propose a novel approach under the framework of the wrapper method, which is named Meta-Wrapper. More specifically, we use a differentiable module as our wrapping operator and then recast its learning problem as a continuous bilevel optimization. Moreover, we use a meta-learning algorithm to solve the optimization and theoretically prove its convergence. Meanwhile, we also provide theoretical analysis to show that our proposed method 1) efficiencies the wrapper-based feature selection, and 2) achieves better resistance to overfitting. Finally, extensive experiments on three public datasets manifest the superiority of our method in boosting the performance of CTR prediction. [ABSTRACT FROM AUTHOR]

Enhancing the visibility in extreme low-light environments is a challenging task. Under nearly lightless condition, existing image denoising methods could easily break down due to significantly low SNR. In this paper, we systematically study the noise statistics in the imaging pipeline of CMOS photosensors, and formulate a comprehensive noise model that can accurately characterize the real noise structures. Our novel model considers the noise sources caused by digital camera electronics which are largely overlooked by existing methods yet have significant influence on raw measurement in the dark. It provides a way to decouple the intricate noise structure into different statistical distributions with physical interpretations. Moreover, our noise model can be used to synthesize realistic training data for learning-based low-light denoising algorithms. In this regard, although promising results have been shown recently with deep convolutional neural networks, the success heavily depends on abundant noisy-clean image pairs for training, which are tremendously difficult to obtain in practice. Generalizing their trained models to images from new devices is also problematic. Extensive experiments on multiple low-light denoising datasets – including a newly collected one in this work covering various devices – show that a deep neural network trained with our proposed noise formation model can reach surprisingly-high accuracy. The results are on par with or sometimes even outperform training with paired real data, opening a new door to real-world extreme low-light photography. [ABSTRACT FROM AUTHOR]

Hyperdimensional computing (HDC) has emerged as an alternative lightweight learning solution to deep neural networks. A key characteristic of HDC is the great extent of parallelism that can facilitate hardware acceleration. However, previous hardware implementations of HDC seldom focus on GPU designs, which were also inefficient partly due to the complexity of accelerating HDC on GPUs. In this paper, we present OpenHD, a flexible and high-performance GPU-powered framework for automating the mapping of general HDC applications including classification and clustering to GPUs. OpenHD takes advantage of memory optimization strategies specialized for HDC, minimizing the access time to different memory subsystems, and removing redundant operations. We also propose a novel training method to enable data parallelism in HDC training. Our evaluation result shows that the proposed training rapidly achieves the target accuracy, reducing the required training epochs by 4×. With OpenHD, users can deploy GPU-accelerated HDC applications without domain expert knowledge. Compared to the state-of-the-art GPU-powered HDC implementation, our evaluation on NVIDIA Jetson TX2 shows that OpenHD is up to 10.5× and 314× faster for HDC-based classification and clustering, respectively. Compared with non-HDC classification and clustering on GPUs, OpenHD-based HDC is 11.7× and 53× faster at comparable accuracy. OpenHD is available at: https://github.com/UCSD-SEELab/openhd. [ABSTRACT FROM AUTHOR]

Attention mechanisms have significantly boosted the performance of video classification neural networks thanks to the utilization of perspective contexts. However, the current research on video attention generally focuses on adopting a specific aspect of contexts (e.g., channel, spatial/temporal, or global context) to refine the features and neglects their underlying correlation when computing attentions. This leads to incomplete context utilization and hence bears the weakness of limited performance improvement. To tackle the problem, this paper proposes an efficient attention-in-attention (AIA) method for element-wise feature refinement, which investigates the feasibility of inserting the channel context into the spatio-temporal attention learning module, referred to as CinST, and also its reverse variant, referred to as STinC. Specifically, we instantiate the video feature contexts as dynamics aggregated along a specific axis with global average and max pooling operations. The workflow of an AIA module is that the first attention block uses one kind of context information to guide the gating weights calculation of the second attention that targets at the other context. Moreover, all the computational operations in attention units act on the pooled dimension, which results in quite few computational cost increase (< 0.02%). To verify our method, we densely integrate it into two classical video network backbones and conduct extensive experiments on several standard video classification benchmarks. The source code of our AIA is available at https://github.com/haoyanbin918/Attention-in-Attention. [ABSTRACT FROM AUTHOR]

Wide models such as generalized linear models and factorization-based models have been extensively used in various predictive applications, e.g., recommendation, CTR prediction, and image recognition. Due to the memory bounded property of the models, the performance improvement on CPU is reaching the limitation. GPU is known to have many computation units and high memory bandwidth, and becomes a promising platform for training machine learning models. However, the GPU training for the wide models is far from optimal due to the sparsity and irregularity in wide models. The existing GPU-based wide models are even slower than the ones using CPU. The classical training schema of the wide models does not optimized for the GPU architecture, which suffers from large amount of random memory accesses and redundant read/write of intermediate values. In this paper, we propose an efficient GPU-training framework for the large-scale wide models, named cuWide. To fully benefit from the memory hierarchy of GPU, cuWide applies a new flow-based schema for training, which leverages the spatial and temporal locality of wide models to drastically reduce the amount of communication with GPU global memory. To do so, we adopt a bigraph computation model to efficiently realize the flow-based schema and exploit three flexible interfaces for programming. Further, we use the 2D partition of mini-batch (in sample and feature dimensions) with proposed graph abstraction to optimize GPU memory access for sparse data, and apply several spatial-temporal caching mechanisms (importance-based model caching and cross-stage accumulation caching mechanisms) to achieve a high performance kernel. To efficiently implement cuWide, we also propose several GPU-oriented optimizations, including feature-oriented data layout to enhance the data locality, replication mechanism to reduce update conflicts in shared memory, and multi-stream scheduling to overlap data transferring and kernel computing. We show that cuWide can be up to more than 20× faster than the state-of-the-art GPU solutions and multi-core CPU solutions. [ABSTRACT FROM AUTHOR]

The Internet of Things (IoT), coupled with the edge computing paradigm, is enabling several pervasive networked applications with stringent real-time requirements, such as telemedicine and haptic telecommunications. Recent advances in network virtualization and artificial intelligence are helping solve network latency and capacity problems, learning from several states of the network stack. However, despite such advances, a network architecture able to meet the demands of next-generation networked applications with stringent real-time requirements still has untackled challenges. In this paper, we argue that only using network (or transport) layer information to predict traffic evolution and other network states may be insufficient, and a more holistic approach that considers predictions of application-layer states is needed to repair the inefficiencies of the TCP/IP architecture. Based on this intuition, we present the design and implementation of Reparo. At its core, the design of our solution is based on the detection of a packet loss and its restoration using a Hidden Markov Model (HMM) empowered with adversarial autoencoders. In our evaluation, we considered a telemedicine use case, specifically a telepathology session, in which a microscope is controlled remotely in real-time to assess histological imagery. Our results confirm that the use of adversarial autoencoders enhances the accuracy of the prediction method satisfying our telemedicine application’s requirements with a notable improvement in terms of throughput and latency perceived by the user. [ABSTRACT FROM AUTHOR]

Edge Computing has emerged as a new computing paradigm dedicated for mobile performance enhancement and energy efficiency purposes. Specifically, it benefits today’s interactive applications on power-constrained devices by offloading compute-intensive tasks to the edge nodes in close proximity. Meanwhile, FPGA is well known for its excellence in accelerating (domain-specific) compute-intensive tasks such as deep learning algorithms in a high performance and energy-efficient manner due to its hardware-customizable nature. In this paper, we make the first attempt to leverage and combine the advantages of these two, and proposed a new network-assisted computing model, namely FPGA-based edge computing. As a case study, we choose three computer vision (CV)-based mobile interactive applications, and implement their back-end computation engines on FPGA. By deploying such application-customized accelerator modules for computation offloading at the network edge, we experimentally demonstrate that this approach can effectively reduce response time for the applications and energy consumption for the entire system in comparison with traditional CPU-based edge/cloud offloading approach. [ABSTRACT FROM AUTHOR]

360-degree video streaming provides users with immersive experience by letting users determine their field-of-views (FoVs) in real time. To efficiently utilize the limited bandwidth resources, recent works have proposed a viewport adaptive 360-degree video streaming model by exploiting the bitrate adaptation in spatial and temporal domains. In this paper, under this video streaming model, we propose an online bitrate selection algorithm to enhance the user’s quality of experience (QoE). This is achieved by characterizing the user’s personalized FoV and real-time downloading capacity in an online fashion. We address the unknown user-specific FoV by introducing the reference FoV and design an online bitrate selection algorithm to learn the difference between the user’s actual FoV and the reference FoV. We prove that as the number of video segments increases, the performance of the proposed online algorithm approaches the optimal performance asymptotically, with a bounded error. We perform trace-driven simulations with real-world datasets. Simulation results show that under the scenario where the available video bitrates are relatively high, our proposed algorithm can improve the user’s viewing quality level between $4.2\!-\!29.4$ 4. 2 - 29. 4 percent and reduce the average intra-segment quality switch by at least 12.4 percent when compared with several existing methods. [ABSTRACT FROM AUTHOR]

Recently, graph-based methods have been widely applied to model fitting. However, in these methods, association information is invariably lost when data points and model hypotheses are mapped to the graph domain. In this paper, we propose a novel model fitting method based on co-clustering on bipartite graphs (CBG) to estimate multiple model instances in data contaminated with outliers and noise. Model fitting is reformulated as a bipartite graph partition behavior. Specifically, we use a bipartite graph reduction technique to eliminate some insignificant vertices (outliers and invalid model hypotheses), thereby improving the reliability of the constructed bipartite graph and reducing the computational complexity. We then use a co-clustering algorithm to learn a structured optimal bipartite graph with exact connected components for partitioning that can directly estimate the model instances (i.e., post-processing steps are not required). The proposed method fully utilizes the duality of data points and model hypotheses on bipartite graphs, leading to superior fitting performance. Exhaustive experiments show that the proposed CBG method performs favorably when compared with several state-of-the-art fitting methods. [ABSTRACT FROM AUTHOR]

False data injection attack (FDIA) is a major threat in wide-area monitoring systems. Being able to differentiate FDIA from normal grid contingencies is a paramount necessity for a grid operator to decide the correct response on a critical prompt basis as well as reduce the overall FDIA's false alarms. Two FDIA's characterization algorithms are developed in this paper. The first is based on the principal component analysis (PCA) while the second is based on the canonical correlation analysis (CCA). Both algorithms are developed in an online platform to reduce the computational complexity. The various designed test cases demonstrate a promising FDIA characterization performance utilizing both algorithms. The testing results of three machine learning-based classifiers indicate that the proposed FDIA's characterization algorithms provide better classification models than conventional PCA-based characterization algorithm with CCA illustrating advanced characterization and detection results. [ABSTRACT FROM AUTHOR]

Machine learning (ML) has attracted a great research interest for physical layer design problems, such as channel estimation, thanks to its low complexity and robustness. Channel estimation via ML requires model training on a dataset, which usually includes the received pilot signals as input and channel data as output. In previous works, model training is mostly done via centralized learning (CL), where the whole training dataset is collected from the users at the base station (BS). This approach introduces huge communication overhead for data collection. In this paper, to address this challenge, we propose a federated learning (FL) framework for channel estimation. We design a convolutional neural network (CNN) trained on the local datasets of the users without sending them to the BS. We develop FL-based channel estimation schemes for both conventional and RIS (intelligent reflecting surface) assisted massive MIMO (multiple-input multiple-output) systems, where a single CNN is trained for two different datasets for both scenarios. We evaluate the performance for noisy and quantized model transmission and show that the proposed approach provides approximately 16 times lower overhead than CL, while maintaining satisfactory performance close to CL. Furthermore, the proposed architecture exhibits lower estimation error than the state-of-the-art ML-based schemes. [ABSTRACT FROM AUTHOR]

Machine learning can provide deep insights into data, allowing machines to make high-quality predictions and having been widely used in real-world applications, such as text mining, visual classification, and recommender systems. However, most sophisticated machine learning approaches suffer from huge time costs when operating on large-scale data. This issue calls for the need of Large-scale Machine Learning (LML), which aims to learn patterns from big data with comparable performance efficiently. In this paper, we offer a systematic survey on existing LML methods to provide a blueprint for the future developments of this area. We first divide these LML methods according to the ways of improving the scalability: 1) model simplification on computational complexities, 2) optimization approximation on computational efficiency, and 3) computation parallelism on computational capabilities. Then we categorize the methods in each perspective according to their targeted scenarios and introduce representative methods in line with intrinsic strategies. Lastly, we analyze their limitations and discuss potential directions as well as open issues that are promising to address in the future. [ABSTRACT FROM AUTHOR]

Recently, model-free power control approaches have been developed to achieve the near-optimal performance of cell-free (CF) massive multiple-input multiple-output (MIMO) with affordable computational complexity. In particular, deep reinforcement learning (DRL) is one of such promising techniques for realizing effective power control. In this paper, we propose a model-free method adopting the deep deterministic policy gradient algorithm (DDPG) with feedforward neural networks (NNs) to solve the downlink max-min power control problem in CF massive MIMO systems. Our result shows that compared with the conventional convex optimization algorithm, the proposed DDPG method can effectively strike a performance-complexity trade-off obtaining 1,000 times faster implementation speed and approximately the same achievable user rate as the optimal solution produced by conventional numerical convex optimization solvers, thereby offering effective power control implementations for large-scale systems. Finally, we extend the DDPG algorithm to both the max-sum and the max-product power control problems, while achieving better performance than that achieved by the conventional deep learning algorithm. [ABSTRACT FROM AUTHOR]

The rapid development of Convolutional Neural Networks (CNNs) in recent years has triggered significant breakthroughs in many machine learning (ML) applications. The ability to understand and compare various CNN models available is thus essential. The conventional approach with visualizing each model's quantitative features, such as classification accuracy and computational complexity, is not sufficient for a deeper understanding and comparison of the behaviors of different models. Moreover, most of the existing tools for assessing CNN behaviors only support comparison between two models and lack the flexibility of customizing the analysis tasks according to user needs. This paper presents a visual analytics system, VAC-CNN (Visual Analytics for Comparing CNNs), that supports the in-depth inspection of a single CNN model as well as comparative studies of two or more models. The ability to compare a larger number of (e.g., tens of) models especially distinguishes our system from previous ones. With a carefully designed model visualization and explaining support, VAC-CNN facilitates a highly interactive workflow that promptly presents both quantitative and qualitative information at each analysis stage. We demonstrate VAC-CNN's effectiveness for assisting novice ML practitioners in evaluating and comparing multiple CNN models through two use cases and one preliminary evaluation study using the image classification tasks on the ImageNet dataset. [ABSTRACT FROM AUTHOR]

Deep neural models, in recent years, have been successful in almost every field, even solving the most complex problem statements. However, these models are huge in size with millions (and even billions) of parameters, demanding heavy computation power and failing to be deployed on edge devices. Besides, the performance boost is highly dependent on redundant labeled data. To achieve faster speeds and to handle the problems caused by the lack of labeled data, knowledge distillation (KD) has been proposed to transfer information learned from one model to another. KD is often characterized by the so-called ‘Student-Teacher’ (S-T) learning framework and has been broadly applied in model compression and knowledge transfer. This paper is about KD and S-T learning, which are being actively studied in recent years. First, we aim to provide explanations of what KD is and how/why it works. Then, we provide a comprehensive survey on the recent progress of KD methods together with S-T frameworks typically used for vision tasks. In general, we investigate some fundamental questions that have been driving this research area and thoroughly generalize the research progress and technical details. Additionally, we systematically analyze the research status of KD in vision applications. Finally, we discuss the potentials and open challenges of existing methods and prospect the future directions of KD and S-T learning. [ABSTRACT FROM AUTHOR]

Actuated by the growing attention to personal healthcare and the pandemic, the popularity of E-health is proliferating. Nowadays, enhancement on medical diagnosis via machine learning models has been highly effective in many aspects of e-health analytics. Nevertheless, in the classic cloud-based/centralized e-health paradigms, all the data will be centrally stored on the server to facilitate model training, which inevitably incurs privacy concerns and high time delay. Distributed solutions like Decentralized Stochastic Gradient Descent (D-SGD) are proposed to provide safe and timely diagnostic results based on personal devices. However, methods like D-SGD are subject to the gradient vanishing issue and usually proceed slowly at the early training stage, thereby impeding the effectiveness and efficiency of training. In addition, existing methods are prone to learning models that are biased towards users with dense data, compromising the fairness when providing E-health analytics for minority groups. In this paper, we propose a Decentralized Block Coordinate Descent (D-BCD) learning framework that can better optimize deep neural network-based models distributed on decentralized devices for E-health analytics. As a gradient-free optimization method, Block Coordinate Descent (BCD) mitigates the gradient vanishing issue and converges faster at the early stage compared with the conventional gradient-based optimization. To overcome the potential data scarcity issues for users’ local data, we propose similarity-based model aggregation that allows each on-device model to leverage knowledge from similar neighbor models, so as to achieve both personalization and high accuracy for the learned models. Benchmarking experiments on three real-world datasets illustrate the effectiveness and practicality of our proposed D-BCD, where additional simulation study showcases the strong applicability of D-BCD in real-life E-health scenarios. [ABSTRACT FROM AUTHOR]

Federated learning (FL) is a popular edge learning approach that utilizes local data and computing resources of network edge devices to train machine learning (ML) models while preserving users’ privacy. Nevertheless, performing efficient learning tasks on the devices and achieving longer battery life are primary challenges faced by federated learning. In this paper, we are the first to study the application of heterogeneous computing (HC) and wireless power transfer (WPT) to federated learning to address these challenges. Especially, we propose a heterogeneous computation and resource allocation framework based on a heterogeneous mobile architecture to achieve effective implementation of FL. To minimize the energy consumption of smart devices and maximize their harvesting energy simultaneously, we formulate an optimization problem featuring multidimensional control, which jointly considers time splitting for WPT, dataset size allocation, transmit power allocation and subcarrier assignment during communications, and processor frequency of processing units (central processing unit (CPU) and graphics processing unit (GPU)). However, the major obstacle is how to design a proper algorithm to solve this optimization problem efficiently. For this purpose, we decouple the optimization variables so as to achieve high efficiency in deriving its solution. Particularly, we first compute the optimal processor frequency and dataset size allocation via employing the Lagrangian dual method, followed by finding the closed-form solution to the optimal time splitting allocation, and finally attain the optimal subcarrier assignment as well as transmit power for transmissions through an iteration algorithm. To evaluate the performance of our proposed scheme, we set up four baseline schemes as comparison, and simulation results show that the proposed scheme converges quite fast and better enhance the energy efficiency of the wireless powered FL system compared with the baseline schemes. [ABSTRACT FROM AUTHOR]

A central challenge in protein modeling research and protein structure prediction in particular is known as decoy selection. The problem refers to selecting biologically-active/native tertiary structures among a multitude of physically-realistic structures generated by template-free protein structure prediction methods. Research on decoy selection is active. Clustering-based methods are popular, but they fail to identify good/near-native decoys on datasets where near-native decoys are severely under-sampled by a protein structure prediction method. Reasonable progress is reported by methods that additionally take into account the internal energy of a structure and employ it to identify basins in the energy landscape organizing the multitude of decoys. These methods, however, incur significant time costs for extracting basins from the landscape. In this paper, we propose a novel decoy selection method based on non-negative matrix factorization. We demonstrate that our method outperforms energy landscape-based methods. In particular, the proposed method addresses both the time cost issue and the challenge of identifying good decoys in a sparse dataset, successfully recognizing near-native decoys for both easy and hard protein targets. [ABSTRACT FROM AUTHOR]

In an Internet of Vehicle (IoV) system, federated learning (FL) is a new approach to process real-time vehicle data in a distributed way, which can improve the driving experience and service quality. However, due to the high mobility and uncertainty of vehicles, the existing federated learning protocols are difficult to meet the full requirements from an IoV system, such as efficient resource allocation, high precision learning, and fast convergence of learning algorithm. To solve the above problems, in this paper, we propose a semi-synchronous federated learning (Semi-SynFed) protocol, to improve the performance of machine learning at Internet of Vehicles. In Semi-SynFed, we first select appropriate nodes to participate the aggregation by its computing capacity, network capacity and learning value of training samples. Meanwhile, the dynamic waiting time technique is designed to adjust the server waiting time at each round dynamically, which makes federated learning process work much more efficient. Finally, a dynamic aggregation scheme is designed to aggregate model parameters in an asynchronous way. To evaluate the reliability of Semi-SynFed, we establish a simulation of IoVs based on the real world data. In the comparative experiments, the results show that the Semi-SynFed outperforms the existing federated learning protocols in terms of convergence speed and resource consumption. [ABSTRACT FROM AUTHOR]

The proliferation of machine learning services in the last few years has raised data privacy concerns. Homomorphic encryption (HE) enables inference using encrypted data but it incurs 100x–10,000x memory and runtime overheads. Secure deep neural network (DNN) inference using HE is currently limited by computing and memory resources, with frameworks requiring hundreds of gigabytes of DRAM to evaluate small models. To overcome these limitations, in this paper we explore the feasibility of leveraging hybrid memory systems comprised of DRAM and persistent memory. In particular, we explore the recently-released Intel® Optane™ PMem technology and the Intel® HE-Transformer nGraph® to run large neural networks such as MobileNetV2 (in its largest variant) and ResNet-50 for the first time in the literature. We present an in-depth analysis of the efficiency of the executions with different hardware and software configurations. Our results conclude that DNN inference using HE incurs on friendly access patterns for this memory configuration, yielding efficient executions. [ABSTRACT FROM AUTHOR]

As an emerging distributed machine learning (ML) method, federated learning (FL) can protect data privacy through collaborative learning of artificial intelligence (AI) models across a large number of devices. However, inefficiency and vulnerability to poisoning attacks have slowed FL performance. Therefore, a blockchain-based asynchronous federated learning (BAFL) framework is proposed to ensure the security and efficiency required by FL. The blockchain ensures that the model data cannot be tampered with while asynchronous learning speeds up global aggregation. A novel entropy weight method is used to evaluate the participating rank and proportion of the local model trained in BAFL of the devices. The energy consumption and local model update efficiency are balanced by adjusting the local training and communication delay and optimizing the block generation rate. The extensive evaluation results show that the proposed BAFL framework has higher efficiency and higher performance for preventing poisoning attacks than other distributed ML methods. [ABSTRACT FROM AUTHOR]

Machine learning approaches have exhibited high potential in power system transient stability assessment (TSA), yet their initial preparation stages of stability knowledge base generation (SKBG) based on time-domain simulations often undergo high computational costs. In fact, how to ease the heavy computational burden of SKBG without sacrificing the reliability is still a significant challenge. To address this problem, this paper develops a semi-supervised ensemble learning (SSEL) framework for reliable SKBG acceleration, without additional need for computing hardware upgrading. In particular, it performs detailed simulations for a minority of cases and fast simulations for the majority ones to reduce the total computation time. Considering the absence of stability status information of those fast simulated cases, an SSEL scheme is systematically designed to reliably label their stability status. Before implementing SSEL, two concise feature descriptors are first introduced to efficiently extract transient features from multiplex system trajectories. Then, all the cases are characterized in a unified feature space, whereby a series of semi-supervised support vector machines are trained in randomly formed subspaces. Afterward, these single machines are systematically combined to derive an enhanced SSEL model, which is able to make reliable and robust labeling decisions. Further, a backtrace strategy is carefully devised for SSEL, so as to maintain the high reliability of SKBG. Test results on the IEEE 39-bus system and the realistic GD Power Grid in South China illustrate the excellent performances of the proposed framework on SKBG acceleration. [ABSTRACT FROM AUTHOR]

Spatial structured models are predictive models that capture dependency structure between samples based on their locations in the space. Learning such models plays an important role in many geoscience applications such as water surface mapping, but it also poses significant challenges due to implicit dependency structure in continuous space and high computational costs. Existing models often assume that the dependency structure is based on either spatial proximity or network topology, and thus cannot incorporate complex dependency structure such as contour and flow direction on a 3D potential surface. To fill the gap, we recently proposed a novel spatial structured model called hidden Markov contour tree (HMCT), which generalizes the traditional hidden Markov model from a total order sequence to a partial order polytree. HMCT also advances existing work on hidden Markov trees through capturing complex contour structures on a 3D surface. We proposed efficient model construction and learning algorithms. This paper extends our initial HMCT model into a post-processor that can refine the classified results from other existing models. We analyzed the theoretical properties of the extended model. Evaluations on real-world flood mapping datasets show that HMCT outperforms multiple baseline methods in classification performance and the HMCT can also effectively enhance the results of other baseline methods. Computational experiments also show that HMCT is scalable to large data sizes (e.g., classifying millions of samples in seconds). [ABSTRACT FROM AUTHOR]

User churn is an important issue in online services that threatens the health and profitability of services. Most of the previous works on churn prediction convert the problem into a binary classification task where the users are labeled as churned and non-churned. More recently, some works have tried to convert the user churn prediction problem into the prediction of user return time. In this approach which is more realistic in real world online services, at each time-step the model predicts the user return time instead of predicting a churn label. However, the previous works in this category suffer from lack of generality and require high computational complexity. In this paper, we introduce ChOracle, an oracle that predicts the user churn by modeling the user return times to service by utilizing a combination of Temporal Point Processes and Recurrent Neural Networks. Moreover, we incorporate latent variables into the proposed recurrent neural network to model the latent user loyalty to the system. We also develop an efficient approximate variational inference algorithm for learning parameters of the proposed RNN by using back propagation through time. Finally, we demonstrate the superior performance of ChOracle on a wide variety of real world datasets. [ABSTRACT FROM AUTHOR]

With the availability of various sensors in the smartphone, identifying a locomotion mode becomes convenient and effortless in recent years. Information about locomotion mode helps to improve journey planning, travel time estimation, and traffic management. Though there exists a significant amount of work towards locomotion mode recognition, the performance of these work is not pertinent and heavily depends on the labeled training instances. As it is impractical to gather a prior information (labeled instances) about all types of locomotion modes, the recognition model should be able to identify a new or unseen locomotion mode without having any corresponding training instance. This paper proposes a sensors based deep learning model to identify a locomotion mode by using labeled training instances. The approach also incorporates a concept of Zero-Shot learning to identify an unseen locomotion mode. The model obtains an attribute matrix based on the fusion of three semantic matrices. It also constructs a feature matrix by extracting the deep learning and hand-crafted features from the training instances. Later, the model builds a classifier by learning a mapping between attribute and feature matrices. Finally, this work evaluates the performance of the approach on collected and existing datasets using accuracy and F1 score. [ABSTRACT FROM AUTHOR]

Crowdsourcing is a hotspot research field which can facilitate machine learning by collecting labels to train models. Consequently, the state-of-the-art research efforts in crowdsourcing focus on truth inference or label integration, to remove inconsistent labels or to alleviate biased labeling. In turn, the integrated labels will be used to fine-tune machine learning models. Particularly, in this paper, we change the target of truth inference in crowdsourcing from discrete labels to multiple comments given by online participants, that is, the integration of the crowdsourced comments. For such a goal, we propose a Self-play and Sentiment-Emphasized Comment Integration Framework (SSECIF), based on deep Q-learning, with three unique features. First, our framework SSECIF can generate the comment integration in a totally self-play way, without relying on the ground truth generated by human effort. Second, the integrated comment generated by SSECIF can include salient content with low redundancy. Third, the proposed framework SSECIF has emphasized, with a higher intensity, the sentiment in the integrated comment, in order to reflect the attitude or opinion more obviously. Extensive evaluation on real-world datasets demonstrates that SSECIF has achieved the best overall performance in terms of both effectiveness and efficiency, compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Federated Edge Learning (FEL) enables a massive number of edge devices (e.g. smart phones) to train machine learning models collaboratively. Due to the inherent unreliability of participating edge devices and the unpredictable network conditions, the final model accuracy is largely determined by the communication time between edge devices and the parameter server (PS). However, there exists very limited work to quantify the influence of the communication time in FEL and show how to minimize its negative impacts from a theoretic perspective. In this paper, we are among the first to develop a formal model of the communication time in FEL and its influence on the final model accuracy. In our work, the set of edge devices involved in each global iteration is defined as the ECP (Engaged Client Pool). We model the communication time cost as a function with respect to the response time distribution of individual devices and the ECP size. By incorporating communication time cost into the convergence rate analysis, we propose the ECPA and H-ECPA algorithms to automatically adjust the size of the ECP so as to maximize the model accuracy in both homogeneous and heterogeneous networks. We also analyze how the tail shape of response time affects the convergence rate, and prove that the heavy tail shape can significantly lower the model accuracy. Finally, we conduct extensive experiments with real datasets, and the results confirm the correctness of our analysis and demonstrate the superiority of our proposed algorithms. [ABSTRACT FROM AUTHOR]

In the era of digitized images, the goal is to extract information from them and create new knowledge thanks to Computer Vision techniques, Machine Learning and Deep Learning. This enables the use of images for early diagnosis and subsequent treatment of a wide range of diseases. In the dermatological field, deep neural networks are used to distinguish between melanoma and non-melanoma images. In this paper, we have underlined two essential points of melanoma detection research. The first aspect considered is how even a simple modification of the parameters in the dataset determines a change of the accuracy of classifiers. In this case, we investigated the Transfer Learning issues. Following the results of this first analysis, we suggest that continuous training-test iterations are needed to provide robust prediction models. The second point is the need to have a more flexible system architecture that can handle changes in the training datasets. In this context, we proposed the development and implementation of a hybrid architecture based on Cloud, Fog and Edge Computing to provide a Melanoma Detection service based on clinical and dermoscopic images. At the same time, this architecture must deal with the amount of data to be analyzed by reducing the running time of the continuous retrain. This fact has been highlighted with experiments carried out on a single machine and different distribution systems, showing how a distributed approach guarantees output achievement in a much more sufficient time. [ABSTRACT FROM AUTHOR]

Many real–world domains involve information naturally represented by graphs, where nodes denote basic patterns while edges stand for relationships among them. The graph neural network (GNN) is a machine learning model capable of directly managing graph–structured data. In the original framework, GNNs are inductively trained, adapting their parameters based on a supervised learning environment. However, GNNs can also take advantage of transductive learning, thanks to the natural way they make information flow and spread across the graph, using relationships among patterns. In this paper, we propose a mixed inductive–transductive GNN model, study its properties and introduce an experimental strategy that allows us to understand and distinguish the role of inductive and transductive learning. The preliminary experimental results show interesting properties for the mixed model, highlighting how the peculiarities of the problems and the data can impact on the two learning strategies. [ABSTRACT FROM AUTHOR]

Data generated at the network edge can be processed locally by leveraging the paradigm of edge computing. To fully utilize the widely distributed data, we concentrate on a wireless edge computing system that conducts model training using decentralized peer-to-peer (P2P) methods. However, there are two major challenges on the way towards efficient P2P model training: limited resources (e.g., network bandwidth and battery life of mobile devices) and time-varying network connectivity due to device mobility or wireless channel dynamics, which receives less attention in recent years. To address these two challenges, this paper studies the impact of topology construction on the P2P training performance. Specifically, we dynamically construct an efficient P2P topology, where model aggregation occurs at the edge. In a nutshell, we first formulate the topology construction for P2P learning (TCPL) problem with resource constraints as an integer programming problem. Then a learning-driven method is proposed to adaptively construct a topology at each training epoch. We evaluate the performance of our proposed algorithm through extensive simulations and physical platform. Evaluation results show that our method can improve the model training efficiency by about 11% with resource constraints, reduce the communication cost by 30% and the network traffic consumption by about 60% under the same accuracy requirement compared to the benchmarks. [ABSTRACT FROM AUTHOR]