Search

We focus on multi-step ahead time series forecasting with the multi-output strategy. From the perspective of multi-task learning (MTL), we recognize imbalanced uncertainties between prediction tasks of different future time steps. Unexpectedly, trained by the standard summed Mean Squared Error (MSE) loss, existing multi-output forecasting models may suffer from performance drops due to the inconsistency between the loss function and the imbalance structure. To address this problem, we reformulate each prediction task as a distinct Gaussian Mixture Model (GMM) and derive a multi-level Gaussian mixture loss function to better fit imbalanced uncertainties in multi-output time series forecasting. Instead of using the two-step Expectation-Maximization (EM) algorithm, we apply the self-attention mechanism on the task-specific parameters to learn the correlations between different prediction tasks and generate the weight distribution for each GMM component. In this way, our method jointly optimizes the parameters of the forecasting model and the mixture model simultaneously in an end-to-end fashion, avoiding the need of two-step optimization. Experiments on three real-world datasets demonstrate the effectiveness of our multi-level Gaussian mixture loss compared to models trained with the standard summed MSE loss function. All the experimental data and source code are available at https://github.com/smallGum/GMM-FNN .

Modern software systems are usually highly configurable, providing users with customized functionality through various configuration options. Understanding how system performance varies with different option combinations is important to determine optimal configurations that meet specific requirements. Due to the complex interactions among multiple options and the high cost of performance measurement under a huge configuration space, it is challenging to study how different configurations influence the system performance. To address these challenges, we propose HINNPerf , a novel hierarchical interaction neural network for performance prediction of configurable systems. HINNPerf employs the embedding method and hierarchic network blocks to model the complicated interplay between configuration options, which improves the prediction accuracy of the method. In addition, we devise a hierarchical regularization strategy to enhance the model robustness. Empirical results on 10 real-world configurable systems show that our method statistically significantly outperforms state-of-the-art approaches by achieving average 22.67% improvement in prediction accuracy. In addition, combined with the Integrated Gradients method, the designed hierarchical architecture provides some insights about the interaction complexity and the significance of configuration options, which might help users and developers better understand how the configurable system works and efficiently identify significant options affecting the performance.

Deep reinforcement learning (DRL), which highly depends on the data representation, has shown its potential in many practical decision-making problems. However, the process of acquiring representations in DRL is easily affected by interference from models, and moreover leaves unnecessary parameters, leading to control performance reduction. In this article, we propose a double sparse DRL via multilayer sparse coding and nonconvex regularized pruning. To alleviate interference in DRL, we propose a multilayer sparse-coding-structural network to obtain deep sparse representation for control in reinforcement learning. Furthermore, we employ a nonconvex łog regularizer to promote strong sparsity, efficiently removing the unnecessary weights with a regularizer-based pruning scheme. Hence, a double sparse DRL algorithm is developed, which can not only learn deep sparse representation to reduce the interference but also remove redundant weights while keeping the robust performance. The experimental results in five benchmark environments of the deep q network (DQN) architecture demonstrate that the proposed method with deep sparse representations from the multilayer sparse-coding structure can outperform existing sparse-coding-based DRL in control, for example, completing Mountain Car with 140.81 steps, achieving near 10% reward increase from the single-layer sparse-coding DRL algorithm, and obtaining 286.08 scores in Catcher, which are over two times the rewards of the other algorithms. Moreover, the proposed algorithm can reduce over 80% parameters while keeping performance improvements from deep sparse representations.

With the rapidly growing attention to multi-view data in recent years, multi-view outlier detection has become a rising field with intense research. These researches have made some success, but still exist some issues that need to be solved. First, many multi-view outlier detection methods can only handle datasets that conform to the cluster structure but are powerless for complex data distributions such as manifold structures. This overly restrictive data assumption limits the applicability of these methods. In addition, almost the majority of multi-view outlier detection algorithms cannot solve the online detection problem of multi-view outliers. To address these issues, we propose a new detection method based on the local similarity relation and data reconstruction, i.e., the Self-Representation Method with Local Similarity Preserving for fast multi-view outlier detection (SRLSP). By using the local similarity structure, the proposed method fully utilizes the characteristics of outliers and detects outliers with an applicable objective function. Besides, a well-designed optimization algorithm is proposed, which completes each iteration with linear time complexity and can calculate each instance parallelly. Also, the optimization algorithm can be easily extended to the online version, which is more suitable for practical production environments. Extensive experiments on both synthetic and real-world datasets demonstrate the superiority of the proposed method on both performance and time complexity.

Federated learning allows multiple participants to collaboratively train an efficient model without exposing data privacy. However, this distributed machine learning training method is prone to attacks from Byzantine clients, which interfere with the training of the global model by modifying the model or uploading the false gradient. In this paper, we propose a novel serverless federated learning framework Committee Mechanism based Federated Learning (CMFL), which can ensure the robustness of the algorithm with convergence guarantee. In CMFL, a committee system is set up to screen the uploaded local gradients. The committee system selects the local gradients rated by the elected members for the aggregation procedure through the selection strategy, and replaces the committee member through the election strategy. Based on the different considerations of model performance and defense, two opposite selection strategies are designed for the sake of both accuracy and robustness. Extensive experiments illustrate that CMFL achieves faster convergence and better accuracy than the typical Federated Learning, in the meanwhile obtaining better robustness than the traditional Byzantine-tolerant algorithms, in the manner of a decentralized approach. In addition, we theoretically analyze and prove the convergence of CMFL under different election and selection strategies, which coincides with the experimental results.

Unmanned aerial vehicles (UAVs) can be leveraged in mobile crowdsensing (MCS) to conduct sensing tasks at remote or rural areas through computation offloading and data sensing. Nonetheless, both computation offloading and data sensing have been separately investigated in most existing studies. In this paper, we propose a novel cooperative data sensing and computation offloading scheme for the UAV-assisted MCS system with an aim to maximize the overall system utility. First, a multi-objective function is formulated to evaluate the system utility by jointly considering flight direction, flight distance,task offloading proportion, and server offload selection for each UAV. Then, the problem is modeled as a partially observable Markov decision process and a multi-agent actor-critic algorithm framework is proposed to train the strategy network for UAVs. Due to high delay and energy cost caused by communications among multiple agents, we leverage the critic network to model other agents and to seek equilibrium among all UAVs rather than adopting the explicit channel for information exchange. Furthermore, we introduce attention mechanism to enhance the convergence performance in model training phases. Finally, experimental results demonstrate the effectiveness and applicability of our scheme. Compared with baselines, our algorithm shows significant advantages in convergence performance and system utility.

With the growing number of competing Web services that provide similar functionality, Quality-of-Service (QoS) prediction is becoming increasingly important for various QoS-aware approaches of Web services. Collaborative filtering (CF), which is among the most successful personalized prediction techniques for recommender systems, has been widely applied to Web service QoS prediction. In addition to using conventional CF techniques, a number of studies extend the CF approach by incorporating additional information about services and users, such as location, time, and other contextual information from the service invocations. There are also some studies that address other challenges in QoS prediction, such as adaptability, credibility, privacy preservation, and so on. In this survey, we summarize and analyze the state-of-the-art CF QoS prediction approaches of Web services and discuss their features and differences. We also present several Web service QoS datasets that have been used as benchmarks for evaluating the predition accuracy and outline some possible future research directions.

Change impact analysis (CIA) is a specialized process of program comprehension that investigates the ripple effects of a code change in a software system. In this paper, we present a boosting way for change impact analysis via mapping the historical change-patterns to current CIA task in a cross-project scenario. The change-patterns reflect the coupling dependencies between changed entities in a change set. A traditional CIA tool (such as ImpactMiner) outputs an initial impact set for a starting entity. To boost the traditional CIA tool, our approach retrieves an equivalent entity from various historical change sets for the starting entity. Then, the change-patterns between the equivalent entity and the rest of entities in the change set are mapped to the CIA task at hand. For current CIA task, if an entity in the initial impact set involves the similar change-pattern with the starting entity when comparing with the mapped change-pattern, we will reward the impacted confidence of the entity. Accuracy improvements are observed in the experiments when applying our boosting method to three famous CIA tools, i.e., ImpactMiner, JRipples and ROSE.

Code review is a common software engineering practice of practical importance to reduce software defects. Review today is often with the help of specialized tools, such as Gerrit. However, even in a tool-supported code review involves a significant amount of human effort to understand the code change, because the information required to inspect code changes may distribute across multiple files that reviewers are not familiar with. Code changes are often organized as commits for review. In this paper, we found that most of the commits contain a salient class(es), which is saliently modified and causes the modification of the rest classes in a commit. Our user studies confirmed that identifying the salient class in a commit can facilitate reviewers in understanding code change. Inspired by the effectiveness of machine learning techniques in the classification field, we model the salient class identification as a binary classification problem and extract a number of discriminative features from commit to characterize the salience of a class. The experiment results show that our approach achieves an accuracy of 88%. A user study with industrial developers shows that our approach can really improve the efficiency of reviewers understanding code changes in a reviewing scenario without using comment.

Exploring the complementary information of multi-view data to improve clustering effects is a crucial issue in multi-view clustering. In this paper, we propose a novel model based on information theory termed Informative Multi-View Clustering (IMVC), which extracts the common and view-specific information hidden in multi-view data and constructs a clustering-oriented comprehensive representation. More specifically, we concatenate multiple features into a unified feature representation, then pass it through a encoder to retrieve the common representation across views. Simultaneously, the features of each view are sent to a encoder to produce a compact view-specific representation, respectively. Thus, we constrain the mutual information between the common representation and view-specific representations to be minimal for obtaining multi-level information. Further, the common representation and view-specific representation are spliced to model the refined representation of each view, which is fed into a decoder to reconstruct the initial data with maximizing their mutual information. In order to form a comprehensive representation, the common representation and all view-specific representations are concatenated. Furthermore, to accommodate the comprehensive representation better for the clustering task, we maximize the mutual information between an instance and its k-nearest neighbors to enhance the intra-cluster aggregation, thus inducing well separation of different clusters at the overall aspect. Finally, we conduct extensive experiments on six benchmark datasets, and the experimental results indicate that the proposed IMVC outperforms other methods., We think the paper writing isn't good enough, so we would like to withdraw the paper and renew the writing manner

Code comments are one of the important documents to help developers review and comprehend source code. In recent studies, researchers have proposed many deep learning models to generate the method header comments (i.e., method comment), which have achieved encouraging results. The comments in the method, which is called inline comment, are also important for program comprehension. Unfortunately, they have not received enough attention in automatic generation when comparing with the method comments. In this paper, we compare and analyze the similarities and differences between the method comments and the inline comments. By applying the existing models of generating method comments to the inline comment generation, we find that these existing models perform worse on the task of inline comment generation. We then further explore the possible reasons and obtain a number of new observations. For example, we find that there are a lot of templates (i.e., comments with same or similar structures) in the method comment dataset, which makes the models perform better. Some terms were thought to be important (e.g., API calls) in the comment generation by previous study does not significantly affect the quality of the generated comments, which seems counter-intuitive. Our findings may give some implications for building the approaches of method comment or inline comment generation in the future.

Cascading failure is a ubiquitous phenomenon that can paralyze networked systems in a short time. Many traditional studies of cascading failures have been conducted from the perspective of either an attacker or a defender. In reality, however, malicious attacks on networks are rarely a one-sided process. Instead, both the attacker and defender are actively involved. We use game theory to study the strategies of both sides in terms of an attacker-defender game on complex networks. Based on the concept of the Stackelberg competition, we propose a multi-round attacker-defender game model on complex networks, allowing high flexibility in the available actions for both sides in the game. The model we propose allows the two sides to specify certain parameters of the network to attack/defend and further allocate a certain amount of resources for the attack/defense. Such flexibility allows the model to capture the actions of the attackers and defenders more precisely and simulate the attack process in a more realistic manner. We propose an iterative search algorithm to search for desirable strategies with systematic experiments on various types of networks and associated parameters and in terms of different relative resource owned by the attacker and defender.

Recently, the microservice becomes a popular architecture to construct cloud native systems due to its agility. In cloud native systems, autoscaling is a key enabling technique to adapt to workload changes by acquiring or releasing the right amount of computing resources. However, it becomes a challenging problem in microservice applications, since such an application usually comprises a large number of different microservices with complex interactions. When the performance decreases due to an unpredictable workload peak, it is difficult to pinpoint the scaling-needed services which need to scale out and evaluate how many resources they need. In this paper, we present a novel system named Microscaler to automatically identify the scaling-needed services and scale them to meet the Service Level Agreement (SLA) with an optimal cost for microservice applications. Microscaler first collects the quality of service (QoS) metrics in the service mesh enabled microservice infrastructure. Then, it determines under-provisioning or over-provisioning service instances along the service dependency graph with a novel scaling-needed service criterion named service power. The service dependency graph could be obtained by correlating each request flow in the service mesh. By combining an online learning approach and a step-by-step heuristic approach, Microscaler can precisely reach the optimal service scale meeting the SLA requirements. The experimental evaluations in a microservice benchmark show that Microscaler achieves an average 93% precision in scaling-needed service determination and converges to the optimal service scale faster than several state-of-the-art methods. Moreover, Microscaler is lightweight and flexible enough to work in a large-scale microservice system.

Much of the current research in Ethereum transaction records focuses on the statistical analysis and measurements of existing data; however, the evolution mechanism of Ethereum transactions is an important, yet seldom discussed issue. In this work, we first collect the transaction data of Ethereum and build network models from a microlevel view and then use a link-prediction-based framework to quantify the impact of network characteristics on Ethereum evolution. Next, we explore the graph structure properties and the driving factors of newly generated transaction relationships. Experimental results show that the local and microscopic structure of Ethereum networks is star-shaped, and the transaction frequency of addresses has a great impact on the evolution of Ethereum transaction relationships. First-layer nodes of microstructures dominate the network evolution. Moreover, the degree of addresses is an effective basis for predicting the direction of new transactions. Potential further studies on Ethereum transaction link prediction are discussed, for example, the label effect of center addresses.

Mobile devices can generate a tremendous amount of unique data, and thus create countless opportunities for deep learning tasks. Due to the concerns of data privacy, it is often impractical to log all the data to a central server for training a satisfactory model. Federated learning, in which the participating devices can train a shared global model collaboratively while keeping their data locally. However, it is not a trivial task to train the Deep Neural Networks (DNNs) with millions and billions of parameters on resource-constrained mobile devices in a federated manner. We replace each FC layer with two low-rank projection matrics to compact the DNNs model, and establish a global error function to recover the outputs of the compressed DNNs model. Then we design a communication-efficient federated optimation algorithm to reduce communication cost further. Consider that the heterogeneous devices may run different models at the same time, we devise three different training patterns to integrate the heterogeneous devices running different models. We conduct extensive experiments on both Independently Identically Distribution (IID) and non-IID datasets. The experimental results demonstrate that the proposed framework can significantly reduce the number of parameters and communication cost while maintaining performance.

App reviews are crowdsourcing knowledge of user experience with the apps, providing valuable information for app release planning, such as major bugs to fix and important features to add. There exist prior explorations on app review mining for release planning, however, most of the studies strongly rely on pre-defined classes or manually-annotated reviews. Also, the new review characteristic, i.e., the number of users who rated the review as helpful, which can help capture important reviews, has not been considered previously. In the paper, we propose a novel framework, named SOLAR, aiming at accurately summarizing helpful user reviews to developers. The framework mainly contains three modules: The review helpfulness prediction module, topic-sentiment modeling module, and multi-factor ranking module. The review helpfulness prediction module assesses the helpfulness of reviews, i.e., whether the review is useful for developers. The topic-sentiment modeling module groups the topics of the helpful reviews and also predicts the associated sentiment, and the multi-factor ranking module aims at prioritizing semantically representative reviews for each topic as the review summary. Experiments on five popular apps indicate that SOLAR is effective for review summarization and promising for facilitating app release planning.

With the continuous development of web technology, Web3.0 has attracted a considerable amount of attention due to its unique decentralized characteristics. The digital economy is an important driver of high-quality economic development and is currently in a rapid development stage. In the digital economy scenario, the centralized nature of the Internet and other characteristics usually bring about security issues such as infringement and privacy leakage. Therefore, it is necessary to investigate how to use Web3.0 technologies to solve the pain points encountered in the development of the digital economy by fully exploring the critical technologies of digital economy and Web3.0. In this paper, we discuss the aspects of Web3.0 that should be integrated with the digital economy to better find the entry point to solve the problems by examining the latest advances of Web3.0 in machine learning, finance, and data management. We hope this research will inspire those who are involved in both academia and industry, and finally help to build a favourable ecology for the digital economy., Comment: 14 pages, 5 figures