Search

Centralized training is widely utilized in the field of multi-agent reinforcement learning (MARL) to assure the stability of training process. Once a joint policy is obtained, it is critical to design a value function factorization method to extract optimal decentralized policies for the agents, which needs to satisfy the individual-global-max (IGM) principle. While imposing additional limitations on the IGM function class can help to meet the requirement, it comes at the cost of restricting its application to more complex multi-agent environments. In this paper, we propose QFree, a universal value function factorization method for MARL. We start by developing mathematical equivalent conditions of the IGM principle based on the advantage function, which ensures that the principle holds without any compromise, removing the conservatism of conventional methods. We then establish a more expressive mixing network architecture that can fulfill the equivalent factorization. In particular, the novel loss function is developed by considering the equivalent conditions as regularization term during policy evaluation in the MARL algorithm. Finally, the effectiveness of the proposed method is verified in a nonmonotonic matrix game scenario. Moreover, we show that QFree achieves the state-of-the-art performance in a general-purpose complex MARL benchmark environment, Starcraft Multi-Agent Challenge (SMAC).

Incorrect placement of methods within classes is a typical code smell called Feature Envy, which causes additional maintenance and cost during evolution. To remove this design flaw, several Move Method refactoring tools have been proposed. To the best of our knowledge, state-of-the-art related techniques can be broadly divided into two categories: the first line is non-machine-learning-based approaches built on software measurement, while the selection and thresholds of software metrics heavily rely on expert knowledge. The second line is machine learning-based approaches, which suggest Move Method refactoring by learning to extract features from code information. However, most approaches in this line treat different forms of code information identically, disregarding their significant variation on data analysis. In this paper, we propose an approach to recommend Move Method refactoring named RMove by automatically learning structural and semantic representation from code fragment respectively. We concatenate these representations together and further train the machine learning classifiers to guide the movement of method to suitable classes. We evaluate our approach on two publicly available datasets. The results show that our approach outperforms three state-of-the-art refactoring tools including PathMove, JDeodorant, and JMove in effectiveness and usefulness. The results also unveil useful findings and provide new insights that benefit other types of feature envy refactoring techniques., Comment: 13 pages, 6 figures. This paper is the corrected version of the "RMove: Recommending Move Method Refactoring Opportunities using Structural and Semantic Representations of Code", which has published on ICSME22,and the missing part, "Method Structural Information Collection", is added in Chapter 3

A class of nonparametric two-sample tests has been proposed in this article. As a generalization of the original \v{S}id\'aks' test, the proposed test statistic is developed as the sum of the maximal precedence and maximal exceedance statistics. Unlike the \v{S}id\'ak-type precedence-exceedance test and the maximal precedence test, the proposed test is suitable for a two-sided alternative while being free from any parametric assumption. Exact distribution of the test statistic is obtained under the null as well as under the Lehmann alternative. Power value comparison has been carried out that shows the competency of the proposed test as a useful alternative to a number of existing tests based on precedence-exceedance statistics. Real-life example is provided to illustrate the application of the proposed test.

The great influence of Bitcoin has promoted the rapid development of blockchain-based digital currencies, especially the altcoins, since 2013. However, most altcoins share similar source codes, resulting in concerns about code innovations. In this paper, an empirical study on existing altcoins is carried out to offer a thorough understanding of various aspects associated with altcoin innovations. Firstly, we construct the dataset of altcoins, including source code repositories, GitHub fork relations, and market capitalizations (cap). Then, we analyze the altcoin innovations from the perspective of source code similarities. The results demonstrate that more than 85% of altcoin repositories present high code similarities. Next, a temporal clustering algorithm is proposed to mine the inheritance relationship among various altcoins. The family pedigrees of altcoin are constructed, in which the altcoin presents similar evolution features as biology, such as power-law in family size, variety in family evolution, etc. Finally, we investigate the correlation between code innovations and market capitalization. Although we fail to predict the price of altcoins based on their code similarities, the results show that altcoins with higher innovations reflect better market prospects., Comment: 10 pages