Search

Pre-trained code representation models such as CodeBERT have demonstrated superior performance in a variety of software engineering tasks, yet they are often heavy in complexity, quadratically with the length of the input sequence. Our empirical analysis of CodeBERT's attention reveals that CodeBERT pays more attention to certain types of tokens and statements such as keywords and data-relevant statements. Based on these findings, we propose DietCode, which aims at lightweight leverage of large pre-trained models for source code. DietCode simplifies the input program of CodeBERT with three strategies, namely, word dropout, frequency filtering, and an attention-based strategy which selects statements and tokens that receive the most attention weights during pre-training. Hence, it gives a substantial reduction in the computational cost without hampering the model performance. Experimental results on two downstream tasks show that DietCodeBERT provides comparable results to CodeBERT with 40% less computational cost in fine-tuning and testing., Comment: Accepted to be published in ESEC/FSE 2022

Learning program representations has been the core prerequisite of code intelligent tasks such as code search and code clone detection. The state-of-the-art pre-trained models such as CodeBERT require the availability of large-scale code corpora. However, gathering training samples can be costly and infeasible for domain-specific languages such as Solidity for smart contracts. In this paper, we propose Zecoler, a zero-shot learning approach for code representations. Zecoler is built upon a pre-trained programming language model. In order to elicit knowledge from the pre-trained models efficiently, Zecoler casts the downstream tasks to the same form of pre-training tasks by inserting trainable prompts into the original input. Then, it employs the prompt learning technique which optimizes the pre-trained model by merely adjusting the original input. This enables the representation model to efficiently fit the scarce task-oriented data while reusing pre-trained knowledge. We evaluate Zecoler in three code intelligent tasks in two program languages that have no training samples, namely, Solidity and Go, with model trained in corpora of common languages such as Java. Experimental results show that our approach significantly outperforms baseline models in both zero-shot and few-shot settings., Comment: In 30th International Conference on Program Comprehension (ICPC 22), May 16 to 17, 2022, Virtual Event, Pittsburgh

Recently, pre-trained programming language models such as CodeBERT have demonstrated substantial gains in code search. Despite showing great performance, they rely on the availability of large amounts of parallel data to fine-tune the semantic mappings between queries and code. This restricts their practicality in domain-specific languages with relatively scarce and expensive data. In this paper, we propose CDCS, a novel approach for domain-specific code search. CDCS employs a transfer learning framework where an initial program representation model is pre-trained on a large corpus of common programming languages (such as Java and Python), and is further adapted to domain-specific languages such as SQL and Solidity. Unlike cross-language CodeBERT, which is directly fine-tuned in the target language, CDCS adapts a few-shot meta-learning algorithm called MAML to learn the good initialization of model parameters, which can be best reused in a domain-specific language. We evaluate the proposed approach on two domain-specific languages, namely, SQL and Solidity, with model transferred from two widely used languages (Python and Java). Experimental results show that CDCS significantly outperforms conventional pre-trained code models that are directly fine-tuned in domain-specific languages, and it is particularly effective for scarce data., Comment: Accepted by ICSE 2022 (The 44th International Conference on Software Engineering)

Cross-project defect prediction is taken as an effective means of predicting software defects when the data shortage exists in the early phase of software development. Unfortunately, the precision of cross-project defect prediction is usually poor, largely because of the differences between the reference and the target projects. Having realized the project differences, this paper proposes CPDP, a feature-based transfer learning approach to cross-project defect prediction. The core insight of CPDP is to (1) filter and transfer highly-correlated data based on data samples in the target projects, and (2) evaluate and choose learning schemas for transferring data sets. Models are then built for predicting defects in the target projects. We have also conducted an evaluation of the proposed approach on PROMISE datasets. The evaluation results show that, the proposed approach adapts to cross-project defect prediction in that f-measure of 81.8% of projects can get improved, and AUC of 54.5% projects improved. It also achieves similar f-measure and AUC as some inner-project defect prediction approaches.

Using the freeware in GitHub, we are often confused by a phenomenon: the new version of GitHub freeware usually was released in an indefinite frequency, and developers often committed nothing for a long time. This evolution phenomenon interferes with our own development plan and architecture design. Why do these updates happen at that time? Can we predict GitHub software version evolution by developers' activities? This paper aims to explore the developer commit patterns in GitHub, and try to mine the relationship between these patterns (if exists) and code evolution. First, we define four metrics to measure commit activity and code evolution: the changes in each commit, the time between two commits, the author of each changes, and the source code dependency. Then, we adopt visualization techniques to explore developers' commit activity and code evolution. Visual techniques are used to describe the progress of the given project and the authors' contributions. To analyze the commit logs in GitHub software repository automatically, Commits Analysis Tool (CAT) is designed and implemented. Finally, eight open source projects in GitHub are analyzed using CAT, and we find that: 1) the file changes in the previous versions may affect the file depend on it in the next version, 2) the average days around "huge commit" is 3 times of that around normal commit. Using these two patterns and developer's commit model, we can predict when his next commit comes and which file may be changed in that commit. Such information is valuable for project planning of both GitHub projects and other projects which use GitHub freeware to develop software.

The research of developers' contribution is an important part of the software evolution area. It allows project owners to find potential long-term contributors earlier and helps the newcomers to improve their behaviors. In this paper, we examined the contribution characteristics of developers in open source environment based on visual analysis, and presented approaches from three aspects-influencing factors, time characteristics and region characteristics. Our analysis used data from github and revealed some regular patterns. We found that the code which newcomers started to contribute with more people engaged in would lead to less contribution in some degree. We also found that there's a relation between developers' early and later period contribution. In addition, developers from different regions were more likely to have dominant relationship. Our findings may provide some support for future research in the area of software evolution.

There are various approaches for web news extraction, including tree-edit distance approach that needs to assume the existence of web templates, visual wrapper based approach that requires large training sets and statistical approach whose flexibility is low. In this paper, a blocking tag based Web news extraction approach is proposed, which automatically detects the blocking tags that break the web page down into functional areas and then analyzes the web page according to the blocking tags to find out the news content. We have implemented the proposed news extraction approach in a news search engine which has been applied in business of an intelligence enterprise. Compared with related work, our approach does not require the web page templates or large training sets, and the complexity is lower.