Understanding the Automated Parameter Optimization on Transfer Learning for Cross-Project Defect Prediction: An Empirical Study

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Authors

  • Ke Li
  • Zilin Xiang
  • Tao Chen
  • Shuo Wang
  • Kay Chen Tan

Colleges, School and Institutes

External organisations

  • UESTC, Chengdu, China
  • Loughborough University
  • City University of Hong Kong
  • University of Exeter

Abstract

Data-driven defect prediction has become increasingly important in software engineering process. Since it is not uncommon that data from a software project is insufficient for training a reliable defect prediction model, transfer learning that borrows data/konwledge from other projects to facilitate the model building at the current project, namely cross-project defect prediction (CPDP), is naturally plausible. Most CPDP techniques involve two major steps, i.e., transfer learning and classification, each of which has at least one parameter to be tuned to achieve their optimal performance. This practice fits well with the purpose of automated parameter optimization. However, there is a lack of thorough understanding about what are the impacts of automated parameter optimization on various CPDP techniques. In this paper, we present the first empirical study that looks into such impacts on 62 CPDP techniques, 13 of which are chosen from the existing CPDP literature while the other 49 ones have not been explored before. We build defect prediction models over 20 real-world software projects that are of different scales and characteristics. Our findings demonstrate that: (1) Automated parameter optimization substantially improves the defect prediction performance of 77% CPDP techniques with a manageable computational cost. Thus more efforts on this aspect are required in future CPDP studies. (2) Transfer learning is of ultimate importance in CPDP. Given a tight computational budget, it is more cost-effective to focus on optimizing the parameter configuration of transfer learning algorithms (3) The research on CPDP is far from mature where it is ‘not difficult’ to find a better alternative by making a combination of existing transfer learning and classification techniques. This finding provides important insights about the future design of CPDP techniques.

Details

Original languageEnglish
Title of host publication42nd International Conference on Software Engineering (ICSE 2020)
Publication statusAccepted/In press - 9 Dec 2019
Event42nd International Conference on Software Engineering (ICSE 2020) - Seoul, Korea, Republic of
Duration: 23 May 202029 May 2020

Conference

Conference42nd International Conference on Software Engineering (ICSE 2020)
CountryKorea, Republic of
CitySeoul
Period23/05/2029/05/20

Keywords

  • Cross-project defect prediction, transfer learning, classification techniques, automated parameter optimization