Towards indicators of instabilities in software product lines: An empirical evaluation of metrics

A Software Product Line (SPL) is a set of software systems (products) that share common functionalities, so-called features. The success of a SPL design is largely dependent on its stability; otherwise, a single implementation change will cause ripple effects in several products. Therefore, there is a growing concern in identifying means to either indicate or predict design instabilities in the SPL source code. However, existing studies up to now rely on conventional metrics as indicators of SPL instability. These conventional metrics, typically used in standalone systems, are not able to capture the properties of SPL features in the source code, which in turn might neglect frequent causes of SPL instabilities. On the other hand, there is a small set of emerging software metrics that take into account specific properties of SPL features. The problem is that there is a lack of empirical validation of the effectiveness of metrics in indicating quality attributes in the context of SPLs. This paper presents an empirical investigation through two set of metrics regarding their power of indicating instabilities in evolving SPLs. A set of conventional metrics was confronted with a set of metrics we instantiated to capture important properties of SPLs. The software evolution history of two SPLs were analysed in our studies. These SPLs are implemented using two different programming techniques and all together they encompass 30 different versions under analysis. Our analysis confirmed that conventional metrics are not good indicators of instabilities in the context of evolving SPLs. The set of employed feature dependency metrics presented a high correlation with instabilities proving its value as indicator of SPL instabilities.