SPLC 2021 Challenge

Variability Fault Localization: A Benchmark

Kien-Tuan Ngo, VNU University of Engineering and Technology, Vietnam
Thu-Trang Nguyen, VNU University of Engineering and Technology, Vietnam
Son Nguyen, VNU University of Engineering and Technology, Vietnam
Hieu Dinh Vo, VNU University of Engineering and Technology, Vietnam
Software fault localization is one of the most expensive, tedious, and time-consuming activities in program debugging. This activity becomes even much more challenging in Software Product Line (SPL) systems due to the variability of failures in SPL systems. These unexpected behaviors are caused by variability faults which can only be exposed under some combinations of system features. Although localizing bugs in non-configurable code has been investigated in-depth, variability fault localization in SPL systems still remains mostly unexplored. To approach this challenge, we propose a benchmark for variability fault localization with a large set of 1,773 buggy versions of six SPL systems and baseline variability fault localization performance results. Our hope is to engage the community to propose new and better approaches to the problem of variability fault localization in SPL systems.

Material

Authors have updated their challenge case description after finding some invalid cases that had to be removed from the study.
We have provided a new description of the challenge with updated figures and numbers.
Challenge case description
Video with an overview of the challenge


The following link provides access to the associated repository. The repo contains two versions.
Please ensure to use the dataset associated with the 2nd version of the case description .
Repository of source code, datasets and more information

Solutions

No solutions yet.

Discussion

No discussion section for the moment.