Product Sampling for Product Lines: The Scalability Challenge
Tobias Pett, TU Braunschweig, Germany
Thomas Thüm, TU Braunschweig, Germany
Tobias Runge, TU Braunschweig, Germany
Sebastian Krieter, University of Magdeburg, Germany
Malte Lochau, TU Darmstadt, Germany
Ina Schaefer, TU Braunschweig, Germany
Quality assurance for product lines is often infeasible for each product separately. Instead, often only a subset of all products (i.e. a sample) is considered in testing such that at least the coverage of certain feature interactions is guaranteed. While pair-wise interaction sampling covers all interactions between two features, its generalization to t-wise interaction sampling ensures coverage for all interactions among t features. However, sampling for large product lines poses a challenge, as today’s algorithms tend to run out of memory, do not terminate, or produce samples, which are too large to be tested. To approach this challenge, we provide a set of large real-world feature-models with up-to 19 thousand features, which are supposed to be sampled. The performance of the sampling is evaluated based on the time and memory consumed to retrieve a sample and the sample size for a given coverage (i.e. t value). A well-performing sampling algorithm achieves full t-wise coverage, while minimizing all three of these properties.