Generating Pairwise Covering Arrays for Highly Configurable Software Systems
Chuan Luo, School of Software, Beihang University ,Beijing, China
Jianping Song, School of Software, Beihang University, Beijing, China
Qiyuan Zhao, School of Computing,National University of Singapore,Singapore
Yibei Li, Department of Physics, Tsinghua University, Beijing, China
Shaowei Cai, Institute of Software,Chinese Academy of Sciences, Beijing, China
Chunming Hu, School of Software, Beihang University, Beijing, China
Highly configurable software systems play crucial roles in real-
world applications, which urgently calls for useful testing methods.
Combinatorial interaction testing (CIT) is an effective methodology
for detecting those faults that are triggered by the interaction of
any 𝑡 options, where 𝑡 is the testing strength. Pairwise testing, i.e.,
CIT with 𝑡 = 2, is known to be the most practical and popular
CIT technique, and the pairwise covering array generation (PCAG)
problem is the most critical problem in pairwise testing. Due to
the practical importance of PCAG, many PCAG algorithms have
been proposed. Unfortunately, existing PCAG algorithms suffer
from the severe scalability problem. To this end, the SPLC Scalability
Challenge ( i.e., Product Sampling for Product Lines: The
Scalability Challenge) has been proposed since 2019, in order to
motivate researchers to develop practical PCAG algorithms for
overcoming this scalability problem. In this work, we present a
practical PCAG algorithm dubbed SamplingCA-ASF . To the best
of our knowledge, our experiments show that SamplingCA-ASF
is the fi rst algorithm that can generate PCAs for Automotive02
and Linux , the two hardest and largest-scale instances in the SPLC
Scalability Challenge, within reasonable time. Our experimental
results indicate that SamplingCA-ASF can effectively alleviate the
scalability problem in pairwise testing.