Ralf Lämmel, Wolfram Schulte. Controllable Combinatorial Coverage in Grammar-Based Testing. In M. Ümit Uyar, Ali Y. Duale, Mariusz A. Fecko, editors, Testing of Communicating Systems, 18th IFIP TC6/WG6.1 International Conference, TestCom 2006, New York, NY, USA, May 16-18, 2006, Proceedings. Volume 3964 of Lecture Notes in Computer Science, pages 19-38, Springer, 2006. [doi]
Given a grammar (or other sorts of meta-data), one can trivially derive combinatorially exhaustive test-data sets up to a specified depth. Without further efforts, such test-data sets would be huge at the least and explosive most of the time. Fortunately, scenarios of grammar-based testing tend to admit non-explosive approximations of naive combinatorial coverage. In this paper, we describe the notion of controllable combinatorial coverage and a corresponding algorithm for test-data generation. The approach is based on a suite of control mechanisms to be used for the characterization of test-data sets as well-defined and understandable approximations of full combinatorial coverage.
The approach has been implemented in the C#-based test-data generator Geno, which has been successfully used in projects that required differential testing, stress testing and conformance testing of grammar-driven functionality.