Lennart C. L. Kats, Rob Vermaas, Eelco Visser. Integrated Language Definition Testing. Enabling Test-Driven Language Development. In Kathleen Fisher, Cristina Videira Lopes, editors, Proceedings of the 2011 ACM international conference on Object oriented programming systems languages and applications, OOPSLA 2011. pages 139-154, ACM, New York, NY, USA, 2011.
The reliability of implementations of language definitions – compilers, interpreters, and development environments – is essential for effective software development and maintenance. They are often tested only as an afterthought. Languages with a smaller scope, such as domain-specific languages, often remain untested. General-purpose testing techniques and test case generation methods fall short in providing a low-threshold solution for test-driven language development. In this paper we introduce the notion of a language-parametric testing language (LPTL) that provides a reusable, generic basis for declaratively specifying language definition tests. We integrate the syntax, semantics, and editor services of a language under test into the LPTL for writing test inputs. This paper describes the design of an LPTL and the tool support provided for it, shows use cases using examples, and describes our implementation in the form of the Spoofax testing language.