publications: - title: "The Vienna Definition Language" author: - name: "Peter Wegner" link: "http://en.wikipedia.org/wiki/Peter_Wegner" year: "1972" doi: "http://dl.acm.org/citation.cfm?id=1074902" links: doi: "http://dl.acm.org/citation.cfm?id=1074902" researchr: "https://researchr.org/publication/Wegner72" cites: 0 citedby: 0 journal: "ACM Computing Surveys" volume: "4" number: "1" pages: "5-63" kind: "article" key: "Wegner72" - title: "On Testing Non-Testable Programs" author: - name: "Elaine J. Weyuker" link: "https://researchr.org/alias/elaine-j.-weyuker" year: "1982" abstract: "A frequently invoked assumption in program testing is that there is an oracle (i.e. the tester or an external mechanism can accurately decide whether or not the output produced by a program is correct). A program is non-testable if either an oracle does not exist or the tester must expend some extraordinary amount of time to determine whether or not the output is correct. The reasonableness of the oracle assumption is examined and the conclusion is reached that in many cases this is not a realistic assumption. The consequences of assuming the availability of an oracle are examined and alternatives investigated. " tags: - "testing" - "programming" researchr: "https://researchr.org/publication/Weyuker82" cites: 0 citedby: 0 journal: "Comput. J." volume: "25" number: "4" pages: "465-470" kind: "article" key: "Weyuker82" - title: "TGGS: A Flexible System for Generating Efficient Test Case Generators" author: - name: "Guilmette, Ronald F" link: "https://researchr.org/alias/guilmette%2C-ronald-f" year: "1995" abstract: "This paper describes the Test Generator Generator System, TGGS, Version 2.0. TGGS is a simple yet flexible system for creating highly efficient automated generator programs. These generator programs may themselves be used to generate arbitrary quantities of randomized test cases for a variety of programs, most notably compilers. TGGS employs a specification language, TGGS/SL, in which the user may express both the syntactic and semantic constraints of the input language for the program to be tested. The TGGS/SL language is described in detail. Descriptions of the TGGS/SL compiler, GTG, and the associated TGGS/SL run-time system are provided, together with application examples." researchr: "https://researchr.org/publication/guilmette1995tggs" cites: 0 citedby: 0 kind: "article" key: "guilmette1995tggs" - title: "Attribute Grammar Evolution" author: - name: "Marina de la Cruz" link: "https://researchr.org/alias/marina-de-la-cruz" - name: "Alfonso Ortega de la Puente" link: "https://researchr.org/alias/alfonso-ortega-de-la-puente" - name: "Manuel Alfonseca" link: "https://researchr.org/alias/manuel-alfonseca" year: "2005" doi: "http://dx.doi.org/10.1007/11499305_19" links: doi: "http://dx.doi.org/10.1007/11499305_19" tags: - "attribute grammars" - "grammar" researchr: "https://researchr.org/publication/CruzPA05" cites: 0 citedby: 0 pages: "182-191" booktitle: "Artificial Intelligence and Knowledge Engineering Applications: A Bioinspired Approach: First International Work-Conference on the Interplay Between Natural and Artificial Computation, IWINAC 2005, Las Palmas, Canary Islands, Spain, June 15-18, 2005, Proc" editor: - name: "José Mira" link: "https://researchr.org/alias/jos%C3%A9-mira" - name: "José R. Álvarez" link: "https://researchr.org/alias/jos%C3%A9-r.-%C3%A1lvarez" volume: "3562" series: "Lecture Notes in Computer Science" publisher: "Springer" isbn: "3-540-26319-5" kind: "inproceedings" key: "CruzPA05" - title: "Controllable Combinatorial Coverage in Grammar-Based Testing" author: - name: "Ralf Lämmel" link: "http://www.uni-koblenz.de/~laemmel/Site/Home.html" - name: "Wolfram Schulte" link: "http://research.microsoft.com/en-us/people/schulte/" year: "2006" doi: "http://dx.doi.org/10.1007/11754008_2" abstract: "Given a grammar (or other sorts of meta-data), one can trivially derive combinatorially exhaustive test-data sets up to a specified depth. Without further efforts, such test-data sets would be huge at the least and explosive most of the time. Fortunately, scenarios of grammar-based testing tend to admit non-explosive approximations of naive combinatorial coverage. In this paper, we describe the notion of controllable combinatorial coverage and a corresponding algorithm for test-data generation. The approach is based on a suite of control mechanisms to be used for the characterization of test-data sets as well-defined and understandable approximations of full combinatorial coverage. The approach has been implemented in the C#-based test-data generator Geno, which has been successfully used in projects that required differential testing, stress testing and conformance testing of grammar-driven functionality. " links: doi: "http://dx.doi.org/10.1007/11754008_2" tags: - "rule-based" - "test coverage" - "meta-model" - "testing" - "data-flow" - "C++" - "Meta-Environment" - "coverage" - "systematic-approach" - "meta-objects" - "grammar" researchr: "https://researchr.org/publication/LammelS06" cites: 0 citedby: 0 pages: "19-38" booktitle: "Testing of Communicating Systems, 18th IFIP TC6/WG6.1 International Conference, TestCom 2006, New York, NY, USA, May 16-18, 2006, Proceedings" editor: - name: "M. Ümit Uyar" link: "https://researchr.org/alias/m.-%C3%BCmit-uyar" - name: "Ali Y. Duale" link: "https://researchr.org/alias/ali-y.-duale" - name: "Mariusz A. Fecko" link: "https://researchr.org/alias/mariusz-a.-fecko" volume: "3964" series: "Lecture Notes in Computer Science" publisher: "Springer" isbn: "3-540-34184-6" kind: "inproceedings" key: "LammelS06" - title: "An Introduction to Grammar Convergence" author: - name: "Ralf Lämmel" link: "http://www.uni-koblenz.de/~laemmel/Site/Home.html" - name: "Vadim Zaytsev" link: "http://grammarware.net" year: "2009" doi: "http://dx.doi.org/10.1007/978-3-642-00255-7_17" abstract: "Grammar convergence is a lightweight verification method for establishing and maintaining the correspondence between grammar knowledge ingrained in all kinds of software artifacts, e.g., object models, XML schemas, parser descriptions, or language documents. The central idea is to extract grammars from diverse software artifacts, and to transform the grammars until they become syntactically identical. The present paper introduces and illustrates the basics of grammar convergence. " links: doi: "http://dx.doi.org/10.1007/978-3-642-00255-7_17" "authors' website": "http://tr.im/grcon" tags: - "model-to-model transformation" - "grammar transformation" - "meta-model" - "XML" - "modeling language" - "XML Schema" - "transformation language" - "language modeling" - "grammar convergence" - "source-to-source" - "object-role modeling" - "model transformation" - "Meta-Environment" - "parsing" - "meta-objects" - "transformation" - "grammar" researchr: "https://researchr.org/publication/Convergence2009" cites: 0 citedby: 0 pages: "246-260" booktitle: "Integrated Formal Methods, 7th International Conference, IFM 2009, Düsseldorf, Germany, February 16-19, 2009. Proceedings" editor: - name: "Michael Leuschel" link: "http://www.stups.uni-duesseldorf.de/~leuschel" - name: "Heike Wehrheim" link: "https://researchr.org/alias/heike-wehrheim" volume: "5423" series: "Lecture Notes in Computer Science" publisher: "Springer" isbn: "978-3-642-00254-0" kind: "inproceedings" key: "Convergence2009" - title: "An Approach to Symbolic Test Generation" author: - name: "Vlad Rusu" link: "https://researchr.org/profile/vladrusu/publications" - name: "Lydie du Bousquet" link: "https://researchr.org/alias/lydie-du-bousquet" - name: "Thierry Jéron" link: "https://researchr.org/alias/thierry-j%C3%A9ron" year: "2000" doi: "http://link.springer.de/link/service/series/0558/bibs/1945/19450338.htm" abstract: "Test generation is a program-synthesis problem: starting from the formal specification of a system under test, and from a test purpose describing a set of behaviours to be tested, compute a reactive program that observes an implementation of the system to detect non-conformant behaviour, while trying to control it towards satisfying the test purpose. In this paper we describe an approach for generating symbolic test cases, in the form of input-output automata with variables and parameters. The pdf file is available in the column located to the left of this abstract." links: doi: "http://link.springer.de/link/service/series/0558/bibs/1945/19450338.htm" technicalreport: "https://researchr.org/publication/preprint-RusuBJ00" tags: - "control systems" - "reactive programming" - "testing" - "programming" - "systematic-approach" researchr: "https://researchr.org/publication/RusuBJ00" cites: 0 citedby: 0 pages: "338-357" booktitle: "Integrated Formal Methods, Second International Conference, IFM 2000, Dagstuhl Castle, Germany, November 1-3, 2000, Proceedings" editor: - name: "Wolfgang Grieskamp" link: "https://researchr.org/alias/wolfgang-grieskamp" - name: "Thomas Santen" link: "https://researchr.org/alias/thomas-santen" - name: "Bill Stoddart" link: "https://researchr.org/alias/bill-stoddart" volume: "1945" series: "Lecture Notes in Computer Science" publisher: "Springer" isbn: "3-540-41196-8" kind: "inproceedings" key: "RusuBJ00" - title: "Automated testing of refactoring engines" author: - name: "Brett Daniel" link: "https://researchr.org/alias/brett-daniel" - name: "Danny Dig" link: "https://researchr.org/alias/danny-dig" - name: "Kely Garcia" link: "https://researchr.org/alias/kely-garcia" - name: "Darko Marinov" link: "https://researchr.org/alias/darko-marinov" year: "2007" doi: "http://doi.acm.org/10.1145/1287624.1287651" abstract: "Refactorings are behavior-preserving program transformations that improve the design of a program. Refactoring engines are tools that automate the application of refactorings: first the user chooses a refactoring to apply, then the engine checks if the transformation is safe, and if so, transforms the program. Refactoring engines are a key component of modern IDEs, and programmers rely on them to perform refactorings. A bug in the refactoring engine can have severe consequences as it can erroneously change large bodies of source code. We present a technique for automated testing of refactoring engines. Test inputs for refactoring engines are programs. The core of our technique is a framework for iterative generation of structurally complex test inputs. We instantiate the framework to generate abstract syntax trees that represent Java programs. We also create several kinds of oracles to automatically check that the refactoring engine transformed the generated program correctly. We have applied our technique to testing Eclipse and NetBeans, two popular open-source IDEs for Java, and we have exposed 21 new bugs in Eclipse and 24 new bugs in NetBeans." links: doi: "http://doi.acm.org/10.1145/1287624.1287651" tags: - "model-to-model transformation" - "Java" - "Eclipse" - "meta programming" - "application framework" - "model checking" - "meta-model" - "abstract syntax" - "model refactoring" - "refactoring" - "testing model transformations" - "testing" - "source-to-source" - "code generation" - "metatheory" - "model transformation" - "Meta-Environment" - "design" - "open-source" - "transformation" - "program transformation" researchr: "https://researchr.org/publication/DanielDGM07" cites: 22 citedby: 0 pages: "185-194" booktitle: "Proceedings of the 6th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering, 2007, Dubrovnik, Croatia, September 3-7, 2007" editor: - name: "Ivica Crnkovic" link: "https://researchr.org/alias/ivica-crnkovic" - name: "Antonia Bertolino" link: "https://researchr.org/alias/antonia-bertolino" publisher: "ACM" isbn: "978-1-59593-811-4" kind: "inproceedings" key: "DanielDGM07" - title: "WebDSL: A Case Study in Domain-Specific Language Engineering" author: - name: "Eelco Visser" link: "http://eelcovisser.org" year: "2007" doi: "http://dx.doi.org/10.1007/978-3-540-88643-3_7" abstract: " The goal of domain-specific languages (DSLs) is to increase the productivity of software engineers by abstracting from low-level boil- erplate code. Introduction of DSLs in the software development process requires a smooth workflow for the production of DSLs themselves. This requires technology for designing and implementing DSLs, but also a methodology for using that technology. That is, a collection of guidelines, design patterns, and reusable DSL components that show developers how to tackle common language design and implementation issues. This paper presents a case study in domain-specific language engineering. It reports on a pro ject in which the author designed and built WebDSL, a DSL for web applications with a rich data model, using several DSLs for DSL engineering: SDF for syntax definition and Stratego/XT for code gener- ation. The paper follows the stages in the development of the DSL. The contributions of the paper are three-fold. (1) A tutorial in the application of the specific SDF and Stratego/XT technology for building DSLs. (2) A description of an incremental DSL development process. (3) A domain- specific language for web-applications with rich data models. The paper concludes with a survey of related approaches. " links: doi: "http://dx.doi.org/10.1007/978-3-540-88643-3_7" "technical report (pdf)": "http://swerl.tudelft.nl/twiki/pub/Main/TechnicalReports/TUD-SERG-2008-023.pdf" "webdsl": "http://webdsl.org" "stratego/xt": "http://strategoxt.org" tags: - "WebDSL" - "reusable components" - "DSL engineering" - "web application development" - "data-flow language" - "pattern language" - "case study" - "software components" - "SDF" - "meta-model" - "abstract syntax" - "modeling language" - "Stratego/XT" - "language engineering" - "software language engineering" - "language modeling" - "software component" - "web engineering" - "language design" - "reuse" - "model-driven development" - "data-flow" - "survey" - "software engineering" - "model-driven engineering" - "web applications" - "DSL" - "Meta-Environment" - "incremental" - "design" - "process modeling" - "systematic-approach" - "ASF+SDF" - "language" - "Stratego" - "domain-specific language" researchr: "https://researchr.org/publication/Visser07" cites: 0 citedby: 9 pages: "291-373" booktitle: "Generative and Transformational Techniques in Software Engineering II, International Summer School, GTTSE 2007" editor: - name: "Ralf Lämmel" link: "http://www.uni-koblenz.de/~laemmel/Site/Home.html" - name: "Joost Visser" link: "http://www.di.uminho.pt/~joost.visser/" - name: "João Saraiva" link: "http://di.uminho.pt/~jas" volume: "5235" series: "Lecture Notes in Computer Science" address: "Braga, Portugal" publisher: "Springer" isbn: "978-3-540-88642-6" kind: "inproceedings" key: "Visser07" - title: "Grammar-based test generation with YouGen" author: - name: "Daniel Hoffman" link: "https://researchr.org/alias/daniel-hoffman" - name: "David Ly-Gagnon" link: "https://researchr.org/alias/david-ly-gagnon" - name: "Paul A. Strooper" link: "https://researchr.org/alias/paul-a.-strooper" - name: "Hong-Yi Wang" link: "https://researchr.org/alias/hong-yi-wang" year: "2011" doi: "http://dx.doi.org/10.1002/spe.1017" links: doi: "http://dx.doi.org/10.1002/spe.1017" dblp: "http://dblp.uni-trier.de/rec/bibtex/journals/spe/HoffmanLSW11" tags: - "rule-based" - "testing" - "grammar" researchr: "https://researchr.org/publication/HoffmanLSW11" cites: 0 citedby: 0 journal: "Software: Practice and Experience" volume: "41" number: "4" pages: "427-447" kind: "article" key: "HoffmanLSW11" - title: "Testing an optimising compiler by generating random lambda terms" author: - name: "Michal H. Palka" link: "http://www.chalmers.se/cse/EN/people/palka-michal" - name: "Koen Claessen" link: "http://www.cse.chalmers.se/~koen/" - name: "Russo, Alejandro" link: "https://researchr.org/alias/russo%2C-alejandro" - name: "John Hughes" link: "http://www.cse.chalmers.se/~rjmh" year: "2011" doi: "http://dl.acm.org/citation.cfm?id=1982615" abstract: "This paper considers random testing of a compiler, using randomly generated programs as inputs, and comparing their behaviour with and without optimisation. Since the generated programs must compile, then we need to take into account syntax, scope rules, and type checking during our random generation. Doing so, while attaining a good distribution of test data, proves surprisingly subtle; the main contribution of this paper is a workable solution to this problem. We used it to generate typed functions on lists, which we compiled using the Glasgow Haskell compiler, a mature production quality Haskell compiler. After around 20,000 tests we triggered an optimiser failure, and automatically simplified it to a program with just a few constructs." links: doi: "http://dl.acm.org/citation.cfm?id=1982615" tags: - "rule-based" - "testing" - "Haskell" - "data-flow programming" - "data-flow" - "rules" - "type checking" - "random testing" - "compiler" researchr: "https://researchr.org/publication/PalkaAST2011" cites: 0 citedby: 0 pages: "91-97" booktitle: "Proceedings of the 6th International Workshop on Automation of Software Test" series: "AST '11" address: "New York, NY, USA" publisher: "ACM" isbn: "978-1-4503-0592-1" kind: "inproceedings" key: "PalkaAST2011" - title: "Generating Test Data with Enhanced Context-Free Grammars" author: - name: "Peter M. Maurer" link: "https://researchr.org/alias/peter-m.-maurer" year: "1990" doi: "http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=56422&tag=1" abstract: "The use of context-free grammars to improve functional testing of very-large-scale integrated circuits is described. It is shown that enhanced context-free grammars are effective tools for generating test data. The discussion covers preliminary considerations, the first tests, generating systematic tests, and testing subroutines. The author's experience using context-free grammars to generate tests for VLSI circuit simulators indicates that they are remarkably effective tools that virtually anyone can use to debug virtually any program." links: doi: "http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=56422&tag=1" tags: - "functional programming" - "testing" - "data-flow programming" - "debugging" - "context-aware" - "systematic-approach" - "grammar" researchr: "https://researchr.org/publication/Maurer90%3A1" cites: 0 citedby: 0 journal: "IEEE Software" volume: "7" number: "4" pages: "50-55" kind: "article" key: "Maurer90:1" - title: "QuickCheck: a lightweight tool for random testing of Haskell programs" author: - name: "Koen Claessen" link: "http://www.cse.chalmers.se/~koen/" - name: "John Hughes" link: "http://www.cse.chalmers.se/~rjmh" year: "2000" doi: "http://doi.acm.org/10.1145/351240.351266" abstract: "Quick Check is a tool which aids the Haskell programmer in formulating and testing properties of programs. Properties are described as Haskell functions, and can be automatically tested on random input, but it is also possible to define custom test data generators. We present a number of case studies, in which the tool was successfully used, and also point out some pitfalls to avoid. Random testing is especially suitable for functional programs because properties can be stated at a fine grain. When a function is built from separately tested components, then random testing suffices to obtain good coverage of the definition under test." links: doi: "http://doi.acm.org/10.1145/351240.351266" tags: - "test coverage" - "case study" - "functional programming" - "testing" - "Haskell" - "data-flow programming" - "data-flow" - "random testing" - "programming" - "coverage" - "QuickCheck" researchr: "https://researchr.org/publication/ClaessenH00" cites: 19 citedby: 0 pages: "268-279" booktitle: "ICFP" kind: "inproceedings" key: "ClaessenH00" - title: "A sentence generator for testing parsers" author: - name: "Purdom, Paul" link: "https://researchr.org/alias/purdom%2C-paul" year: "1972" doi: "http://www.springerlink.com/content/t0773j3737245nj3/export-citation/" abstract: "A fast algorithm is given to produce a small set of short sentences from a context free grammar such that each production of the grammar is used at least once. The sentences are useful for testing parsing programs and for debugging grammars (finding errors in a grammar which causes it to specify some language other than the one intended). Some experimental results from using the sentences to test some automatically generated simpleLR(1) parsers are also given." note: "10.1007/BF01932308" links: doi: "http://www.springerlink.com/content/t0773j3737245nj3/export-citation/" "url": "http://dx.doi.org/10.1007/BF01932308" tags: - "parsing algorithm" - "programming languages" - "testing" - "debugging" - "context-aware" - "parsing" - "grammar" researchr: "https://researchr.org/publication/springerlink%3A10.1007-BF01932308" cites: 0 citedby: 0 journal: "BIT Numerical Mathematics" volume: "12" pages: "366-375" kind: "article" key: "springerlink:10.1007-BF01932308" - title: "Grammatical evolution" author: - name: "Michael O Neill" link: "https://researchr.org/alias/michael-o-neill" - name: "Conor Ryan" link: "https://researchr.org/alias/conor-ryan" year: "2001" doi: "http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=942529&tag=1" abstract: "We present grammatical evolution, an evolutionary algorithm that can evolve complete programs in an arbitrary language using a variable-length binary string. The binary genome determines which production rules in a Backus–Naur form grammar definition are used in a genotype-to-phenotype mapping process to a program. We demonstrate how expressions and programs of arbitrary complexity may be evolved and compare its performance to genetic programming." links: doi: "http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=942529&tag=1" tags: - "programming languages" - "rule-based" - "completeness" - "rules" - "programming" - "genetic programming" - "grammar" researchr: "https://researchr.org/publication/ONeillR01" cites: 0 citedby: 0 journal: "IEEE Trans. Evolutionary Computation" volume: "5" number: "4" pages: "349-358" kind: "article" key: "ONeillR01" - title: "Formal verification of a realistic compiler" author: - name: "Xavier Leroy" link: "http://pauillac.inria.fr/~xleroy/" year: "2009" doi: "http://doi.acm.org/10.1145/1538788.1538814" abstract: "This paper reports on the development and formal verification (proof of semantic preservation) of CompCert, a compiler from Clight (a large subset of the C programming language) to PowerPC assembly code, using the Coq proof assistant both for programming the compiler and for proving its correctness. Such a verified compiler is useful in the context of critical software and its formal verification: the verification of the compiler guarantees that the safety properties proved on the source code hold for the executable compiled code as well." links: doi: "http://doi.acm.org/10.1145/1538788.1538814" tags: - "programming languages" - "semantics" - "semantic preservation" - "proof assistant" - "CompCert" - "program verification" - "formal semantics" - "type soundness" - "source-to-source" - "C++" - "compiler" - "programming" - "context-aware" - "open-source" researchr: "https://researchr.org/publication/Leroy09" cites: 0 citedby: 0 journal: "Communications of the ACM" volume: "52" number: "7" pages: "107-115" kind: "article" key: "Leroy09" - title: "Combined static and dynamic automated test generation" author: - name: "Sai Zhang" link: "https://researchr.org/alias/sai-zhang" - name: "David Saff" link: "https://researchr.org/alias/david-saff" - name: "Yingyi Bu" link: "https://researchr.org/alias/yingyi-bu" - name: "Michael D. Ernst" link: "https://researchr.org/alias/michael-d.-ernst" year: "2011" doi: "http://doi.acm.org/10.1145/2001420.2001463" abstract: "In an object-oriented program, a unit test often consists of a sequence of method calls that create and mutate objects, then use them as arguments to a method under test. It is challenging to automatically generate sequences that are legal and behaviorally-diverse, that is, reaching as many different program states as possible. This paper proposes a combined static and dynamic automated test generation approach to address these problems, for code without a formal specification. Our approach first uses dynamic analysis to infer a call sequence model from a sample execution, then uses static analysis to identify method dependence relations based on the fields they may read or write. Finally, both the dynamically-inferred model (which tends to be accurate but incomplete) and the statically-identified dependence information (which tends to be conservative) guide a random test generator to create legal and behaviorally-diverse tests. Our Palus tool implements this testing approach. We compared its effectiveness with a pure random approach, a dynamic-random approach (without a static phase), and a static-random approach (without a dynamic phase) on several popular open-source Java programs. Tests generated by Palus achieved higher structural coverage and found more bugs. Palus is also internally used in Google. It has found 22 previously unknown bugs in four well-tested Google products." links: doi: "http://doi.acm.org/10.1145/2001420.2001463" dblp: "http://dblp.uni-trier.de/rec/bibtex/conf/issta/ZhangSBE11" tags: - "object-oriented programming" - "rule-based" - "Java" - "program analysis" - "test coverage" - "meta programming" - "meta-model" - "points-to analysis" - "testing" - "analysis" - "static analysis" - "source-to-source" - "code generation" - "object-role modeling" - "random testing" - "information models" - "subject-oriented programming" - "Meta-Environment" - "Google" - "coverage" - "systematic-approach" - "feature-oriented programming" - "open-source" - "meta-objects" researchr: "https://researchr.org/publication/ZhangSBE11" cites: 0 citedby: 0 pages: "353-363" booktitle: "Proceedings of the 20th International Symposium on Software Testing and Analysis, ISSTA 2011, Toronto, ON, Canada, July 17-21, 2011" editor: - name: "Matthew B. Dwyer" link: "https://researchr.org/alias/matthew-b.-dwyer" - name: "Frank Tip" link: "http://domino.research.ibm.com/comm/research_people.nsf/pages/tip.index.html" publisher: "ACM" isbn: "978-1-4503-0562-4" kind: "inproceedings" key: "ZhangSBE11" - title: "Using Attributed Grammars to Test Designs and Implementations" author: - name: "A. G. Duncan" link: "https://researchr.org/alias/a.-g.-duncan" - name: "J. S. Hutchison" link: "https://researchr.org/alias/j.-s.-hutchison" year: "1981" doi: "http://dl.acm.org/citation.cfm?id=802529" abstract: "We present a method for generating test cases that can be used throughout the entire life cycle of a program. This method uses attributed translation grammars to generate both inputs and outputs, which can then be used either as is, in order to test the specifications, or in conjunction with automatic test drivers to test an implementation against the specifications. The grammar can generate test cases either randomly or systematically. The attributes are used to guide the generation process, thereby avoiding the generation of many superfluous test cases. The grammar itself not only drives the generation of test cases but also serves as a concise documentation of the test plan. In the paper, we describe the test case generator, show how it works in typical examples, compare it with related techniques, and discuss how it can be used in conjunction with various testing heuristics." links: doi: "http://dl.acm.org/citation.cfm?id=802529" tags: - "attribute grammars" - "translation" - "testing" - "random testing" - "systematic-approach" - "grammar" researchr: "https://researchr.org/publication/DuncanH81" cites: 0 citedby: 0 pages: "170-178" booktitle: "ICSE" kind: "inproceedings" key: "DuncanH81" - title: "HDD: hierarchical Delta Debugging" author: - name: "Ghassan Misherghi" link: "https://researchr.org/alias/ghassan-misherghi" - name: "Zhendong Su" link: "https://researchr.org/alias/zhendong-su" year: "2006" doi: "http://doi.acm.org/10.1145/1134307" links: doi: "http://doi.acm.org/10.1145/1134307" tags: - "debugging" researchr: "https://researchr.org/publication/Misherghi06" cites: 0 citedby: 0 pages: "142-151" booktitle: "28th International Conference on Software Engineering (ICSE 2006), Shanghai, China, May 20-28, 2006" editor: - name: "Leon J. Osterweil" link: "https://researchr.org/alias/leon-j.-osterweil" - name: "H. Dieter Rombach" link: "https://researchr.org/alias/h.-dieter-rombach" - name: "Mary Lou Soffa" link: "https://researchr.org/alias/mary-lou-soffa" publisher: "ACM" isbn: "1-59593-375-1" kind: "inproceedings" key: "Misherghi06" - title: "Volatiles are miscompiled, and what to do about it" author: - name: "Eric Eide" link: "https://researchr.org/alias/eric-eide" - name: "John Regehr" link: "https://researchr.org/alias/john-regehr" year: "2008" doi: "http://doi.acm.org/10.1145/1450058.1450093" abstract: "C's volatile qualifier is intended to provide a reliable link between operations at the source-code level and operations at the memory-system level. We tested thirteen production-quality C compilers and, for each, found situations in which the compiler generated incorrect code for accessing volatile variables. This result is disturbing because it implies that embedded software and operating systems---both typically coded in C, both being bases for many mission-critical and safety-critical applications, and both relying on the correct translation of volatiles---may be being miscompiled. Our contribution is centered on a novel technique for finding volatile bugs and a novel technique for working around them. First, we present access summary testing: an efficient, practical, and automatic way to detect code-generation errors related to the volatile qualifier. We have found a number of compiler bugs by performing access summary testing on randomly generated C programs. Some of these bugs have been confirmed and fixed by compiler developers. Second, we present and evaluate a workaround for the compiler defects we discovered. In 96% of the cases in which one of our randomly generated programs is miscompiled, we can cause the faulty C compiler to produce correctly behaving code by applying a straightforward source-level transformation to the test program." links: doi: "http://doi.acm.org/10.1145/1450058.1450093" tags: - "model-to-model transformation" - "translation" - "operating system" - "embedded software" - "testing" - "source-to-source" - "C++" - "code generation" - "transformation system" - "random testing" - "compiler" - "model transformation" - "open-source" - "transformation" - "program transformation" researchr: "https://researchr.org/publication/EideR08" cites: 0 citedby: 0 pages: "255-264" booktitle: "Proceedings of the 8th ACM & IEEE International conference on Embedded software, EMSOFT 2008, Atlanta, GA, USA, October 19-24, 2008" editor: - name: "Luca de Alfaro" link: "https://researchr.org/alias/luca-de-alfaro" - name: "Jens Palsberg" link: "https://researchr.org/alias/jens-palsberg" publisher: "ACM" isbn: "978-1-60558-468-3" kind: "inproceedings" key: "EideR08" - title: "Program Transformation with Scoped Dynamic Rewrite Rules" author: - name: "Martin Bravenboer" link: "http://martin.bravenboer.name/" - name: "Arthur van Dam" link: "https://researchr.org/profile/arthurvandam/publications" - name: "Karina Olmos" link: "https://researchr.org/profile/karinaolmos/publications" - name: "Eelco Visser" link: "http://eelcovisser.org" year: "2006" doi: "https://content.iospress.com/articles/fundamenta-informaticae/fi69-1-2-06" abstract: "The applicability of term rewriting to program transformation is limited by the lack of control over rule application and by the context-free nature of rewrite rules. The first problem is addressed by languages supporting user-definable rewriting strategies. The second problem is addressed by the extension of rewriting strategies with scoped dynamic rewrite rules. Dynamic rules are defined at run-time and can access variables available from their definition context. Rules defined within a rule scope are automatically retracted at the end of that scope. In this paper, we explore the design space of dynamic rules, and their application to transformation problems. The technique is formally defined by extending the operational semantics underlying the program transformation language Stratego, and illustrated by means of several program transformations in Stratego, including constant propagation, bound variable renaming, dead code elimination, function inlining, and function specialization. " links: doi: "https://content.iospress.com/articles/fundamenta-informaticae/fi69-1-2-06" "technical report": "http://www.cs.uu.nl/research/techreps/UU-CS-2005-005.html" tags: - "programming languages" - "semantics" - "rule-based" - "formal semantics" - "graph transformation" - "dynamic rewrite rules" - "Stratego/XT" - "transformation language" - "term rewriting" - "language design" - "graph-rewriting" - "rules" - "operational semantics" - "context-aware" - "access control" - "rewriting" - "design" - "role-based access control" - "rewriting strategies" - "transformation" - "Stratego" - "program transformation" researchr: "https://researchr.org/publication/BravenboerDOV06" cites: 0 citedby: 4 journal: "Fundamenta Informaticae" volume: "69" number: "1-2" pages: "123-178" kind: "article" key: "BravenboerDOV06" - title: "Program Checkers for Probability Generation" author: - name: "Sampath Kannan" link: "https://researchr.org/alias/sampath-kannan" - name: "Andrew Chi-Chih Yao" link: "https://researchr.org/alias/andrew-chi-chih-yao" year: "1991" researchr: "https://researchr.org/publication/KannanY91" cites: 0 citedby: 0 pages: "163-173" booktitle: "Automata, Languages and Programming, 18th International Colloquium, ICALP91, Madrid, Spain, July 8-12, 1991, Proceedings" editor: - name: "Javier Leach Albert" link: "https://researchr.org/alias/javier-leach-albert" - name: "Burkhard Monien" link: "https://researchr.org/alias/burkhard-monien" - name: "Mario Rodríguez-Artalejo" link: "https://researchr.org/alias/mario-rodr%C3%ADguez-artalejo" volume: "510" series: "Lecture Notes in Computer Science" publisher: "Springer" isbn: "3-540-54233-7" kind: "inproceedings" key: "KannanY91" - title: "Finding and understanding bugs in C compilers" author: - name: "Xuejun Yang" link: "https://researchr.org/alias/xuejun-yang" - name: "Yang Chen" link: "https://researchr.org/alias/yang-chen" - name: "Eric Eide" link: "https://researchr.org/alias/eric-eide" - name: "John Regehr" link: "https://researchr.org/alias/john-regehr" year: "2011" doi: "http://doi.acm.org/10.1145/1993498.1993532" abstract: "Compilers should be correct. To improve the quality of C compilers, we created Csmith, a randomized test-case generation tool, and spent three years using it to find compiler bugs. During this period we reported more than 325 previously unknown bugs to compiler developers. Every compiler we tested was found to crash and also to silently generate wrong code when presented with valid input. In this paper we present our compiler-testing tool and the results of our bug-hunting study. Our first contribution is to advance the state of the art in compiler testing. Unlike previous tools, Csmith generates programs that cover a large subset of C while avoiding the undefined and unspecified behaviors that would destroy its ability to automatically find wrong-code bugs. Our second contribution is a collection of qualitative and quantitative results about the bugs we have found in open-source C compilers." links: doi: "http://doi.acm.org/10.1145/1993498.1993532" dblp: "http://dblp.uni-trier.de/rec/bibtex/conf/pldi/YangCER11" tags: - "case study" - "testing" - "source-to-source" - "C++" - "code generation" - "random testing" - "compiler" - "open-source" researchr: "https://researchr.org/publication/YangCER11" cites: 0 citedby: 0 pages: "283-294" booktitle: "Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2011, San Jose, CA, USA, June 4-8, 2011" editor: - name: "Mary W. Hall" link: "https://researchr.org/alias/mary-w.-hall" - name: "David A. Padua" link: "https://researchr.org/alias/david-a.-padua" publisher: "ACM" isbn: "978-1-4503-0663-8" kind: "inproceedings" key: "YangCER11" - title: "Test case characterisation by regular path expressions" author: - name: "Ralf Lämmel" link: "http://www.uni-koblenz.de/~laemmel/Site/Home.html" - name: "Jörg Harm" link: "https://researchr.org/alias/j%C3%B6rg-harm" year: "2001" month: "august" doi: "http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.24.4315" abstract: "A signature-parametric (say generic) framework for test case characterisation, and definition of test set coverage criteria is developed. The signature might correspond to a programming language syntax, the format of a data structure, a computational or semantical structure, e.g., for derivations, proof trees, or control-flow graphs. Test set characterisation is based on regular expressions describing paths for terms over the signature at hand. Necessary and convenient properties for test set coverage criteria can be conceived in the framework." links: doi: "http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.24.4315" tags: - "programming languages" - "rule-based" - "syntax definition" - "test coverage" - "data-flow language" - "generic programming" - "testing" - "data-flow programming" - "data-flow" - "graph-rewriting" - "coverage" researchr: "https://researchr.org/publication/LH01-FATES" cites: 0 citedby: 0 pages: "109-124" booktitle: "Proc.\\ Formal Approaches to Testing of Software (FATES'01)" series: "Notes Series NS-01-4" publisher: "BRICS" kind: "inproceedings" key: "LH01-FATES" - title: "Directed random generation of sentences" author: - name: "Joyce Friedman" link: "https://researchr.org/alias/joyce-friedman" year: "1969" doi: "http://doi.acm.org/10.1145/362835.362846" links: doi: "http://doi.acm.org/10.1145/362835.362846" researchr: "https://researchr.org/publication/Friedman69" cites: 0 citedby: 0 journal: "Communications of the ACM" volume: "12" number: "1" pages: "40-46" kind: "article" key: "Friedman69" - title: "Simplifying failure-inducing input" author: - name: "Ralf Hildebrandt" link: "https://researchr.org/alias/ralf-hildebrandt" - name: "Andreas Zeller" link: "https://researchr.org/alias/andreas-zeller" year: "2000" doi: "http://doi.acm.org/10.1145/347324.348938" links: doi: "http://doi.acm.org/10.1145/347324.348938" researchr: "https://researchr.org/publication/HildebrandtZ00" cites: 0 citedby: 0 pages: "135-145" booktitle: "ISSTA" kind: "inproceedings" key: "HildebrandtZ00" - title: "A practical guide for using statistical tests to assess randomized algorithms in software engineering" author: - name: "Andrea Arcuri" link: "https://researchr.org/alias/andrea-arcuri" - name: "Lionel C. Briand" link: "https://researchr.org/alias/lionel-c.-briand" year: "2011" doi: "http://doi.acm.org/10.1145/1985793.1985795" links: doi: "http://doi.acm.org/10.1145/1985793.1985795" dblp: "http://dblp.uni-trier.de/rec/bibtex/conf/icse/ArcuriB11" tags: - "testing" - "software engineering" - "C++" - "random testing" researchr: "https://researchr.org/publication/ArcuriB11-0" cites: 0 citedby: 0 pages: "1-10" booktitle: "Proceedings of the 33rd International Conference on Software Engineering, ICSE 2011, Waikiki, Honolulu , HI, USA, May 21-28, 2011" editor: - name: "Richard N. Taylor" link: "https://researchr.org/alias/richard-n.-taylor" - name: "Harald Gall" link: "https://researchr.org/alias/harald-gall" - name: "Nenad Medvidovic" link: "https://researchr.org/alias/nenad-medvidovic" publisher: "ACM" isbn: "978-1-4503-0445-0" kind: "inproceedings" key: "ArcuriB11-0" - title: "When and how to develop domain-specific languages" author: - name: "Marjan Mernik" link: "http://lpm.uni-mb.si/mernik/" - name: "Jan Heering" link: "http://homepages.cwi.nl/~jan/" - name: "Anthony M. Sloane" link: "http://www.comp.mq.edu.au/~asloane" year: "2005" doi: "http://doi.acm.org/10.1145/1118890.1118892" abstract: "Domain-specific languages (DSLs) are languages tailored to a specific application domain. They offer substantial gains in expressiveness and ease of use compared with general-purpose programming languages in their domain of application. DSL development is hard, requiring both domain knowledge and language development expertise. Few people have both. Not surprisingly, the decision to develop a DSL is often postponed indefinitely, if considered at all, and most DSLs never get beyond the application library stage.Although many articles have been written on the development of particular DSLs, there is very limited literature on DSL development methodologies and many questions remain regarding when and how to develop a DSL. To aid the DSL developer, we identify patterns in the decision, analysis, design, and implementation phases of DSL development. Our patterns improve and extend earlier work on DSL design patterns. We also discuss domain analysis tools and language development systems that may help to speed up DSL development. Finally, we present a number of open problems. " links: doi: "http://doi.acm.org/10.1145/1118890.1118892" tags: - "programming languages" - "DSL engineering" - "program analysis" - "pattern language" - "DSL implementation approach" - "dsl examples" - "language engineering" - "points-to analysis" - "domain analysis" - "analysis" - "language design" - "source-to-source" - "survey" - "DSL" - "programming" - "design" - "systematic-approach" - "open-source" - "domain-specific language" researchr: "https://researchr.org/publication/MernikHS05" cites: 154 citedby: 3 journal: "ACM Computing Surveys" volume: "37" number: "4" pages: "316-344" kind: "article" key: "MernikHS05" - title: "Code Generation by Model Transformation" author: - name: "Zef Hemel" link: "http://zef.me" - name: "Lennart C. L. Kats" link: "http://www.lclnet.nl/" - name: "Eelco Visser" link: "http://eelcovisser.org" year: "2008" doi: "http://dx.doi.org/10.1007/978-3-540-69927-9_13" abstract: "The realization of model-driven software development requires effective techniques for implementing code generators. In this paper, we present a case study of code generation by model transformation with Stratego, a high-level transformation language based on the paradigm of rewrite rules with programmable strategies that integrates model-to-model, model-to-code, and code-to-code transformations. The use of concrete object syntax guarantees syntactic correctness of code patterns, and enables the subsequent transformation of generated code. The composability of strategies supports two dimensions of transformation modularity. Vertical modularity is achieved by designing a generator as a pipeline of model-to-model transformations that gradually transforms a high-level input model to an implementation. Horizontal modularity is achieved by supporting the definition of plugins which implement all aspects of a language feature. We discuss the application of these techniques in the implementation of WebDSL, a domain-specific language for dynamic web applications with a rich data model. " links: doi: "http://dx.doi.org/10.1007/978-3-540-69927-9_13" successor: "https://researchr.org/publication/HemelKGV10" "technical report (pdf)": "http://swerl.tudelft.nl/twiki/pub/Main/TechnicalReports/TUD-SERG-2008-012.pdf" tags: - "programming languages" - "model-to-model transformation" - "object-oriented programming" - "WebDSL" - "concrete object syntax" - "rule-based" - "transformation engineering" - "syntax definition" - "meta programming" - "data-flow language" - "pattern language" - "case study" - "graph transformation" - "meta-model" - "modeling language" - "modeling" - "language engineering" - "transformation language" - "software language engineering" - "language modeling" - "web engineering" - "data-flow programming" - "model-driven development" - "data-flow" - "source-to-source" - "graph-rewriting" - "software engineering" - "rules" - "model-driven engineering" - "C++" - "programming paradigms" - "code generation" - "object-role modeling" - "aspect oriented programming" - "model transformation" - "web applications" - "DSL" - "subject-oriented programming" - "Meta-Environment" - "rewriting" - "rewriting strategies" - "feature-oriented programming" - "concrete syntax" - "open-source" - "meta-objects" - "transformation" - "Stratego" - "program transformation" - "domain-specific language" researchr: "https://researchr.org/publication/HemelKV08" cites: 0 citedby: 3 pages: "183-198" booktitle: "Theory and Practice of Model Transformations, First International Conference, ICMT 2008, Zürich, Switzerland, July 1-2, 2008, Proceedings" editor: - name: "Antonio Vallecillo" link: "http://www.lcc.uma.es/~av/" - name: "Jeffrey G. Gray" link: "http://www.gray-area.org/" - name: "Alfonso Pierantonio" link: "http://www.di.univaq.it/alfonso" volume: "5063" series: "Lecture Notes in Computer Science" publisher: "Springer" isbn: "978-3-540-69926-2" kind: "inproceedings" key: "HemelKV08" - title: "Pure and declarative syntax definition: paradise lost and regained" author: - name: "Lennart C. L. Kats" link: "http://www.lclnet.nl/" - name: "Eelco Visser" link: "http://eelcovisser.org" - name: "Guido Wachsmuth" link: "https://www.linkedin.com/in/guidowachsmuth/" year: "2010" doi: "http://doi.acm.org/10.1145/1869459.1869535" abstract: "Syntax definitions are pervasive in modern software systems, and serve as the basis for language processing tools like parsers and compilers. Mainstream parser generators pose restrictions on syntax definitions that follow from their implementation algorithm. They hamper evolution, maintainability, and compositionality of syntax definitions. The pureness and declarativity of syntax definitions is lost. We analyze how these problems arise for different aspects of syntax definitions, discuss their consequences for language engineers, and show how the pure and declarative nature of syntax definitions can be regained." links: doi: "http://doi.acm.org/10.1145/1869459.1869535" dblp: "http://dblp.uni-trier.de/rec/bibtex/conf/oopsla/KatsVW10" "pdf (tech report)": "http://swerl.tudelft.nl/twiki/pub/Main/TechnicalReports/TUD-SERG-2010-019.pdf" tags: - "parsing algorithm" - "syntax definition" - "composition" - "software evolution" - "C++" - "compiler" - "parsing" researchr: "https://researchr.org/publication/KatsVW10" cites: 0 citedby: 1 pages: "918-932" booktitle: "Proceedings of the 25th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2010, October 17-21, 2010, Reno/Tahoe, Nevada, USA" editor: - name: "William R. Cook" link: "http://www.cs.utexas.edu/~wcook/" - name: "Siobhán Clarke" link: "https://www.cs.tcd.ie/Siobhan.Clarke/" - name: "Martin C. Rinard" link: "https://researchr.org/alias/martin-c.-rinard" address: "Reno/Tahoe, Nevada" publisher: "ACM" isbn: "978-1-4503-0203-6" kind: "inproceedings" key: "KatsVW10" - title: "Grammar-based whitebox fuzzing" author: - name: "Patrice Godefroid" link: "https://researchr.org/alias/patrice-godefroid" - name: "Adam Kiezun" link: "https://researchr.org/alias/adam-kiezun" - name: "Michael Y. Levin" link: "https://researchr.org/alias/michael-y.-levin" year: "2008" doi: "http://doi.acm.org/10.1145/1375581.1375607" abstract: "Whitebox fuzzing is a form of automatic dynamic test generation, based on symbolic execution and constraint solving, designed for security testing of large applications. Unfortunately, the current effectiveness of whitebox fuzzing is limited when testing applications with highly-structured inputs, such as compilers and interpreters. These applications process their inputs in stages, such as lexing, parsing and evaluation. Due to the enormous number of control paths in early processing stages, whitebox fuzzing rarely reaches parts of the application beyond those first stages. In this paper, we study how to enhance whitebox fuzzing of complex structured-input applications with a grammar-based specification of their valid inputs. We present a novel dynamic test generation algorithm where symbolic execution directly generates grammar-based constraints whose satisfiability is checked using a custom grammar-based constraint solver. We have implemented this algorithm and evaluated it on a large security-critical application, the JavaScript interpreter of Internet Explorer 7 (IE7). Results of our experiments show that grammar-based whitebox fuzzing explores deeper program paths and avoids dead-ends due to non-parsable inputs. Compared to regular whitebox fuzzing, grammar-based whitebox fuzzing increased coverage of the code generation module of the IE7 JavaScript interpreter from 53% to 81% while using three times fewer tests." links: doi: "http://doi.acm.org/10.1145/1375581.1375607" tags: - "parsing algorithm" - "JavaScript" - "rule-based" - "test coverage" - "interpreter" - "testing" - "constraints" - "code generation" - "security" - "compiler" - "parsing" - "coverage" - "grammar" researchr: "https://researchr.org/publication/GodefroidKL08" cites: 0 citedby: 0 pages: "206-215" booktitle: "Proceedings of the ACM SIGPLAN 2008 Conference on Programming Language Design and Implementation, Tucson, AZ, USA, June 7-13, 2008" editor: - name: "Rajiv Gupta" link: "https://researchr.org/alias/rajiv-gupta" - name: "Saman P. Amarasinghe" link: "https://researchr.org/alias/saman-p.-amarasinghe" publisher: "ACM" isbn: "978-1-59593-860-2" kind: "inproceedings" key: "GodefroidKL08" - title: "Differential Testing for Software" author: - name: "William M. McKeeman" link: "https://researchr.org/alias/william-m.-mckeeman" year: "1998" doi: "http://www.cs.dartmouth.edu/~mckeeman/references/DifferentialTestingForSoftware.pdf" links: doi: "http://www.cs.dartmouth.edu/~mckeeman/references/DifferentialTestingForSoftware.pdf" tags: - "testing" researchr: "https://researchr.org/publication/McKeeman98" cites: 0 citedby: 0 journal: "Digital Technical Journal" volume: "10" number: "1" pages: "100-107" kind: "article" key: "McKeeman98" - title: "Grammar Testing" author: - name: "Ralf Lämmel" link: "http://www.uni-koblenz.de/~laemmel/Site/Home.html" year: "2001" doi: "http://link.springer.de/link/service/series/0558/bibs/2029/20290201.htm" abstract: "Grammar testing is discussed in the context of grammar engineering (i.e., software engineering for grammars). We propose a gen- eralisation of the known rule coverage for grammars, that is, context-dependent branch coverage. We investigate grammar testing, especially coverage analysis, test set generation, and integration of testing and grammar transformations. Grammar recovery is chosen as a subfield of grammar engineering to illustrate the developed concepts. Grammar recovery is concerned with the derivation of a language’s grammar from some available resource such as a semi-formal language reference." links: doi: "http://link.springer.de/link/service/series/0558/bibs/2029/20290201.htm" tags: - "rule-based" - "transformation engineering" - "test coverage" - "language engineering" - "transformation language" - "software language engineering" - "testing" - "analysis" - "software engineering" - "rules" - "context-aware" - "coverage" - "transformation" - "grammar" researchr: "https://researchr.org/publication/Lammel01" cites: 0 citedby: 0 pages: "201-216" booktitle: "Fundamental Approaches to Software Engineering, 4th International Conference, FASE 2001 Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2001 Genova, Italy, April 2-6, 2001, Proceedings" editor: - name: "Heinrich Hußmann" link: "https://researchr.org/alias/heinrich-hu%C3%9Fmann" volume: "2029" series: "Lecture Notes in Computer Science" publisher: "Springer" isbn: "3-540-41863-6" kind: "inproceedings" key: "Lammel01" - title: "Little languages: little maintenance?" author: - name: "Arie van Deursen" link: "http://www.st.ewi.tudelft.nl/~arie/" - name: "Paul Klint" link: "http://homepages.cwi.nl/~paulk/" year: "1998" doi: "http://dx.doi.org/10.1002/(SICI)1096-908X(199803/04)10:2<75::AID-SMR168>3.0.CO;2-5" abstract: "So-called little,or domain-specific languages (DSLs), have the potential to make software main- tenance simpler: domain-experts can directly use the DSL to make required routine modifica- tions. On the negative side, however,more substantial changes may become more difficult: such changes may involve altering the domain-specific language. This will require compiler technol- ogy knowledge,which not every commercial enterprise has easily available. Based on experience taken from industrial practice, we discuss the role of DSLs in software maintenance, the dangers introduced by using them, and techniques for controlling the risks involved." links: doi: "http://dx.doi.org/10.1002/(SICI)1096-908X(199803/04)10:2<75::AID-SMR168>3.0.CO;2-5" tags: - "rule-based" - "little language" - "maintenance" - "compiler" - "DSL" - "domain-specific language" researchr: "https://researchr.org/publication/DeursenK98" cites: 0 citedby: 1 journal: "Journal of Software Maintenance" volume: "10" number: "2" pages: "75-92" kind: "article" key: "DeursenK98" - title: "Automating software tests with partial oracles in integrated environments" author: - name: "René Just" link: "https://researchr.org/alias/ren%C3%A9-just" - name: "Franz Schweiggert" link: "http://www.mathematik.uni-ulm.de/sai/swg/" year: "2010" doi: "10.1145/1808266.1808280" abstract: "The oracle problem is an essential part in current research on automating software tests. Partial oracles seem to be a viable solution, but their suitability for different testing steps and general applicability for various systems remains still to be shown. In the present paper a study is presented in which partial oracles are applied in order to automatically test an image processing application in an integrated environment. Furthermore, the efficiency of these partial oracles is measured by means of Mutation Analysis to determine their adequacy for this purpose." links: "url": "http://doi.acm.org/10.1145/1808266.1808280" tags: - "meta-model" - "Meta-Environment" researchr: "https://researchr.org/publication/Just%3A2010%3AAST%3A1808266.1808280" cites: 0 citedby: 0 pages: "91-94" booktitle: "Proceedings of the 5th Workshop on Automation of Software Test" series: "AST '10" address: "New York, NY, USA" publisher: "ACM" isbn: "978-1-60558-970-1" kind: "inproceedings" key: "Just:2010:AST:1808266.1808280" - title: "Using production grammars in software testing" author: - name: "Emin Gün Sirer" link: "https://researchr.org/alias/emin-g%C3%BCn-sirer" - name: "Brian N. Bershad" link: "https://researchr.org/alias/brian-n.-bershad" year: "1999" doi: "http://doi.acm.org/10.1145/331960.331965" abstract: " Extensible typesafe systems, such as Java, rely critically on a large and complex software base for their overall protection and integrity, and are therefore difficult to test and verify. Traditional testing techniques, such as manual test generation and formal verification, are too time consuming, expensive, and imprecise, or work only on abstract models of the implementation and are too simplistic. Consequently, commercial virtual machines deployed so far have exhibited numerous bugs and security holes.In this paper, we discuss our experience with using production grammars in testing large, complex and safety-critical software systems. Specifically, we describe lava, a domain specific language we have developed for specifying production grammars, and relate our experience with using lava to generate effective test suites for the Java virtual machine. We demonstrate the effectiveness of production grammars in generating complex test cases that can, when combined with comparative and variant testing techniques, achieve high code and value coverage. We also describe an extension to production grammars that enables concurrent generation of certificates for test cases. A certificate is a behavioral description that specifies the intended outcome of the generated test case, and therefore acts as an oracle by which the correctness of the tested system can be evaluated in isolation. We report the results of applying these testing techniques to commercial Java implementations. We conclude that the use of production grammars in combination with other automated testing techniques is a powerful and effective method for testing software systems, and is enabled by a special purpose language for specifying extended production grammars. " links: doi: "http://doi.acm.org/10.1145/331960.331965" tags: - "rule-based" - "Java" - "deployment" - "test coverage" - "modeling language" - "language modeling" - "testing" - "certification" - "code generation" - "security" - "DSL" - "abstract machine" - "coverage" - "extensible language" - "grammar" - "domain-specific language" researchr: "https://researchr.org/publication/SirerB99" cites: 0 citedby: 1 pages: "1-13" booktitle: "DSL" kind: "inproceedings" key: "SirerB99" - title: "Semantic Mutation Testing" author: - name: "John A. Clark" link: "https://researchr.org/alias/john-a.-clark" - name: "Haitao Dan" link: "https://researchr.org/alias/haitao-dan" - name: "Robert M. Hierons" link: "https://researchr.org/alias/robert-m.-hierons" year: "2010" doi: "http://dx.doi.org/10.1109/ICSTW.2010.8" links: doi: "http://dx.doi.org/10.1109/ICSTW.2010.8" dblp: "http://dblp.uni-trier.de/rec/bibtex/conf/icst/ClarkDH10" tags: - "testing" researchr: "https://researchr.org/publication/ClarkDH10" cites: 0 citedby: 0 pages: "100-109" booktitle: "Third International Conference on Software Testing, Verification and Validation, ICST 2010, Paris, France, April 7-9, 2010, Workshops Proceedings" publisher: "IEEE Computer Society" kind: "inproceedings" key: "ClarkDH10" - title: "The Spoofax language workbench: rules for declarative specification of languages and IDEs" author: - name: "Lennart C. L. Kats" link: "http://www.lclnet.nl/" - name: "Eelco Visser" link: "http://eelcovisser.org" year: "2010" doi: "https://doi.org/10.1145/1869459.1869497" abstract: "Spoofax is a language workbench for efficient, agile development of textual domain-specific languages with state-of-the-art IDE support. Spoofax integrates language processing techniques for parser generation, meta-programming, and IDE development into a single environment. It uses concise, declarative specifications for languages and IDE services. In this paper we describe the architecture of Spoofax and introduce idioms for high-level specifications of language semantics using rewrite rules, showing how analyses can be reused for transformations, code generation, and editor services such as error marking, reference resolving, and content completion. The implementation of these services is supported by language-parametric editor service classes that can be dynamically loaded by the Eclipse IDE, allowing new languages to be developed and used side-by-side in the same Eclipse environment." links: doi: "https://doi.org/10.1145/1869459.1869497" dblp: "http://dblp.uni-trier.de/rec/bibtex/conf/oopsla/KatsV10" "acm dl": "https://doi.org/10.1145/1932682.1869497" tags: - "programming languages" - "model-to-model transformation" - "workbench" - "semantics" - "rule-based" - "Eclipse" - "meta programming" - "model editor" - "graph transformation" - "meta-model" - "transformation language" - "architecture" - "reuse" - "model-driven development" - "graph-rewriting" - "rules" - "C++" - "code completion" - "code generation" - "model transformation" - "programming" - "language workbench" - "Spoofax" - "Meta-Environment" - "rewriting" - "parsing" - "meta-objects" - "transformation" - "program transformation" - "domain-specific language" researchr: "https://researchr.org/publication/KatsV10" cites: 0 citedby: 2 pages: "444-463" booktitle: "Proceedings of the 25th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2010, October 17-21, 2010, Reno/Tahoe, Nevada, USA" editor: - name: "William R. Cook" link: "http://www.cs.utexas.edu/~wcook/" - name: "Siobhán Clarke" link: "https://www.cs.tcd.ie/Siobhan.Clarke/" - name: "Martin C. Rinard" link: "https://researchr.org/alias/martin-c.-rinard" address: "Reno/Tahoe, Nevada" publisher: "ACM" isbn: "978-1-4503-0203-6" kind: "inproceedings" key: "KatsV10" - title: "Automatic Generation of Test Cases" author: - name: "Kenneth V. Hanford" link: "https://researchr.org/alias/kenneth-v.-hanford" year: "1970" doi: "http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5388302&tag=1" abstract: "The machine production of programs for testing certain aspects of programming products has been achieved. The system has been successfully used on a number of products to establish their reliability in accepting new test cases without error. The input to the system is a syntax definition in a formal notation. The construction of such a definition for a high-level language is an exacting task. It yields important bonuses by deepening knowledge of the structure of the language and showing up obscurities or ambiguities in the existing documentation, Definitions exist for ECMA Algol, FORTRAN IV, and a major subset of PL/I. The future effort required to adapt the definitions to particular versions of these languages should be small." links: doi: "http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5388302&tag=1" tags: - "programming languages" - "syntax definition" - "testing" - "reliability" researchr: "https://researchr.org/publication/Hanford70" cites: 0 citedby: 0 journal: "IBM Systems Journal" volume: "9" number: "4" pages: "242-257" kind: "article" key: "Hanford70" - title: "Random testing for higher-order, stateful programs" author: - name: "Casey Klein" link: "http://users.eecs.northwestern.edu/~clk800/" - name: "Matthew Flatt" link: "http://www.cs.utah.edu/~mflatt/" - name: "Robby Findler" link: "http://www.eecs.northwestern.edu/~robby/" year: "2010" doi: "http://doi.acm.org/10.1145/1869459.1869505" abstract: "Testing is among the most effective tools available for finding bugs. Still, we know of no automatic technique for generating test cases that expose bugs involving a combination of mutable state and callbacks, even though objects and method overriding set up exactly that combination. For such cases, a test generator must create callbacks or subclasses that aggressively exercise side-effecting operations using combinations of generated objects. This paper presents a new algorithm for randomly testing programs that use state and callbacks. Our algorithm exploits a combination of contracts and environment bindings to guide the test-case generator toward interesting inputs. Our prototype implementation for Racket (formerly PLT Scheme) - which has a Java-like class system, but with first-class classes as well as gbeta-like augmentable methods - uncovered dozens of bugs in a well-tested and widely used text-editor library. We describe our approach in a precise, formal notation, borrowing the techniques used to describe operational semantics and type systems. The formalism enables us to provide a compact and self-contained explanation of the core of our technique without the ambiguity usually present in pseudo-code descriptions." links: doi: "http://doi.acm.org/10.1145/1869459.1869505" dblp: "http://dblp.uni-trier.de/rec/bibtex/conf/oopsla/KleinFF10" tags: - "object-oriented programming" - "semantics" - "Java" - "meta programming" - "formal semantics" - "contracts" - "testing" - "type system" - "random testing" - " mutable objects" - "subject-oriented programming" - "operational semantics" - "Meta-Environment" - "systematic-approach" - "feature-oriented programming" - "meta-objects" researchr: "https://researchr.org/publication/KleinFF10" cites: 0 citedby: 0 pages: "555-566" booktitle: "Proceedings of the 25th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA 2010, October 17-21, 2010, Reno/Tahoe, Nevada, USA" editor: - name: "William R. Cook" link: "http://www.cs.utexas.edu/~wcook/" - name: "Siobhán Clarke" link: "https://www.cs.tcd.ie/Siobhan.Clarke/" - name: "Martin C. Rinard" link: "https://researchr.org/alias/martin-c.-rinard" address: "Reno/Tahoe, Nevada" publisher: "ACM" isbn: "978-1-4503-0203-6" kind: "inproceedings" key: "KleinFF10" - title: "Software Testing with QuickCheck" author: - name: "John Hughes" link: "http://www.cse.chalmers.se/~rjmh" year: "2009" doi: "http://dx.doi.org/10.1007/978-3-642-17685-2_6" abstract: "This paper presents a tutorial, with extensive exercises, in the use of Quviq QuickCheck—a property-based testing tool for Erlang, which enables developers to formulate formal specifications of their code and to use them for testing. We cover the basic concepts of properties and test-data generators, properties for testing abstract data types, and a state-machine modelling approach to testing stateful systems. Finally we discuss applications of QuickCheck in industry." links: doi: "http://dx.doi.org/10.1007/978-3-642-17685-2_6" dblp: "http://dblp.uni-trier.de/rec/bibtex/conf/cefp/Hughes09" tags: - "rule-based" - "meta-model" - "testing" - "type system" - "data-flow" - "Meta-Environment" - "state machines" - "abstract machine" - "QuickCheck" - "systematic-approach" researchr: "https://researchr.org/publication/Hughes09-2" cites: 0 citedby: 0 pages: "183-223" booktitle: "Central European Functional Programming School - Third Summer School, CEFP 2009, Budapest, Hungary, May 21-23, 2009 and Komárno, Slovakia, May 25-30, 2009, Revised Selected Lectures" editor: - name: "Zoltán Horváth" link: "https://researchr.org/alias/zolt%C3%A1n-horv%C3%A1th" - name: "Rinus Plasmeijer" link: "https://researchr.org/alias/rinus-plasmeijer" - name: "Viktória Zsók" link: "https://researchr.org/alias/vikt%C3%B3ria-zs%C3%B3k" volume: "6299" series: "Lecture Notes in Computer Science" publisher: "Springer" isbn: "978-3-642-17684-5" kind: "inproceedings" key: "Hughes09-2" - title: "Test-case reduction for C compiler bugs" author: - name: "John Regehr" link: "http://www.cs.utah.edu/~regehr/" - name: "Chen, Yang" link: "https://researchr.org/alias/chen%2C-yang" - name: "Cuoq, Pascal" link: "https://researchr.org/alias/cuoq%2C-pascal" - name: "Eide, Eric" link: "https://researchr.org/alias/eide%2C-eric" - name: "Ellison, Chucky" link: "https://researchr.org/alias/ellison%2C-chucky" - name: "Yang, Xuejun" link: "https://researchr.org/alias/yang%2C-xuejun" year: "2012" month: "jun" doi: "10.1145/2345156.2254104" abstract: "To report a compiler bug, one must often find a small test case that triggers the bug. The existing approach to automated test-case reduction, delta debugging, works by removing substrings of the original input; the result is a concatenation of substrings that delta cannot remove. We have found this approach less than ideal for reducing C programs because it typically yields test cases that are too large or even invalid (relying on undefined behavior). To obtain small and valid test cases consistently, we designed and implemented three new, domain-specific test-case reducers. The best of these is based on a novel framework in which a generic fixpoint computation invokes modular transformations that perform reduction operations. This reducer produces outputs that are, on average, more than 25 times smaller than those produced by our other reducers or by the existing reducer that is most commonly used by compiler developers. We conclude that effective program reduction requires more than straightforward delta debugging." links: "url": "http://doi.acm.org/10.1145/2345156.2254104" tags: - "rule-based" - "generic programming" - "testing" - "debugging" - "compiler" - "Pascal" - "systematic-approach" - "transformation" - "program transformation" researchr: "https://researchr.org/publication/Regehr%3A2012%3ATRC%3A2345156.2254104" cites: 0 citedby: 0 journal: "SIGPLAN Notices" volume: "47" number: "6" pages: "335-346" kind: "article" key: "Regehr:2012:TRC:2345156.2254104" - title: "Randomized Testing in PLT Redex" author: - name: "Casey Klein" link: "http://users.eecs.northwestern.edu/~clk800/" - name: "Robby Findler" link: "http://www.eecs.northwestern.edu/~robby/" year: "2009" abstract: "This paper presents new support for randomized testing in PLT Redex, a domain-specific language for formalizing operational semantics. In keeping with the overall spirit of Redex, the testing support is as lightweight as possible—Redex programmers simply write down predicates that correspond to facts about their calculus and the tool randomly generates program expressions in an attempt to falsify the predicates. Redex’s automatic test case generation begins with simple expressions, but as time passes, it broadens its search to include increasingly complex expressions. To improve test coverage, test generation exploits the structure of the model’s metafunction and reduction relation definitions. The paper also reports on a case-study applying Redex’s testing support to the latest revision of the Scheme standard. Despite a community review period, as well as a comprehensive, manually constructed test suite, Redex’s random test case generation was able to identify several bugs in the semantics." tags: - "PLT Redex" - "programming languages" - "semantics" - "test coverage" - "meta programming" - "case study" - "meta-model" - "modeling language" - "language modeling" - "testing" - "reviewing" - "random testing" - "operational semantics" - "Meta-Environment" - "search" - "coverage" - "domain-specific language" researchr: "https://researchr.org/publication/KleinFindler2009" cites: 0 citedby: 0 booktitle: "Workshop on Scheme and Functional Programming (SFP)" kind: "inproceedings" key: "KleinFindler2009"