Michael L. van de Vanter. Preserving the Documentary Structure of Source Code in Language-Based Transformation Tools. In 1st IEEE International Workshop on Source Code Analysis and Manipulation (SCAM 2001), 10 November 2001, Florence, Italy. pages 133-143, IEEE Computer Society, 2001. [doi]
Language-based tools necessarily translate textual source code into grammar-based representations. During translation, tools such as compilers and analyzers are generally free to discard comments and white space, which have no impact on the code’s formal meaning. Tools that produce transformed source code for human consumption enjoy no such freedom. Comments and white space are crucial to the comprehensibility and maintainability of source code and thus to its ultimate value. However, it is not always practical or desirable for transformation tools to replicate comments and white space in their entirety. An analysis of the documentary (as opposed to linguistic) structure of source code leads to a practical strategy for preserving its comprehensibility when processed by such tools.