A comparative evaluation of techniques for syntactic level source code analysis

Anthony Cox, Charles Clarke

Abstract: Many program maintenance tools rely on traditional parsing techniques to obtain syntactic level models of the code being maintained. When, for some reason, code cannot be parsed, software maintainers are forced to fall back on ad hoc tools and techniques, such as grep. As an alternative, hierarchical lexical analysis augmented with simple data structures can be used to extract an approximation of the abstract syntax for a source file. Experiments indicate that such an approach is feasible and produces results comparable to those obtained using a parser.