A generator for production quality compilers

Software: Practice and Experience, 1995

Despite the parsing power of© © algorithms, e.g. YACC 1 , programmers often choose to write recursive-descent parsers by hand to obtain increased flexibility, better error handling, and ease of debugging. We introduce ANTLR, a public-domain parser generator that combines the flexibility of hand-coded parsing with the convenience of a parser generator, which is a component of PCCTS 2 . ANTLR has many features that make it easier to use than other language tools. Most important, ANTLR provides predicates which let the programmer systematically direct the parse via arbitrary expressions using semantic and syntactic context; in practice, the use of predicates eliminates the need to hand-tweak the ANTLR output, even for difficult parsing problems. ANTLR also integrates the description of lexical and syntactic analysis, accepts " ! g rammars for $ # & % w ith extended BNF notation, and can automatically generate abstract syntax trees.

IEE Proceedings - Software, 2003

Recursive descent parsing has the advantage of more convenient semantic processing than in the case of bottom-up parsing. However, it is limited to LL(1) grammars, which has often prevented its use for popular programming languages, such as Java or C#. The authors extended their compiler generator Coco/R, which generates recursive descent parsers, so that the user can specify rules for LL(1) conflict resolution. The simple technique works by evaluating user-defined Boolean expressions to allow the parser to make its parsing decisions where a one-symbol lookahead does not suffice. Thus multi-symbol lookahead or even semantic information can now guide the parsing process as well. Using their extended compiler generator, the authors implemented a compiler front end for C# that can be used as a framework for implementing a variety of tools, two of which are described in more detail: a white-box testing tool that measures path coverage, and a performance hint tool that evaluates source code according to user-defined rules about performance enhancements or coding style.

Database and Expert Systems …, 2007

The parsing schemata formalism allows us to describe parsing algorithms in a simple, declarative way by capturing their fundamental semantics while abstracting low-level detail. In this work, we present a compilation technique allowing the automatic transformation of parsing schemata to efficient executable implementations of their corresponding algorithms. Our technique is general enough to be able to handle all kinds of schemata for context-free grammars, tree adjoining grammars and other grammatical formalisms, providing an extensibility mechanism which allows the user to define custom notational elements.

Computer Languages, 1990

Communications of the ACM, 1992

Lecture Notes in Computer Science, 1986

We consider the problem of automatically deriving correct compilers from relational semantic specifications of programming languages. A relational semantics is an assignment of initial-state final-state relations (defined by means of proof rules) to programs. Compilers are generated in three steps. First, the language definition is transformed into a stack semantics in which the sto~age of semantic values is made explicit. Next, stack rules are assembled into a socalled language scheme containing exactly one rule for each language construct. We consider languages for which non-deterministic branches may be replaced by deterministic ones. Finally, rules are expanded for the purpose of recursion detection, thus obtaining schemes which may be used for code generation in a syntax-directed compiler.

Software: Practice and Experience, 1990

A variant of the well-known recursive descent parsing technique is developed, based on the assumption that each non-terminal of the language is implemented through a separate module. The modularization principle requires that the recursive descent parser of one non-terminal must be written knowing only the production rules of that non-terminal, and not the other parts of the language. This leads to a parsing technique which is 'lazy' in the sense that part of the parser is constructed during parsing, according to the needs of a particular input text. It turns out that this technique can be implemented as efficiently as conventional recursive descent parsing, which is one of the fastest known parsing methods. The main benefit of the technique is that it allows a language to be implemented in small pieces which are easy to modify, replace, and reuse.

2011

Despite the power of Parser Expression Grammars (PEGs) and GLR, parsing is not a solved problem. Adding nondeterminism (parser speculation) to traditional LL and LR parsers can lead to unexpected parse-time behavior and introduces practical issues with error handling, single-step debugging, and side-effecting embedded grammar actions. This paper introduces the LL(*) parsing strategy and an associated grammar analysis algorithm that constructs LL(*) parsing decisions from ANTLR grammars. At parse-time, decisions gracefully throttle up from conventional fixed k ≥ 1 lookahead to arbitrary lookahead and, finally, fail over to backtracking depending on the complexity of the parsing decision and the input symbols. LL(*) parsing strength reaches into the context-sensitive languages, in some cases beyond what GLR and PEGs can express. By statically removing as much speculation as possible, LL(*) provides the expressivity of PEGs while retaining LL's good error handling and unrestricted grammar actions. Widespread use of ANTLR (over 70,000 downloads/year) shows that it is effective for a wide variety of applications.

Information Processing Letters, 1983

Formal languages let us define the textual representation of data with precision. Formal grammars, typically in the form of BNF-like productions, describe the language syntax, which is then annotated for syntax-directed translation and completed with semantic actions. When, apart from the textual representation of data, an explicit representation of the corresponding data structure is required, the language designer has to devise the mapping between the suitable data model and its proper language specification, and then develop the conversion procedure from the parse tree to the data model instance. Unfortunately, whenever the format of the textual representation has to be modified, changes have to propagated throughout the entire language processor tool chain. These updates are time-consuming, tedious, and error-prone. Besides, in case different applications use the same language, several copies of the same language specification have to be maintained. In this paper, we introduce ModelCC, a model-based parser generator that decouples language specification from language processing, hence avoiding many of the problems caused by grammar-driven parsers and parser generators. ModelCC incorporates reference resolution within the parsing process. Therefore, instead of returning mere abstract syntax trees, ModelCC is able to obtain abstract syntax graphs from input strings.