ANTLR generates code, which you have to assume works as it should. However, the output is a variable factor, determined only by your grammar files. Considering that the rest of the application, or at least the part which loads the AST into your object model, depends on a specific output from your parser, it’s smart to test your parser output.
Consider the following grammar:
grammar expression;
options {
output=AST;
ASTLabelType=CommonTree;
}
expression : additiveExpression*;
additiveExpression : value ('+' value)* -> ^('+' value+);
value : NUMERIC;
NUMERIC : '0'..'9'+;
This grammar accepts input in the form of x+y, where x is an integer value. Thanks to the rewrite rule we’ve defined, the output will be '(+ x y)’. This is quite handy for parsing/building expression trees, but that’s out of scope for today. The trick is that the code which deals with the generated AST directly is very dependant on the output. If you decide to change your grammar setup, which could be for any reason ranging from aesthetics to simply adding features, you risk breaking your code. This could result in compiler errors, but I found that more often than not it’s a silent bug. And you know, silent bugs are quite annoying to track down. Now, let me show you a quite simple approach on how to test the output of the parser. I’ll use NUnit for the test syntax, combined with the NUnit extension methods.
Every parser eventually returns an object which extends ParserRuleReturnScope, which provides us with a way of getting the AST out in the form of an object which implements ITree. A very handy helper method on that interface is the ‘ToStringTree’ method. All it does is convert your tree into a string, in prefix notation. An example:
Will be formatted as (html (head title) (body div)). The first element in the list is the root, the consecutive elements the children. We’ll use this method to our advantage in our tests.
To get to the point of being able to get the string representation of an AST, you first have to do some plumbing. I’ll use the simplest form possible.
public expressionParser CreateParser(string input)
{
var stream = new ANTLRStringStream(input);
var lexer = new expressionLexer(stream);
return new expressionParser(new CommonTokenStream(lexer));
}
This will create a parser we can use, which in turn allows us to get the AST we want to use. The test fixture itself is pretty straightforward. We know which output we expect, all we have to do is assert it’s equal.
[TestFixture]
public class ParserTest
{
[Test]
public void Parser_TestAdditiveExpression()
{
string input = "10+2";
expressionParser parser = CreateParser(input);
additiveExpression_return result = parser.additiveExpression();
ITree tree = result.Tree as ITree;
tree.ToStringTree().Should().Be.EqualTo("(+ 10 2)");
// Or: Assert.AreEqual("(+ 10 2)", tree.ToStringTree());
}
}
That's basically it! Now, even though this test works, it has a lot of repetition when you add multiple tests with different input. For this, you can use a simple extension method, which operates on a string.
public static string GetStringTree(this string input, Func<expressionParser, ParserRuleReturnScope> func)
{
var parser = CreateParser(input);
return (func(parser).Tree as ITree).ToStringTree();
}
Which you can use to simplify the test, as such:
[TestFixture]
public class ParserTest
{
[Test]
public void Parser_TestAdditiveExpression()
{
string result = "10+2".GetStringTree(p => p.additiveExpression());
result.Should().Be.EqualTo("(+ 10 2)");
// Or: Assert.AreEqual("(+ 10 2)", result);
}
}
Happy testing!