Generate AST as a point file

Question

I work with ANTLR4 to generate the java source code AST, and I had to switch to ANTLR3 because I did not get much help and documentation, and it was very difficult to continue. I managed to create an AST, but not in a visual format. Then I came up with an amazing answer, and I was really able to create an AST in a DOT file, but there was a little problem.

My code is:

import org.antlr.runtime.CommonTokenStream; import org.antlr.runtime.ANTLRFileStream; import org.antlr.runtime.tree.CommonTree; import org.antlr.runtime.tree.DOTTreeGenerator; import org.antlr.stringtemplate.StringTemplate; class Main { public static void main(String[] args) throws Exception { parseFile("/home/satnam-sandhu/Workstation/ASTGenerator/resource/java/Blabla.java"); } public static void parseFile(String f)throws Exception { JavaLexer lexer = new JavaLexer(new ANTLRFileStream(f)); CommonTokenStream tokens = new CommonTokenStream(lexer); JavaParser parser = new JavaParser(tokens); CommonTree tree = (CommonTree)parser.compilationUnit().getTree(); DOTTreeGenerator gen = new DOTTreeGenerator(); StringTemplate st = gen.toDOT(tree); System.out.println(st); } } 

I use gradle, so I am building a project:

 gradle clean build 

And then run it and pass the output to the point file:

 java -jar ASTGenerator.jar > ast.dot 

Now the problem I am facing is that for the source code:

 class example{ public static void print(int a){ int b = a +1; System.out.println(b); } public static void main(){ print(15); } } 

I get this as output:

 digraph { ordering=out; ranksep=.4; bgcolor="lightgrey"; node [shape=box, fixedsize=false, fontsize=12, fontname="Helvetica-bold", fontcolor="blue" width=.25, height=.25, color="black", fillcolor="white", style="filled, solid, bold"]; edge [arrowsize=.5, color="black", style="bold"] n0 [label=""]; n1 [label="class"]; n2 [label="example"]; n3 [label="{"]; n4 [label="public"]; n5 [label="static"]; n6 [label="void"]; n7 [label="print"]; n8 [label="("]; n9 [label="int"]; n10 [label="a"]; n11 [label=")"]; n12 [label="{"]; n13 [label="int"]; n14 [label="b"]; n15 [label="="]; n16 [label="a"]; n17 [label="+"]; n18 [label="1"]; n19 [label=";"]; n20 [label="System"]; n21 [label="."]; n22 [label="out"]; n23 [label="."]; n24 [label="println"]; n25 [label="("]; n26 [label="b"]; n27 [label=")"]; n28 [label=";"]; n29 [label="}"]; n30 [label="public"]; n31 [label="static"]; n32 [label="void"]; n33 [label="main"]; n34 [label="("]; n35 [label=")"]; n36 [label="{"]; n37 [label="print"]; n38 [label="("]; n39 [label="15"]; n40 [label=")"]; n41 [label=";"]; n42 [label="}"]; n43 [label="}"]; n0 -> n1 // "" -> "class" n0 -> n2 // "" -> "example" n0 -> n3 // "" -> "{" n0 -> n4 // "" -> "public" n0 -> n5 // "" -> "static" n0 -> n6 // "" -> "void" n0 -> n7 // "" -> "print" n0 -> n8 // "" -> "(" n0 -> n9 // "" -> "int" n0 -> n10 // "" -> "a" n0 -> n11 // "" -> ")" n0 -> n12 // "" -> "{" n0 -> n13 // "" -> "int" n0 -> n14 // "" -> "b" n0 -> n15 // "" -> "=" n0 -> n16 // "" -> "a" n0 -> n17 // "" -> "+" n0 -> n18 // "" -> "1" n0 -> n19 // "" -> ";" n0 -> n20 // "" -> "System" n0 -> n21 // "" -> "." n0 -> n22 // "" -> "out" n0 -> n23 // "" -> "." n0 -> n24 // "" -> "println" n0 -> n25 // "" -> "(" n0 -> n26 // "" -> "b" n0 -> n27 // "" -> ")" n0 -> n28 // "" -> ";" n0 -> n29 // "" -> "}" n0 -> n30 // "" -> "public" n0 -> n31 // "" -> "static" n0 -> n32 // "" -> "void" n0 -> n33 // "" -> "main" n0 -> n34 // "" -> "(" n0 -> n35 // "" -> ")" n0 -> n36 // "" -> "{" n0 -> n37 // "" -> "print" n0 -> n38 // "" -> "(" n0 -> n39 // "" -> "15" n0 -> n40 // "" -> ")" n0 -> n41 // "" -> ";" n0 -> n42 // "" -> "}" n0 -> n43 // "" -> "}" } 

When using http://viz-js.com/ to render output, it looks like this:

All my work is still loading here if you guys want to delve into the grammar file that I use. I think that the options indicated in the grammar file may also be the reason. I beginner cannot continue without a little help. Thanks in advance.:)

Answer 1

The answer I was looking for answers here on Bart Kiers , but for people who want to generate DOT files without changing the grammar, they can print the desired syntax tree using this repository help. Since I did not find much documentation on generating points in ANTLR4 and did not use any other option than modifying the ANTLR3 grammar file, I used the Federico Tomassetti example and modified it a bit to create my own DOT file.

You can print the output dot file:

 import java.io.File; import java.io.IOException; import java.util.ArrayList; import java.nio.charset.Charset; import java.nio.file.Files; import org.antlr.v4.runtime.ANTLRInputStream; import org.antlr.v4.runtime.CommonTokenStream; import org.antlr.v4.runtime.ParserRuleContext; import org.antlr.v4.runtime.RuleContext; import org.antlr.v4.runtime.tree.ParseTree; public class ASTGenerator { static ArrayList<String> LineNum = new ArrayList<String>(); static ArrayList<String> Type = new ArrayList<String>(); static ArrayList<String> Content = new ArrayList<String>(); private static String readFile() throws IOException { File file = new File("resource/java/Blabla.java"); byte[] encoded = Files.readAllBytes(file.toPath()); return new String(encoded, Charset.forName("UTF-8")); } public static void main(String args[]) throws IOException{ String inputString = readFile(); ANTLRInputStream input = new ANTLRInputStream(inputString); Java8Lexer lexer = new Java8Lexer(input); CommonTokenStream tokens = new CommonTokenStream(lexer); Java8Parser parser = new Java8Parser(tokens); ParserRuleContext ctx = parser.compilationUnit(); generateAST(ctx, false, 0); System.out.println("digraph G {"); printDOT(); System.out.println("}"); } private static void generateAST(RuleContext ctx, boolean verbose, int indentation) { boolean toBeIgnored = !verbose && ctx.getChildCount() == 1 && ctx.getChild(0) instanceof ParserRuleContext; if (!toBeIgnored) { String ruleName = Java8Parser.ruleNames[ctx.getRuleIndex()]; LineNum.add(Integer.toString(indentation)); Type.add(ruleName); Content.add(ctx.getText()); } for (int i = 0; i < ctx.getChildCount(); i++) { ParseTree element = ctx.getChild(i); if (element instanceof RuleContext) { generateAST((RuleContext) element, verbose, indentation + (toBeIgnored ? 0 : 1)); } } } private static void printDOT(){ printLabel(); int pos = 0; for(int i=1; i<LineNum.size();i++){ pos=getPos(Integer.parseInt(LineNum.get(i))-1, i); System.out.println((Integer.parseInt(LineNum.get(i))-1)+Integer.toString(pos)+"->"+LineNum.get(i)+i); } } private static void printLabel(){ for(int i =0; i<LineNum.size(); i++){ System.out.println(LineNum.get(i)+i+"[label=\""+Type.get(i)+"\\n "+Content.get(i)+" \"]"); } } private static int getPos(int n, int limit){ int pos = 0; for(int i=0; i<limit;i++){ if(Integer.parseInt(LineNum.get(i))==n){ pos = i; } } return pos; } } 

For source code like this:

 class example{ public static void main(){ int a; a = 5; } } 

The output will be:

 digraph G { 00[label="compilationUnit\n classexample{publicstaticvoidmain(){inta;a=5;}}<EOF> "] 11[label="normalClassDeclaration\n classexample{publicstaticvoidmain(){inta;a=5;}} "] 22[label="classBody\n {publicstaticvoidmain(){inta;a=5;}} "] 33[label="methodDeclaration\n publicstaticvoidmain(){inta;a=5;} "] 44[label="methodModifier\n public "] 45[label="methodModifier\n static "] 46[label="methodHeader\n voidmain() "] 57[label="result\n void "] 58[label="methodDeclarator\n main() "] 49[label="block\n {inta;a=5;} "] 510[label="blockStatements\n inta;a=5; "] 611[label="localVariableDeclarationStatement\n inta; "] 712[label="localVariableDeclaration\n inta "] 813[label="integralType\n int "] 814[label="variableDeclaratorId\na "] 615[label="expressionStatement\na=5; "] 716[label="assignment\na=5 "] 817[label="expressionName\na "] 818[label="assignmentOperator\n = "] 819[label="literal\n 5 "] 00->11 11->22 22->33 33->44 33->45 33->46 46->57 46->58 33->49 49->510 510->611 611->712 712->813 712->814 510->615 615->716 716->817 716->818 716->819 } 

Paste this piece of output into http://viz-js.com/ You will get this as an output:

You can also transfer the output to the ast.dot file with:

 java -jar path-to-jar-file.jar > ast.dot 

Now this is not an ideal method, but enough for me. :)

Hope this helps.