Simple lexical analysis of a java program

My small project is a lexical analysis program in which I have to take every word found in an arbitrary .java file and list each line that it displays in the file. I need to have one lookup table designed only for reserved words, and another for all additional words found in the document. So, for a program such as:

public class xxxx { int xyz; xyz = 0; } 

The output should be:

 Reserved words: class: 1 int: 2 public: 1 Other words: xxxx: 1 xyz: 2, 3 

But I have a lot of problems with my current program, so I have no idea what is happening, so amendments to my program or full correspondence are welcome. I'm just trying to understand Java as a hobby, so all help is appreciated as long as I understand what is going on. I am sure there is a simple solution to this problem, but my attempt did not work :( Thanks for any help ^^

 import java.io.File; import java.io.FileNotFoundException; import java.util.ArrayList; import java.util.HashMap; import java.util.Iterator; import java.util.List; import java.util.Map; import java.util.Scanner; public class LexicalAnalysis { private String[] keywords = { "abstract", "boolean", "byte", "case", "catch", "char", "class", "continue", "default", "do", "double", "else", "extends", "final", "finally", "float", "for", "if", "implements", "import", "instanceof", "int", "interface", "long", "native", "new", "package", "private", "protected", "public", "return", "short", "static", "super", "switch", "synchronized", "this", "throw", "throws", "transient", "try", "void", "volatile", "while", "false", "true", "null" }; HashMap<String, ArrayList<Integer>> keywordsTable; HashMap<String, ArrayList<Integer>> otherWords = new HashMap<String, ArrayList<Integer>>(); public LexicalAnalysis(String fileName){ Scanner kb = null; int lineNumber = 0; try { kb = new Scanner(new File(fileName)); } catch (FileNotFoundException e) { e.printStackTrace(); } keywordsTable = new HashMap<String, ArrayList<Integer>>(); for(int i = 0; i < 47; i++){ keywordsTable.put(keywords[i], new ArrayList<Integer>()); } while(kb.hasNextLine()){ lineNumber++; String line = kb.nextLine(); String[] lineparts = line.split("\\s+|\\.+|\\;+|\\(+|\\)+|\\\"+|\\:+|\\[+|\\]+"); for(String x: lineparts){ ArrayList<Integer> list = keywordsTable.get(x); if(list == null){ list = otherWords.get(x); if(list == null){ ArrayList<Integer> temp = new ArrayList<Integer>(); temp.add(lineNumber); otherWords.put(x,temp); }else{ otherWords.remove(x); ArrayList<Integer> temp = new ArrayList<Integer>(); temp.add(lineNumber); otherWords.put(x, temp); } }else{ keywordsTable.remove(x); ArrayList<Integer> temp = new ArrayList<Integer>(); temp.add(lineNumber); keywordsTable.put(x, temp); } } } System.out.println("Keywords:"); printMap(keywordsTable); System.out.println(); System.out.println("Other Words:"); printMap(otherWords); } public static void printMap(Map<String, ArrayList<Integer>> mp) { Iterator<Map.Entry<String, ArrayList<Integer>>> it = mp.entrySet().iterator(); while (it.hasNext()) { Map.Entry<String, ArrayList<Integer>> pairs = (Map.Entry<String, ArrayList<Integer>>)it.next(); System.out.print(pairs.getKey() + " = "); printList(pairs.getValue()); System.out.println(); it.remove(); } } public static void printList(List x){ for(Object m : x){ System.out.print(m + ", "); } } public static void main(String args[]){ new LexicalAnalysis("lexitest.txt"); } } 
+4
source share
2 answers

The easiest way to do this is to use JFlex with the correct keywords for the lex file. Once you do this, counting identifiers and keywords is trivial.

+1
source

I found one error that, I think, fixed everything. You need to specify the directory of the file you are restoring, basically. For example, you now have a new LexicalAnalysis ("lexitest.txt");

In my example, I use my flashdrive, so this will be the new LexicalAnalysis ("F" \ lexitest.txt ");

0
source

Source: https://habr.com/ru/post/1391354/


All Articles