PLY: Token Shift Problem in C parser

Question

I am writing a C parser using PLY and have recently encountered a problem. This code:

typedef int my_type;
my_type x;

Correct C code, because my_type is defined as a type earlier as such. I process it by filling out a table of type characters in a parser, which is used by the lexer to distinguish between types and simple identifiers.

However, while the type declaration rule ends with SEMI (token ;;), PLY shifts the token my_typefrom the second line before making a decision from the first. Because of this, I have no way to pass the update in the type symbol table to lexer, and this sees my_type as an identifier, not a type.

Any ideas for a fix?

Full code: http://code.google.com/p/pycparser/source/browse/trunk/src/c_parser.py I don’t know how I can create a smaller example from this.

Edit:

The problem is resolved. See My solution below.

Answer 1

With some help from Dave Beasley (creator of PLY), my problem was resolved.

The idea is to use special subheadings and perform actions on them. In my case, I divided the rule declarationinto:

def p_decl_body(self, p):
    """ decl_body : declaration_specifiers init_declarator_list_opt
    """
    # <<Handle the declaration here>>        

def p_declaration(self, p):
    """ declaration : decl_body SEMI 
    """
    p[0] = p[1]

decl_body always shortens before the token after the SEMI offset, so my action is performed at the right time.

Answer 2

Not sure why you are doing this level of analysis in your lexer.

, , (, , ..). , , typedef ..

, lexx yacc, .

Answer 3

, , TYPEID c_lexer.py c_parser.py.

, 1 , .

, , TYPEID , , .

Pax Diablo , lexer/tokenizer , . .