Using character literals as terminals in a bison

Question

I'm trying to understand flex / bison, but the documentation is a little complicated for me, and I probably misunderstood something. Here's a test case: http://namakajiri.net/misc/bison_charlit_test/

File "a" contains a single character "a". "foo.y" has a trivial grammar:

%% file: 'a' ; 

The generated parser cannot parse file "a"; it gives a syntax error.

The grammar "bar.y" is almost the same, only I changed the letter character for the named token:

 %token TOK_A; %% file: TOK_A; 

and then in bar.lex:

 a { return TOK_A; } 

It works well.

What am I doing wrong when trying to use character literals directly as can terminals, for example, in documents?

I would like my grammar to look like "statement: selector" {'property': 'value'; ''} '', not "statement: selector LBRACE property COLON value SEMIC RBRACE" ...

I am running bison 2.5 and flex 2.5.35 in debian wheezy.

Answer 1

Overwrite

The problem is the runtime problem, not the compile time problem.

The problem is that you have two completely different lexical analyzers.

The bar.lex analyzer recognizes a at the input and returns it as TOK_A and ignores everything else.

The foo.lex analyzer echoes each individual character, but that’s it.

foo.lex - as it is written

 %{ #include "foo.tab.h" %} %% 

foo.lex - equivalent

 %{ #include "foo.tab.h" %} %% . { ECHO; } 

foo.lex - required

 %{ #include "foo.tab.h" %} %% . { return *yytext; } 

---

Working code

Here is the working code with diagnostic printing in place.

Foo-lex.l

 %% . { printf("Flex: %d\n", *yytext); return *yytext; } 

foo.y

 %{ #include <stdio.h> void yyerror(char *s); %} %% file: 'a' { printf("Bison: got file!\n") } ; %% int main(void) { yyparse(); } void yyerror(char *s) { fprintf(stderr, "%s\n", s); } 

Compilation and Execution

 $ flex foo-lex.l $ bison foo.y $ gcc -o foo foo.tab.c lex.yy.c -lfl $ echo a | ./foo Flex: 97 Bison: got file! $ 

---

Detail point: how did this empty line get into the output? Answer: the lexical analyzer put it there. Template . does not match the new line, so the new line indicated that there is a rule:

 \n { ECHO; } 

That is why the entry was accepted. If you change the file foo-lex.l to:

 %% . { printf("Flex-1: %d\n", *yytext); return *yytext; } \n { printf("Flex-2: %d\n", *yytext); return *yytext; } 

and then recompile and run again, exit:

 $ echo a | ./foo Flex-1: 97 Bison: got file! Flex-2: 10 syntax error $ 

no empty lines. This is because the grammar does not allow the appearance of a new line in the actual "file".