How to capture a string without quotation marks

Question

I am trying to grab quoted strings without quotes. I have this terminal

%token <string> STRING 

and this production

 constant: | QUOTE STRING QUOTE { String($2) } 

along with these lexer rules

 | '\'' { QUOTE } | [^ '\'']* { STRING (lexeme lexbuf) } //final regex before eof 

He seems to interpret everything that leads to QUOTE as the only token that does not parse. So maybe my problem elsewhere in the grammar is not sure. Am I doing it right? He knew it well before he tried to exclude quotes from the lines.

Update

I think there might be some ambiguity with the following lexer rules

 let name = alpha (alpha | digit | '_')* let identifier = name ('.' name)* 

The following rule precedes STRING

 | identifier { ID (lexeme lexbuf) } 

Is there a way to fix them without including quotes in STRING regex?

Answer 1

It is pretty normal to do lexer semantic analysis for constants such as strings and numeric literals, so you can consider the lex rule for your string constants such as

 | '\'' [^ '\'']* '\'' { STRING (let s = lexeme lexbuf in s.Substring(1, s.Length - 2)) } 

Answer 2

You can use lexeme with quotes, but trigger quotes in the parser

Lexer:

 let constant = ("'" ([^ '\''])* "'") ... | constant { STRING(lexeme lexbuf) } 

Parser:

 %token <string> STRING ... constant: | STRING { ($1).Trim([|'''|]) } 

Or , if you want to extract quotes from a string:

Lexer:

 let name = alpha (alpha | digit | '_')* let identifier = name ('.' name)* ... | '\'' { QUOTE } | identifier { ID (lexeme lexbuf) } | _ { STRING (lexeme lexbuf) } 

Identifier

will display characters from STRING, so your lexeme stream might look like: QUOTE ID STRING ID .. QUOTE, and you should handle this in the parser:

Parser:

 constant: | QUOTE content QUOTE { String($2) } content: | ID content { $1+$2 } | STRING content { $1+$2 } | ID { $1 } | STRING { $1 } 

Answer 3

I had a similar problem. I commit them to the file "lexic.l" using states. Here is my answering machine