How to implement include directives with boost :: spirit :: lex?

Question

I have a simple configuration parser built from spirit :: lex and spirit :: qi. When the lexer reaches the include "path" template, I want the text of the file to be included. As you know, the spirit :: lexer :: begin () starts the scanning process:

 // Read file contents into a std::string ... // _first and _last are const char* _first = _contents.c_str(); _last = &_first[_input.size()]; // _token is a lexer::iterator_type for the current token _token = _lexer.begin(_first, _last); 

My idea is to have a stack that stores the lexer state, represented as a struct:

 struct LexerState { const char* first; const char* last; std::string contents; }; 

The lexer will be created to recognize the template for include "path" and in a semantic action will extract the path to the included file. Then the current lexer state is pushed onto the stack, the contents of the file are loaded into the string, and the new state is initialized, as indicated above, using lexer :: begin ().

When the lexer finds the EOF character, the stack is popped, and the lexer :: begin () method is called using the previous lexer state variables.

Is it possible to repeatedly call lexer :: begin () like this? How to get lex :: lexer to recognize the include "path" pattern and the EOF character without returning the marker in the qi parser?

Finally, are there any alternative or better ways to do this?

Answer 1

See how Boost Wave does things:

The Wave C ++ preprocessor library uses the Spirit parser constructor library to implement the C ++ lexicologist with standard ISO / ANSI preprocessing capabilities. It provides an iterator interface that returns the current pre-processed token from the input stream. This pre-processed token is generated on the fly, iterating over the sequence of the preprocessor iterator (in STL terminology, these iterators are forward iterators).

And about the possibilities:

The C ++ preprocessor provides four separate tools that you can use as you wish:

• Include header files
• Macro extension
• Conditional compilation
• Line control

Their quick start example shows how you use the lexer boost wave interface.