How to use non-blocking or asynchronous IO with Boost Spirit?

Question

Does the Spirit provide any opportunities for working with non-blocking IO?

To provide a more specific example: I would like to use the Boost Spirit parsing structure to parse data coming from a network socket that was placed in non-blocking mode. If the data is not fully accessible, I would like to use this thread to do other work, and not to block it.

The trivial answer is to simply read all the data before calling Spirit, but potentially gigabytes of data must be received and analyzed from the socket.

It seems that to support non-blocking I / O during parsing, the Spirit will need some ability to partially parse the data and be able to pause and maintain its parsing state when more data is not available. In addition, when updating data, it will be necessary to resume parsing from the saved parsing state. Or maybe I'm making it too complicated?

Answer 1

TODO An example of a simple, single-threaded event-based parsing model will be published. This is largely trivial, but it may just be what you need.

For anything less trivial, please pay attention to the following considerations / hints / tips:

---

How would you consume the result? In any case, would you not have synthesized attributes before, or do you intend to use semantic actions "on the fly"?

This usually does not work due to return. Reservations could be circumvented by careful and prudent use of qi :: hold, qi :: locals and place semantic actions with side effects only on stations that will never be returned. In other words:

• it will be a mistake
• this, of course, applies only to a limited set of grammars (these grammars with rich contextual information will not lend themselves well to this treatment).

Now everything can be forced, of course, but, in general, experienced programmers had to learn how to avoid swimming upstream.

Now, if you still want to do this:

You should be able to get safe / reentrant-style virus library streams by specifying BOOST_SPIRIT_THREADSAFE and contacting libboost_thread. Note , this does the gobals used by Spirit threadafe (due to fine-grained locking), but not your parsers: you cannot share your own parsers / rules / subgraphs / expressions by stream. In fact, you can only share your functions (Phoenix / Fusion) if they are thread safe, and any other extensions defined outside the main Spirit library should be checked for thread safety.

If you handle the above, I think the best approach would seem to be to

• use boost :: spirit :: istream_iterator (or for binary / raw character streams, I would prefer to define a similar boost::spirit::istreambuf_iterator using the template class boost::spirit::multi_pass<> ) to consume input. Note that depending on your grammar, enough buffering can be used for buffering, and performance is not optimal.
• run the analyzer on your own stream (or a logical stream, for example, Boost Asio "strands" or its famous "stack copies")
• use crude semantic actions, as shown above, to pass messages to another logical thread that does the actual processing.

Some looser pointers:

• you can easily "merge" some functions to handle the lazy evaluation of your semantic action handlers using BOOST_FUSION_ADAPT_FUNCTION and friends; This reduces the number of cracks you have to write to get simple things working like normal C ++ overload resolution in semantic actions - especially if you don't use C ++ 0X and BOOST_RESULT_OF_USE_DECLTYPE
• Since you need to avoid semantic actions with side effects, you should probably take a look at Inherited Attributes and qi::locals<> to coordinate state between rules in "pure functional mode".