How does shadoworflow tf.contrib.training.batch_sequences_with_states API work?

Question

I am dealing with long serial data that must be transmitted to RNN. There are two options for performing truncated BPTT and batch processing:

• Create a package by combining the corresponding segments from different sequences. Save the final state of each sequence in a batch and pass it on to the next batch.
• Consider each sequence as a mini-packet with segments from a sequence becoming members of the packet. Save the state of the last step in one segment and transfer it to the first step of the next segment.

I came across tf.contrib.training.batch_sequences_with_states , which seems to do one of two things. The documentation is confusing for me, and so I want to be sure how it generates batches.

I guess this is the first way. The fact is that if batch processing is performed in the second way, then we cannot take advantage of vectorization, because in order to maintain the state between the last time step of one segment to the first time step of the next segment, RNN must process one token sequentially.

Question:

Which of these two dosing strategies is implemented in tf.contrib.training.batch_sequences_with_states ?

Answer 1

tf.contrib.training.batch_sequences_with_states implements the old behavior. Each record for a mini-channel represents a segment from a different sequence (each sequence, which may consist of a variable number of segments, has a unique key, and this key is passed to batch_sequences_with_states ). When used with state_saving_rnn final state for each segment is saved back to a special storage container, which allows the next segment of this sequence to be executed in the next sess.run . End segments free up the chip slot for another sequence.