How low-key bidirectional RNNs work in Keras

Question

In Keras, the Bidirectional wrapper for RNN also supports stateful=true . I really don't understand how this should work:

In a unidirectional model with a fixed state, the state of the lot is transferred to the next lot. I think it works the same for the straight layer in a bidirectional model.

But where does the back layer get states from it? If I understand everything correctly, he should technically get it from the “next” batch. But, obviously, the “next” batch has not yet been calculated, so how does it work?

Answer 1

You might think of a Bidirectional layer as follows:

 forward = Recurrent(..)(input) backward = Recurrent(..., reverse_input=True)(input) output = merge([forward, backward], ...) 

So - as you can see - you are losing your temporal orientation. You analyze input from start to finish. In this case, setting stateful=True simply takes its initial state from the previous pattern in accordance with the direction of the bidirectional branch ( forward accepts from forward , backward accepts from backward ).

This causes your model to lose interpretation - selections from parallel batches can be interpreted as a compact sequence divided into batches.