Multilevel bidirectional_dynamic_rnn: incompatible with MultiRNNCell?

Question

I want to create a multilayer bidirectional LSTM in Tensorflow. Currently, my single-layer model looks like this:

cell_fw = tf.contrib.rnn.LSTMCell(hidden_size)
cell_bw = tf.contrib.rnn.LSTMCell(hidden_size)

(self.out_fw, self.out_bw), _ = tf.nn.bidirectional_dynamic_rnn(cell_fw, cell_bw, input, ...)

To turn this into a multi-layer, I suspect I can't just wrap a few LSTMCellwith the MultiRNNCellfollowing way:

multi_cell_fw = tf.contrib.rnn.MultiRNNCell([cell_fw] * num_layers, ...)

and feed them in bidirectional_dynamic_rnn, since both LSTMs in the forward and reverse directions in each layer need to exit both the forward and reverse directions of the previous layer. My current solution is to create mine bidirectional_dynamic_rnnin a loop by feeding the concatenated LSTM output of the previous layers.

However, this is not very clean and, frankly, I'm not sure if this is correct, although it works on a set of toys. Is there a better way that is relatively elegant to use something like MultiRNNCell?

I am using Tensorflow API r1.0.

Answer 1

Just do:

multi_cell_fw = tf.contrib.rnn.MultiRNNCell([cell_fw for _ in range(num_layers)], ...)

That should work.