ValueError: attempt to reuse RNNCell with a different scope of variables than its first use

Question

The following code snippet

import tensorflow as tf from tensorflow.contrib import rnn hidden_size = 100 batch_size = 100 num_steps = 100 num_layers = 100 is_training = True keep_prob = 0.4 input_data = tf.placeholder(tf.float32, [batch_size, num_steps]) lstm_cell = rnn.BasicLSTMCell(hidden_size, forget_bias=0.0, state_is_tuple=True) if is_training and keep_prob < 1: lstm_cell = rnn.DropoutWrapper(lstm_cell) cell = rnn.MultiRNNCell([lstm_cell for _ in range(num_layers)], state_is_tuple=True) _initial_state = cell.zero_state(batch_size, tf.float32) iw = tf.get_variable("input_w", [1, hidden_size]) ib = tf.get_variable("input_b", [hidden_size]) inputs = [tf.nn.xw_plus_b(i_, iw, ib) for i_ in tf.split(input_data, num_steps, 1)] if is_training and keep_prob < 1: inputs = [tf.nn.dropout(input_, keep_prob) for input_ in inputs] outputs, states = rnn.static_rnn(cell, inputs, initial_state=_initial_state) 

produces the following error:

ValueError: attempt to reuse RNNCell < tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl.BasicLSTMCell object in 0x10210d5c0> with a different scope of variables than its first use. The first use of the cell was with the scope 'rnn/multi_rnn_cell/cell_0/basic_lstm_cell' , this attempt has the scope 'rnn / multi_rnn_cell / cell_1 / basic_lstm_cell``.

Create a new cell instance if you want it to use a different set of weights.

If before use: MultiRNNCell([BasicLSTMCell(...)] * num_layers) , change to: MultiRNNCell([BasicLSTMCell(...) for _ in range(num_layers)]) .

If before you used the same cell instance as the forward and backward cells of a bidirectional RNN, just create two instances (one for the forward, one for the reverse).

In May 2017, we will begin the transition to this cell behavior in order to use the existing stored weights, if any, when it is called using scope=None (which can lead to degradation of the noiseless model, so this error will remain until then.)

How to solve this problem?

My version of Tensorflow 1.0.

Answer 1

As stated in the comments, my solution is:
change of this

 cell = tf.contrib.rnn.LSTMCell(state_size, state_is_tuple=True) cell = tf.contrib.rnn.DropoutWrapper(cell, output_keep_prob=0.8) rnn_cells = tf.contrib.rnn.MultiRNNCell([cell for _ in range(num_layers)], state_is_tuple = True) outputs, current_state = tf.nn.dynamic_rnn(rnn_cells, x, initial_state=rnn_tuple_state, scope = "layer") 

in

 def lstm_cell(): cell = tf.contrib.rnn.LSTMCell(state_size, reuse=tf.get_variable_scope().reuse) return tf.contrib.rnn.DropoutWrapper(cell, output_keep_prob=0.8) rnn_cells = tf.contrib.rnn.MultiRNNCell([lstm_cell() for _ in range(num_layers)], state_is_tuple = True) outputs, current_state = tf.nn.dynamic_rnn(rnn_cells, x, initial_state=rnn_tuple_state) 

which seems to solve the reuse issue. I basically don’t understand the main problem, but this solved the problem for me on TF 1.1rc2
Hooray!