TensorFlow LSTM Generator Model

Question

I am working on a tutorial on the LSTM language model, which was discussed here .

With language models, a model is usually used to create a new sentence from scratch after training (i.e. a sample from the model).

I'm new to TensorFlow, but I'm trying to use my trained model to generate new words to the end of sentence marker.

My initial attempt:

x = tf.zeros_like(m.input_data) state = m.initial_state.eval() for step in xrange(m.num_steps): state = session.run(m.final_state, {m.input_data: x, m.initial_state: state}) x = state 

Error with error:

ValueError: setting an array element with a sequence.

Answer 1

The problem here looks like m.input_data: x mapping in feed_dict past session.run() . In this case, TensorFlow expects x be a numpy array (or some object that can be implicitly converted to a numpy array), but this is a TensorFlow Tensor value (the result is tf.zeros_like() ).

Fortunately, the solution is simple. Replace x = tf.zeros_like(m.input_data) following text:

 x = tf.zeros_like(m.input_data).eval() 

... which ensures that x converted to a numpy array.

(Note that a more direct way to achieve this would be to create an initial x as a numpy array of the appropriate size.)