Error calculating resume in TensorFlow

Question

I am trying to use TensorFlow to create summaries and visualize them using TensorBoard. However, I get an error ( InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder_1' with dtype float ), which I do not understand.

This is the complete source of my program:

 from tensorflow.examples.tutorials.mnist import input_data mnist = input_data.read_data_sets("MNIST_data/", one_hot=True) import tensorflow as tf x = tf.placeholder(tf.float32, [None, 784]) W = tf.Variable(tf.zeros([784, 10])) b = tf.Variable(tf.zeros([10])) y = tf.nn.softmax(tf.matmul(x, W) + b) _ = tf.histogram_summary("weights", W) _ = tf.histogram_summary("biases", b) _ = tf.histogram_summary("y", y) y_ = tf.placeholder(tf.float32, [None, 10]) with tf.name_scope("xent") as scope: cross_entropy = -tf.reduce_sum(y_*tf.log(y)) _ = tf.scalar_summary("cross entropy", cross_entropy) with tf.name_scope("train") as scope: train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy) init = tf.initialize_all_variables() sess = tf.Session() sess.run(init) with tf.name_scope("test") as scope: correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1)) accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float")) _ = tf.scalar_summary("accuracy", accuracy) merged = tf.merge_all_summaries() writer = tf.train.SummaryWriter("/tmp/mnist_nn", sess.graph_def) for i in range(1000): if (i % 10) == 0: feed = {x: mnist.test.images, y_: mnist.test.labels} result = sess.run([merged, accuracy], feed_dict=feed) summary_str = result[0] acc = result[1] print("Accuracy at step %s: %s" % (i, acc)) else: batch_xs, batch_ys = mnist.train.next_batch(100) feed = {x: batch_xs, y_: batch_ys} sess.run(train_step, feed_dict=feed) print(accuracy.eval({x: mnist.test.images, y_: mnist.test.labels})) 

However, when I try to run the above code, the following error occurs:

 --------------------------------------------------------------------------- InvalidArgumentError Traceback (most recent call last) <ipython-input-23-584a7bc91816> in <module>() 39 if (i % 10) == 0: 40 feed = {x: mnist.test.images, y_: mnist.test.labels} ---> 41 result = sess.run([merged, accuracy], feed_dict=feed) 42 summary_str = result[0] 43 acc = result[1] /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow/python/client/session.pyc in run(self, fetches, feed_dict) 366 367 # Run request and get response. --> 368 results = self._do_run(target_list, unique_fetch_targets, feed_dict_string) 369 370 # User may have fetched the same tensor multiple times, but we /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tensorflow/python/client/session.pyc in _do_run(self, target_list, fetch_list, feed_dict) 442 # pylint: disable=protected-access 443 raise errors._make_specific_exception(node_def, op, error_message, --> 444 e.code) 445 # pylint: enable=protected-access 446 six.reraise(e_type, e_value, e_traceback) InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder_1' with dtype float [[Node: Placeholder_1 = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]] Caused by op u'Placeholder_1', defined at: 

It seems like an error that the placeholder in my source did not get the corresponding value. As far as I can tell, I am loading values ​​for all placeholders ( x and y_ ).

If you need, I will add a complete log to this issue.

I also found that when I first select mnist, it works (with the following output), but it does not create a TensorBoard render:

 Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes. Extracting MNIST_data/train-images-idx3-ubyte.gz Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes. Extracting MNIST_data/train-labels-idx1-ubyte.gz Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes. Extracting MNIST_data/t10k-images-idx3-ubyte.gz Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes. Extracting MNIST_data/t10k-labels-idx1-ubyte.gz Tensor("MergeSummary/MergeSummary:0", shape=TensorShape([]), dtype=string) merged Accuracy at step 0: 0.098 Accuracy at step 10: 0.7404 Accuracy at step 20: 0.8041 Accuracy at step 30: 0.814 ... 

Answer 1

From your error message, it looks like you are using IPython. One mistake when using IPython to create the TensorFlow model is that functions like tf.merge_all_summaries() will remember every resume created in the current session, including cells that are not in error. This is the result of TensorFlow using the default graph to collect all the operations, summaries, etc. that are created in the process unless you explicitly specify the graph. I suspect that your call to tf.merge_all_summaries() returns more than three histogram summaries that you created in the code, while older ones tf.merge_all_summaries() dependency on a previously created placeholder.

There are two main ways you could fix it. The simplest thing is to explicitly merge the resume, rather than using tf.merge_all_summaries() :

 weights_summary = tf.histogram_summary("weights", W) biases_summary = tf.histogram_summary("biases", b) y_summary = tf.histogram_summary("y", y) merged = tf.merge_summary([weights_summary, biases_summary, y_summary]) 

An alternative would be to set an explicit default schedule before building your model. This is inconvenient if you want to split your model into several IPython cells, but should also work:

 # Sets a new default graph, and stores it in `g`. with tf.Graph().as_default() as g: x = tf.placeholder(tf.float32, [None, 784]) # ... # Summaries are added to `g`. _ = tf.histogram_summary("weights", W) _ = tf.histogram_summary("biases", b) _ = tf.histogram_summary("y", y) # ... # `merged` contains only summaries from `g`. merged = tf.merge_all_summaries() # ... 

Answer 2

for keras users

You may encounter this error when using the TensorBoard callback and installing a new model from scratch. In this case, the solution is to call

 from keras import backend as K K.clear_session() 

before creating a new model. checked with keras 2.1.5 and tensorflow 1.6.0