TensorFlow: tf.layers vs. low-level API

Question

I am currently planning my first Conv. NN at Tensorflow, and read the many educational materials available on the Tensorflow website for understanding.

There seem to be two ways to create custom CNN:

1) Use the Tensorflow tf.layers layer module, which is a "high level API". Using this method, you define a model definition function consisting of tf.layers objects, and in the main function, create an instance of tf.learn.Estimator , passing it the model definition function. Here, the fit() and evaluate() methods can be called on an Estimator object, which trains and validates accordingly. Link: https://www.tensorflow.org/tutorials/layers . The main function below:

 def main(unused_argv): # Load training and eval data mnist = learn.datasets.load_dataset("mnist") train_data = mnist.train.images # Returns np.array train_labels = np.asarray(mnist.train.labels, dtype=np.int32) eval_data = mnist.test.images # Returns np.array eval_labels = np.asarray(mnist.test.labels, dtype=np.int32) # Create the Estimator mnist_classifier = learn.Estimator( model_fn=cnn_model_fn, model_dir="/tmp/mnist_convnet_model") # Set up logging for predictions # Log the values in the "Softmax" tensor with label "probabilities" tensors_to_log = {"probabilities": "softmax_tensor"} logging_hook = tf.train.LoggingTensorHook( tensors=tensors_to_log, every_n_iter=50) # Train the model mnist_classifier.fit( x=train_data, y=train_labels, batch_size=100, steps=20000, monitors=[logging_hook]) # Configure the accuracy metric for evaluation metrics = { "accuracy": learn.MetricSpec( metric_fn=tf.metrics.accuracy, prediction_key="classes"), } # Evaluate the model and print results eval_results = mnist_classifier.evaluate( x=eval_data, y=eval_labels, metrics=metrics) print(eval_results) 

Full code here

---

2) Use the Tensorflow "low level API" in which layers are defined in the definition function. Here, the levels are determined manually, and the user must perform many calculations manually. In the main function, the user runs tf.Session() and manually configures training / validation using for the loop (s). Link: https://www.tensorflow.org/get_started/mnist/pros . The main function below:

 def main(_): # Import data mnist = input_data.read_data_sets(FLAGS.data_dir, one_hot=True) # Create the model x = tf.placeholder(tf.float32, [None, 784]) # Define loss and optimizer y_ = tf.placeholder(tf.float32, [None, 10]) # Build the graph for the deep net y_conv, keep_prob = deepnn(x) with tf.name_scope('loss'): cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y_conv) cross_entropy = tf.reduce_mean(cross_entropy) with tf.name_scope('adam_optimizer'): train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy) with tf.name_scope('accuracy'): correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y_, 1)) correct_prediction = tf.cast(correct_prediction, tf.float32) accuracy = tf.reduce_mean(correct_prediction) graph_location = tempfile.mkdtemp() print('Saving graph to: %s' % graph_location) train_writer = tf.summary.FileWriter(graph_location) train_writer.add_graph(tf.get_default_graph()) with tf.Session() as sess: sess.run(tf.global_variables_initializer()) for i in range(20000): batch = mnist.train.next_batch(50) if i % 100 == 0: train_accuracy = accuracy.eval(feed_dict={ x: batch[0], y_: batch[1], keep_prob: 1.0}) print('step %d, training accuracy %g' % (i, train_accuracy)) train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5}) print('test accuracy %g' % accuracy.eval(feed_dict={ x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0})) 

Full code here

---

My dilemma: I like the simplicity of defining a neural network using tf.layers (option 1), but I want the training customization to be implemented with a "low-level API" (option 2). In particular, when using the tf.layers implementation, tf.layers there a way to report the accuracy of the verification of each nth learning iteration? Or, more generally, can I train / check with tf.Session() , or am I limited to using the tf.learn.Estimator fit() and evaluate() methods?

It seems strange that it would be necessary to get a final grade point after all the training has been completed, since I thought that the whole point of verification is to track the development of the network during the training. Otherwise, what's the difference between validation and testing?

Any help would be appreciated.

Answer 1

You are almost right, however tf.layers is separate from the Estimator function class, etc. If you wanted you to be able to use tf.Layers to define your layers, but then create your own training cycles or something else that you like, you can think of tf.layers just about the functions that you could create in the second version above.

If you are interested in quickly creating a basic model, but are able to expand it with other functions, your own training cycles, etc., then there is no reason why you cannot use layers to create your model and interact with her no matter how you wish.

tf.layers - https://www.tensorflow.org/api_docs/python/tf/layers

tf.Estimator - https://www.tensorflow.org/api_docs/python/tf/estimator