How to calculate the GPU memory needed to run a model in TensorFlow?

Question

Is there an easy way to find the GPU memory consumed by, say, the original-resnet-v2 model, which is initialized in the tensor stream? This includes the output and backprop memories required.

Answer 1

Since using gpu.options.allow_growth and gpu_options.per_process_gpu_memory_fraction to estimate model size is currently a trial and error and tedious solution, I suggest using tf.RunMetadata() in combination with a tensogram.

Example:

 run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE) run_metadata = tf.RunMetadata() summary, _ = sess.run(train_step, feed_dict, options=run_options, run_metadata=run_metadata) train_writer.add_run_metadata(run_metadata, 'step%d' % i) 

Run your model and tensogram, go to the desired part of the graph and read the node statistics.

Source: https://www.tensorflow.org/get_started/graph_viz

Answer 2

You can explicitly calculate the memory needed to store the parameters, but I'm afraid it will be difficult to calculate the size of all the buffers needed for training. Probably a smarter way would be to make TF for you. Set the gpu_options.allow_growth configuration gpu_options.allow_growth to True and see how much it consumes. Another option is to try lower values ​​for gpu_options.per_process_gpu_memory_fraction until it finishes with out of memory.