Tensorflow "knows" when not to put data in the GPU?

Question

I tried to use the tensogram along with the tensor flow, and I did this as a setup:

rand = tf.placeholder(dtype=tf.float32)    # this will be visualised in tensorboard later on 
tf.summary.image('random_noise_visualisation', rand,max_outputs=5)
merged_summary_op = tf.summary.merge_all() # to me this seems like a helper to 
                                           # merge all tensorboard related operations

Then I evaluate my own merged_summary_opand give it a very large array about 1 GB in size.

It does not seem to use the extra GPU memory from the memory already in use.

I also just tried to evaluate my place rand, thinking that perhaps the summary operators have special processing to prevent data transfer to the GPU. I did:

random_value = np.random.randn(3000,224,224,1)
sess.run(rand,feed_dict={rand:random_value})

Again, there was no use of the GPU.

However when i do

sess.run(rand + 2 ,feed_dict={rand:random_value}) # forced to do some calculation

There is additional use of the GPU with an increase of approximately 1 GB.

for all of the above experiments, I used my session as:

sess = tf.InteractiveSession(graph=tf.Graph())

My questions:

• Tensorflow, GPU?
• Interactive session ?
• - ?

Answer 1

Tensorflow, GPU?

.

, rand - , rand feed_dict. session.py:

self._final_fetches = [x for x in self._fetches if x not in feeds]

... :

# We only want to really perform the run if fetches or targets are provided,
# or if the call is a partial run that specifies feeds.
if final_fetches or final_targets or (handle and feed_dict_tensor):
  results = self._do_run(handle, final_targets, final_fetches,
                         feed_dict_tensor, options, run_metadata)
else:
  results = []

---

, . Tensorflow , , , .

, log_device_placement=True:

with tf.Session(config=tf.ConfigProto(log_device_placement=True)) as sess:
  random_value = np.random.randn(300,224,224,1)
  print(sess.run(rand + 2, feed_dict={rand: random_value}).shape)

---

, : ImageSummary op GPU. (core/kernels/summary_image_op.cc):

REGISTER_KERNEL_BUILDER(Name("ImageSummary").Device(DEVICE_CPU),
                        SummaryImageOp);

, CPU , session.run() :

# THIS FAILS!
with tf.device('/gpu:0'):
  tf.summary.image('random_noise_visualisation', rand,max_outputs=5)
  merged_summary_op = tf.summary.merge_all() # to me this seems like a helper to
                                             # merge all tensorboard related operations

, - -.

ImageSummary , , . GitHub , , GPU, , .

, tensorflow , , GPU , GPU CPU.

?

. InteractiveSession . , InteractiveSession , Session with.

- ?

, , , . .