Can we see the send and receive nodes in the tensor flow of GraphDef?

Question

Is it possible to see the send and receive nodes in the graph strain GraphDef or using the python API?

I try the following code

import tensorflow as tf

with tf.device("/gpu:0"):
    x = tf.constant(1.0)
with tf.device("/gpu:1"):
    y = tf.constant(2.0)
with tf.device("/cpu:0"):
    sum = tf.add(x, y)

graph_def = tf.get_default_graph().as_graph_def()
print(graph_def)

But there are no send / return nodes in graph_def. Are there any send / return nodes added to the schedule for transfer xand yto cpu?

Answer 1

The send and return nodes are added only to the schedule on the first attempt to execute the schedule on a call tf.Session.run()... and, indeed, the set of send and recv nodes to add will depend on the specific tensors that you submit and select in this call.

, , , tf.RunOptions(output_partition_graphs=True) Session.run() :

options = tf.RunOptions(output_partition_graphs=True)
metadata = tf.RunMetadata()

sess.run(..., options=options, metadata=metadata)

for partition_graph_def in metadata.partition_graphs:
    print partition_graph_def  # Contains all the nodes that ran on a single device.