Tensor Flow: Using Parameter Servers in a Distributed Workout

Question

It is not entirely clear how parameter servers know what to do in distributed training with tensor flow.

For example, in this SO> question , the following code is used to configure parameter server tasks and work tasks:

if FLAGS.job_name == "ps":
    server.join()
elif FLAGS.job_name == "worker":
    ##some training code

How does it server.join()indicate that this task should be a parameter server? Is the parameter serving default behavior for tasks? Is there anything else you can / should specify a parameter maintenance task?

Change . This SO> question touches on some of my questions: "The logic there ensures that Variable objects are provided equally to the workers, acting as a parameter server." But how does the parameter server know that it is a parameter server? Is it enough server.join()?

Answer 1

TL; DR: TensorFlow " ", . , "/job:ps", . , train_op, "/job:ps", .

server.join() TensorFlow , ( , , , ).

PS , ... ## some training code. , TensorFlow , . with tf.device(tf.train.replica_device_setter(...)): TensorFlow PS, "/job:ps/task:{i}" ( {i}, ).

sess.run(train_op), TensorFlow , , , . "/job:ps", .