What is the difference between tf.initialize_all_variables () and tf.initialize_local_variables ()?

Question

I am looking at the code in this example: fully_connected_reader.py

I am confused with lines 147 and 148:

init_op = tf.group(tf.initialize_all_variables(), tf.initialize_local_variables()) 

I do not know which variables are all variables and which are local variables . Any ideas?

Answer 1

tf.initialize_all_variables() - is a shortcut for tf.initialize_variables(tf.all_variables()) , tf.initialize_local_variables() - is a shortcut for tf.initialize_variables(tf.local_variables()) and initializes variables GraphKeys.VARIABLES in GraphKeys.VARIABLES GraphKeys.LOCAL_VARIABLE collections, respectively.

Variables in the GraphKeys.LOCAL_VARIABLES collection are variables that are added to the chart but are not saved or restored ( source ).

tf.Variable() tf.Variable() default, tf.Variable() adds a new variable to the GraphKeys.VARIABLE collection, which can be controlled using the collection = argument.

Answer 2

A local variable in TF is any variable that was created using collections=[tf.GraphKeys.LOCAL_VARIABLES] . For instance:

 e = tf.Variable(6, name='var_e', collections=[tf.GraphKeys.LOCAL_VARIABLES]) 

LOCAL_VARIABLES: a subset of Variable objects that are local to each machine. Commonly used for temporary variables such as counters. Note: use tf.contrib.framework.local_variable to add to this collection.

Usually they are not saved / not restored to the control point and are not used for temporary or intermediate values. For a more detailed answer, see here .

A global variable is basically any other variable that you initialize.

---

In the new version of TF, you should use tf.global_variables_initializer() , tf.local_variables_initializer() because the previous functions were deprecated.

Answer 3

• GLOBAL_VARIABLES

The key is to collect variable objects that are global (shared on machines). The default collection for all variables except local ones.

• LOCAL_VARIABLES

The key is to collect local variables that are local to the machine and are not saved / not restored.