Is there a difference between a placeholder and a variable when the model is not built?

Question

I am trying to understand the difference between placeholder and variable in TensorFlow:

X = tf.placeholder("float") W = tf.Variable(rng.randn(), name="weight") 

I also read the stack overflow question below. I understand the difference when they introduce the model.

InvalidArgumentError: you must specify a value for the Placeholder tender placeholder

However, generally speaking, if we are not building a model, is there a difference between tf.placeholder() and tf.Variable() ?

Answer 1

Aggregate

A placeholder is used to feed external data into a Tensorflow calculation (material off-schedule). Here is some documentation: ( https://www.tensorflow.org/versions/r0.10/how_tos/reading_data/#feeding )

The TensorFlow feed mechanism allows you to enter data into any tensor in a computational graph. Thus, python calculation can directly transfer data to the graph.

I personally would make an analogy with placeholders for reading from standard input.

 x = raw_input() X = tf.placeholder("float") 

When you read from standard input, you need to "enter data" from an external source. Same thing with placeholder. It allows you to "enter data" that are external to the calculation schedule.

If you are learning a learning algorithm, a clear example of using placeholder is the transfer of your training data. Training data is not stored in the calculation schedule. How are you going to get on the schedule? By entering it through the placeholder. The placeholder basically you say on the chart "I don’t have this for you yet, but I will ask you when I ask you to run."

Variables

A variable is used to store the state on your chart. This requires an initial value. One use case may represent the weight characteristics of a neural network or something similar. Here's the documentation: ( https://www.tensorflow.org/versions/r0.10/api_docs/python/state_ops/variables#Variable )

The variable maintains state in the graph between run () calls. You add a variable to the graph by building an instance of the Variable class.

The constructor Variable () requires an initial value for the variable, which can be a tensor of any type and shape. The initial value determines the type and shape of the variable. After construction, the type and shape of the variable is fixed. The value can be changed using one of the assignment methods.

I personally would make an analogy between Tensorflow Variables and assigning a variable in Python to anything that does not depend on external material. For instance,

 # Tensorflow: W = tf.Variable(rng.randn(), name="weight") # Standard python: w = 5 w = "hello" w = [1, 2, 3, 4, 5] 

W represents some result of your calculation. Just as you must initialize all your variables in Python (you cannot just run the x command, you must say x = ...something... ), you must initialize all Variable objects in Tensorflow.

Variable vs Placeholder

In my opinion, there is not much connected between tf.Variable and tf.placeholder . You use Variable if you need to save state. You use a placeholder if you need to enter external data.

If you are not building a model, you should still use tf.placeholder if you want to insert external data that you do not have to use while you define the schedule. If you are not building a model, you still need tf.Variable if you want to save some result of your calculations during the execution of the graph.

Why both?

I am not an expert in Tensorflow, so I can only guess why there are both in the design.

The big difference between placeholders and variables is that placeholders can be of variable size, but the tf.Variable form must be specified tf.Variable .

Substitute variables for font size: maybe I only want to enter a batch of size 5 right now, but maybe I want to increase the batch size later. Perhaps I do not know in advance how many training examples I will receive.

Variable size variables do not make sense: tf.Variable contains the extracted parameters of your model, and the number of parameters should not change. In addition, Tensorflow extends to distributed computing. If you had Variables , the shape of which changed during the calculations, it would be very difficult to correctly distribute it among 1000 computers.

Usually you create a model, and all parameters are known in advance, so it is possible that tf.Variable used for presentation. tf.placeholder is probably for everything else outside of your model (or calculation graph) and therefore can be more flexible.

Answer 2

The most obvious difference between tf.Variable and tf.placeholder is that

---

you use variables to store and update parameters. Buffer variables in memory containing tensors. They must be explicitly initialized and can be saved to disk during and after training. You can later restore the stored values ​​for implementation or analysis of the model.

Variables are initialized using sess.run(tf.global_variables_initializer()) . In addition, when creating a variable, you need to pass the Tensor value as its initial value to the Variable () constructor, and when you create a variable, you always know its shape.

---

On the other hand, you cannot update the placeholder. They also should not be initialized, but since they promise to have a tensor, you need to pass the value to them sess.run(<op>, {a: <some_val>}) . And finally, compared to a variable, the placeholder may not know the form. You can either provide parts of the measurements or show nothing.

---

Other differences:

• the values ​​inside the variable can be updated during optimization
• variables may be shared and may not be trained
• values ​​inside a variable can be saved after training
• when a variable is created, 3 ops are added to the graph (variable op, initializer op, ops for the initial value)
• placeholder is a function, a variable is a class (hence the upper case)
• when you use TF in a distributed environment, the variables are stored in a special place ( parameter server ) and distributed between workers.

The interesting part is that you can fill in more than just placeholders. You can pass the value of a variable and even a constant.