In the tensor flow, what is the difference between the trainee and the stop gradient

Question

I would like to know the difference between the trainable=Falseand option tf.stop_gradient(). If I make an option trainable False, my optimizer will not consider a variable for training? Does this parameter make it a constant value during training?

Answer 1

trainee = False

Here the value of the variable will be constant during training. The optimizer will not consider this variable for training, without updating the gradient.

stop_gradient

op , ; . trinable=False, .

stop_gradient ops; op , .

y1 = tf.stop_gradient(W1x+b1)
y2 = W2y1+b2
cost = cost_function(y2, y)
# this following op wont optimize the cost with respect to W1 and b1
train_op_w2_b2 = tf.train.MomentumOptimizer(0.001, 0.9).minimize(cost)

W1 = tf.get_variable('w1', trainable=False)
y1 = W1x+b1
y2 = W2y1+b2
cost = cost_function(y2, y)
# this following op wont optimize the cost with respect to W1
train_op = tf.train.MomentumOptimizer(0.001, 0.9).minimize(cost)