How to set weight strength in TensorFlow?

Question

I am trying to use TensorFlow with my deep learning project.

When I use Momentum Gradient Descent, how is weight determined?
(Λ in this formula .)

Answer 1

The term for cost / weight decay is not part of the optimizers in TensorFlow.

It is easy to include, however, adding an additional penalty to the cost function with the loss of L2 on the scales:

C = <your initial cost function> l2_loss = tf.add_n([tf.nn.l2_loss(v) for v in tf.trainable_variables()]) C = C + lambda * l2_loss 

tf.nn.l2_loss(v) link is just 0.5 * tf.reduce_sum(v * v) , and the gradients for the individual weights will be lambda * w , which should be equivalent to your bound equation.