How can I reuse the dense layer?

Question

I have a network in Tensorflow, and I want to define a function that passes it through the tf.layers.dense layer (obviously the same one). I see the reuse argument, but to use it correctly, it seems to me that I need to save the global variable in order to remember whether my function has already been called. Is there a cleaner way?

Answer 1

As far as I know, there is no cleaner way. The best thing we can do is wrap tf.layers.dense in our abstraction and use it as an object, hiding the scope variable variable trunk:

 def my_dense(*args, **kwargs): scope = tf.variable_scope(None, default_name='dense').__enter__() def f(input): r = tf.layers.dense(input, *args, name=scope, **kwargs) scope.reuse_variables() return r return f a = [[1,2,3], [4,5,6]] a = tf.constant(a, dtype=tf.float32) layer = my_dense(3) a = layer(a) a = layer(a) print(*[[int(a) for a in v.get_shape()] for v in tf.trainable_variables()]) # Prints: "[3, 3] [3]" (one pair of (weights and biases)) 

Answer 2

You can create a layer with a constant of the desired size and ignore the result.

Thus, a variable is declared, but the operation should be shortened from the graph.

for instance

 tf.layers.dense(tf.zeros(1, 128), 3, name='my_layer') ... later hidden = tf.layers.dense(input, 3, name='my_layer', reuse=True) 

Answer 3

I find tf.layers.Dense more cleanly than the answers above. All you need is a dense object defined in advance. Then you can reuse it as many times as you like.

 import tensorflow as tf # Define Dense object which is reusable my_dense = tf.layers.Dense(3, name="optional_name") # Define some inputs x1 = tf.constant([[1,2,3], [4,5,6]], dtype=tf.float32) x2 = tf.constant([[4,5,6], [7,8,9]], dtype=tf.float32) # Use the Dense layer y1 = my_dense(x1) y2 = my_dense(x2) with tf.Session() as sess: sess.run(tf.global_variables_initializer()) y1 = sess.run(y1) y2 = sess.run(y2) print(y1) print(y2) 

In fact, the tf.layers.dense function internally creates a Dense object and passes your input to that object. Check the code for more information.