Dynamic size for tf.zeros () (for use with placeholders with None sizes)

Question

Consider the following code:

x = tf.placeholder("float", shape=[42, 4]) y = tf.zeros([42, 4], "float") xy_stacked = tf.concat(1, [x, y]) print(x.get_shape()) print(y.get_shape()) print(xy_stacked.get_shape()) 

This will result in the following result:

 TensorShape([Dimension(42), Dimension(4)]) TensorShape([Dimension(42), Dimension(4)]) TensorShape([Dimension(42), Dimension(8)]) 

However, what if the placeholder has a dynamic dimension, which is determined at run time by the value passed to feed_dict= how often the placeholder does this:

 x = tf.placeholder("float", shape=[None, 4]) y = tf.zeros([None, 4], "float") xy_stacked = tf.concat(1, [x, y]) 

This will result in an error for tf.zeros([None, 4], "float") . Apparently Dimension(None) not allowed for tf.zeros :

 TypeError Traceback (most recent call last) <ipython-input-24-277eca38a392> in <module>() 2 3 x = tf.placeholder("float", shape=[None, 4]) ----> 4 y = tf.zeros([None, 4], "float") 5 xy_stacked = tf.concat(1, [x, y]) 6 [...] /usr/local/lib/python3.4/dist-packages/numpy/core/_methods.py in _prod(a, axis, dtype, out, keepdims) 33 34 def _prod(a, axis=None, dtype=None, out=None, keepdims=False): ---> 35 return umr_prod(a, axis, dtype, out, keepdims) 36 37 def _any(a, axis=None, dtype=None, out=None, keepdims=False): TypeError: unsupported operand type(s) for *: 'NoneType' and 'int' 

I found out that this does not cause an error if I set the first measurement of my tensor of zero value to non-None, for example 1:

 x = tf.placeholder("float", shape=[None, 4]) y = tf.zeros([1, 4], "float") xy_stacked = tf.concat(1, [x, y]) 

but then the resulting xy_stacked tensor xy_stacked truncated to this size:

 TensorShape([Dimension(None), Dimension(4)]) TensorShape([Dimension(1), Dimension(4)]) TensorShape([Dimension(1), Dimension(8)]) 

How can I apply a placeholder tensor to zeros to get the tensor of the form TensorShape([Dimension(None), Dimension(8)]) in this example?

The only “solutions” I have found so far are something like the following:

 x = tf.placeholder("float", shape=[None, 4]) y = 0 * x xy_stacked = tf.concat(1, [x, y]) 

Or simply declaring y as a placeholder and always passing a zero array of the right size.

But none of them looks like a pure solution to the problem, and such hacks so quickly get out of hand in an application that is more complex than this simple example.

I am using tensorflow-0.6.0-py3 .

Answer 1

The recommended way to create a null tensor with the same shape as another tensor is to use tf.zeros_like() op:

 x = tf.placeholder(tf.float32, shape=[None, 4]) y = tf.zeros_like(x) 

The resulting tensor y seems to have the form [None, None] according to Tensor.get_shape() , but at runtime it will expand to the same shape as x :

 print y.get_shape() # ==> TensorShape([Dimension(None), Dimension(None)]) sess = tf.Session() y_result = sess.run(y, feed_dict={x: np.random.rand(4, 4)}) print y_result.shape # ==> (4, 4) 

The static form [None, None] returned because the form output was not specialized for tf.zeros_like() . I have logged a GitHub issue for this , and this should be fixed soon.

---

EDIT: In your comment, you asked how to deal with the case where the zero tensor was based on, but different from the original tensor. It is also possible using tf.shape() and tf.stack() to create dimensions and tf.fill() to create a null tensor:

 x = tf.placeholder(tf.float32, shape=[None, 4]) # Use tf.shape() to get the runtime size of `x` in the 0th dimension. zeros_dims = tf.stack([tf.shape(x)[0], 7]) y = tf.fill(zeros_dims, 0.0) sess = tf.Session() y_result = sess.run(y, feed_dict={x: np.random.rand(4, 4)}) print y_result.shape # ==> (4, 7)