Are there any disadvantages of creating TensorFlow placeholders for variable size and fixed size inputs?

Question

I am wondering if there are significant drawbacks (for example, regarding computational efficiency, memory ...) when creating a TensorFlow placeholder for variable size inputs (compared to fixed sizes)?

Let's say I'm doing mini-batch training and initializing the schedule with a placeholder, where I assume a fixed batch_size pre-package:

tf.placeholder(..., shape=[batch_size, ...]) 

Alternatively, I can initialize the placeholder variable so that it accepts inputs with a variable size:

 tf.placeholder(..., shape=[None, ...]) 

I am not familiar with the implementation of the low-level tensor level under the hood, but should the latter check sizes, allocate memory and create new arrays at each iteration to take into account the case when the size of my mini-camera changes during training? So, depending on the implementation, would it be computationally wasteful if I work with a fixed lot size?

Answer 1

There is a certain tension between the provision of fully defined forms (which can lead to increased productivity) and allows resizing (which simplifies the reuse of data flow graphs). As you suspect, there are some drawbacks to using the tf.placeholder() variables to represent input in the TensorFlow model:

• TensorFlow can often simplify the flow chart when the forms are fully known. For example, calling tf.shape(x) returns tf.Tensor containing the true dynamic form of the tensor x . If this shape is fully defined at the time of plotting, TensorFlow will replace the shape calculation with tf.constant() , and this will be used with constant folding optimization to reduce the amount of work performed at runtime.

• As a last resort , the XLA compiler requires that all forms of the input tensors be fully defined before the code is generated, can generate much more efficient kernel code, where the boundaries of the array (etc.) are compile-time constants. XLA recompiles the kernel code for each combination of input forms, so using fixed size tensors avoids recompilation.

• The TensorFlow memory allocator currently allocates new arrays for each intermediate and output tensors with every call to tf.Session.run() . However, basic memory allocators (a BFC allocator for GPU memory and tcmalloc or jemalloc for CPU memory) tend to work better if they have a static allocation of allocation requests (since requests can be satisfied from buffers that were recently freed).