How to do maximum merge on one-dimensional ConvNet (conv1d) in TensowFlow?

Question

I train a convolutional neural network in text (character level) and I want to do max-pooling. tf.nn.max_pool expects rank 4 of Tensor, but 1st convnets is rank 3 in tensorflow ([batch, width, depth]), so when I pass the output of conv1d to the maximum pool function, this is an error:

ValueError: Shape (1, 144, 512) must have rank 4 

I am new to tensor flow and deep learning in general and would like to consult with the best practices here because I can imagine that there are many workarounds. How can I do the maximum join in the 1st case?

Thanks.

Answer 1

A quick way is to add an extra singleton size, i.e. create a form (1, 1, 144, 512), from there you can reduce it with tf.squeeze.

I'm still interested in learning about other approaches.

Answer 2

Another option is to use the tf.nn.pool function. Assuming the input is of the form [batch_size, length, num_channels], the function can be used as:

 B = tf.nn.pool(A, [size], 'MAX', 'SAME', strides = [stride]) 

As an example, the matrix of the form [8, 10, 1] is transformed into [8, 2, 1] using the maximum union.

 [[[0, 1, ... 9]], [[[4, 9]], [[0, 1, ..., 9]], ---> [[4, 9]], ..., ..., [[0, 1, ..., 9]]] [[4, 9]]] 

Python Code:

 import numpy as np import tensorflow as tf X = np.tile(np.arange(10), 8).reshape(8, 10, 1) A = tf.Variable(X) B = tf.nn.pool(A, [5], 'MAX', 'SAME', strides = [5]) sess = tf.Session() init = tf.global_variables_initializer() sess.run(init) sess.run([B]) C = B.eval(session = sess) print(C)