Tensorflow: ValueError: The form should be rank 2, but rank 3

I am new to tensorflow and I am trying to update some code for bidirectional LSTM from the old version of tensorflow to the newest (1.0), but I get this error:

The form must be of rank 2, but it is of rank 3 for "MatMul_3" (op: "MatMul") with input forms: [100,?, 400], [400,2].

Error on pred_mod.

_weights = { # Hidden layer weights => 2*n_hidden because of foward + backward cells 'w_emb' : tf.Variable(0.2 * tf.random_uniform([max_features,FLAGS.embedding_dim], minval=-1.0, maxval=1.0, dtype=tf.float32),name='w_emb',trainable=False), 'c_emb' : tf.Variable(0.2 * tf.random_uniform([3,FLAGS.embedding_dim],minval=-1.0, maxval=1.0, dtype=tf.float32),name='c_emb',trainable=True), 't_emb' : tf.Variable(0.2 * tf.random_uniform([tag_voc_size,FLAGS.embedding_dim], minval=-1.0, maxval=1.0, dtype=tf.float32),name='t_emb',trainable=False), 'hidden_w': tf.Variable(tf.random_normal([FLAGS.embedding_dim, 2*FLAGS.num_hidden])), 'hidden_c': tf.Variable(tf.random_normal([FLAGS.embedding_dim, 2*FLAGS.num_hidden])), 'hidden_t': tf.Variable(tf.random_normal([FLAGS.embedding_dim, 2*FLAGS.num_hidden])), 'out_w': tf.Variable(tf.random_normal([2*FLAGS.num_hidden, FLAGS.num_classes]))} _biases = { 'hidden_b': tf.Variable(tf.random_normal([2*FLAGS.num_hidden])), 'out_b': tf.Variable(tf.random_normal([FLAGS.num_classes]))} #~ input PlaceHolders seq_len = tf.placeholder(tf.int64,name="input_lr") _W = tf.placeholder(tf.int32,name="input_w") _C = tf.placeholder(tf.int32,name="input_c") _T = tf.placeholder(tf.int32,name="input_t") mask = tf.placeholder("float",name="input_mask") # Tensorflow LSTM cell requires 2x n_hidden length (state & cell) istate_fw = tf.placeholder("float", shape=[None, 2*FLAGS.num_hidden]) istate_bw = tf.placeholder("float", shape=[None, 2*FLAGS.num_hidden]) _Y = tf.placeholder("float", [None, FLAGS.num_classes]) #~ transfortm into Embeddings emb_x = tf.nn.embedding_lookup(_weights['w_emb'],_W) emb_c = tf.nn.embedding_lookup(_weights['c_emb'],_C) emb_t = tf.nn.embedding_lookup(_weights['t_emb'],_T) _X = tf.matmul(emb_x, _weights['hidden_w']) + tf.matmul(emb_c, _weights['hidden_c']) + tf.matmul(emb_t, _weights['hidden_t']) + _biases['hidden_b'] inputs = tf.split(_X, FLAGS.max_sent_length, axis=0, num=None, name='split') lstmcell = tf.contrib.rnn.BasicLSTMCell(FLAGS.num_hidden, forget_bias=1.0, state_is_tuple=False) bilstm = tf.contrib.rnn.static_bidirectional_rnn(lstmcell, lstmcell, inputs, sequence_length=seq_len, initial_state_fw=istate_fw, initial_state_bw=istate_bw) pred_mod = [tf.matmul(item, _weights['out_w']) + _biases['out_b'] for item in bilstm] 

Any help was appreciated.

+6
source share
1 answer

For those facing this problem in the future, the snippet above should not be used .

From tf.contrib.rnn.static_bidirectional_rnn v1.1 documentation:

Return:

Tuple (outputs, output_state_fw, output_state_bw) where: the outputs are a list of T output outputs (one for each input), which are depth-concatenated forward and backward outputs. output_state_fw - The final state of the front rnn. output_state_bw is the final state of the reverse rnn.

The above list expects LSTM outputs, and the correct way to get them:

 outputs, _, _ = tf.contrib.rnn.static_bidirectional_rnn(lstmcell, lstmcell, ...) pred_mod = [tf.matmul(item, _weights['out_w']) + _biases['out_b'] for item in outputs] 

This will work because each item in outputs has the form [batch_size, 2 * num_hidden] and can be multiplied by scales by tf.matmul() .


Addition: from tensorflow v1.2 +, the recommended function to use is in another package: tf.nn.static_bidirectional_rnn . The returned tensors are the same, so the code does not change much:

 outputs, _, _ = tf.nn.static_bidirectional_rnn(lstmcell, lstmcell, ...) pred_mod = [tf.matmul(item, _weights['out_w']) + _biases['out_b'] for item in outputs] 
+2
source

Source: https://habr.com/ru/post/1015516/


All Articles