How to define multiple labels in parallel (in a neural network) using the softmax-output layer in a tensor stream?

Question

In connection with the design work of my master's research, I implement a neural network using the Google tensor flow library. In this case, I would like to identify (at the output level of my primary neural network) several labels in parallel. And as a function to activate the output layer, I want to use the softmax function. So I want to have a specific output, this is a vector that looks like this:

vec = [0.1, 0.8, 0.1,   0.3, 0.2, 0.5]

Here, the first three numbers are the probabilities of the three classes of the first classification, and the remaining three numbers are the probabilities of the three classes of the second classification. Therefore, in this case, I would say that the labels are:

[ class2 , class3 ]

In my first attempt, I tried to implement this by first changing the (1x6) vector to the (2x3) matrix using tf.reshape (), then applying the softmax function on the tf.nn.softmax () matrix and finally converting the matrix back to vector. Unfortunately, due to the rebuild, Gradient-Descent-Optimizer gets problems with calculating the gradient, so I tried something else.

What I'm doing now, I take the (1x6) vector and multiply it by my matrix, which has a (3x3) identity matrix at the top and (3x3) zero matrix at the bottom. Whit this I retrieve the first three entries of the vector. Then I can apply the softmax function and return it to the old form (1x6) using another matrix multiplication. This must be repeated for the other three vector elements.

outputSoftmax  = tf.nn.softmax( vec * [[1,0,0],[0,1,0],[0,0,1],[0,0,0],[0,0,0],[0,0,0]] ) *  tf.transpose( [[1,0,0],[0,1,0],[0,0,1],[0,0,0],[0,0,0],[0,0,0]] )
               + tf.nn.softmax( vec * [[0,0,0],[0,0,0],[0,0,0],[1,0,0],[0,1,0],[0,0,1]] ) *  tf.transpose( [[0,0,0],[0,0,0],[0,0,0],[1,0,0],[0,1,0],[0,0,1]] )

, . , 91, 91 .

- , , softmax , " " 91 ?

Answer 1

tf.split 91 ( ), softmax .

classes_split = tf.split(0, 91, all_in_one)

for c in classes_split:
    softmax_class = tf.nn.softmax(c)
    # use softmax_class to compute some loss, add it to overall loss

:

classes_split = tf.split(0, 91, all_in_one)

# softmax each split individually
classes_split_softmaxed = [tf.nn.softmax(c) for c in classes_split]
# Concatenate again
all_in_one_softmaxed = tf.concat(0, classes_split_softmaxed)