How to implement multiclass semantic segmentation?

Question

I can train U-net with tagged images that have binary classification.

But I find it difficult to determine how to configure the final levels in Keras / Theano to classify multi-class (4 classes).

I have 634 images and corresponding 634 masks, which are unit864 x 64 pixels.

My masks, instead of being black (0) and white (1), have color-labeled objects in three categories plus a background as follows:

• black (0), background
• red (1), object class 1
• green (2), object class 2
• yellow (3), object class 3

Before starting workouts, the array containing the masks is one hot coding as follows:

mask_train = to_categorical(mask_train, 4)

mask_train.shape (634, 1, 64, 64) (2596864, 4).

Unet, , , .

[...]
up3 = concatenate([UpSampling2D(size=(2, 2))(conv7), conv2], axis=1)
conv8 = Conv2D(128, (3, 3), activation='relu', padding='same')(up3)
conv8 = Conv2D(128, (3, 3), activation='relu', padding='same')(conv8)

up4 = concatenate([UpSampling2D(size=(2, 2))(conv8), conv1], axis=1)
conv9 = Conv2D(64, (3, 3), activation='relu', padding='same')(up4)
conv10 = Conv2D(64, (3, 3), activation='relu', padding='same')(conv9)

# here I used number classes = number of filters and softmax although
# not sure if a dense layer should be here instead
conv11 = Conv2D(4, (1, 1), activation='softmax')(conv10)

model = Model(inputs=[inputs], outputs=[conv11])

# here categorical cross entropy is being used but may not be correct
model.compile(optimizer='sgd', loss='categorical_crossentropy',
              metrics=['accuracy'])

return model

- , , ? , , .

Answer 1

(634,4,64,64), channels_first.
(634,64,64,4), channels_last.

. 0 1, 1 , , 0 , .

- 634 , , 64x64 , 1 .

, , :

mask_train = to_categorical(mask_train, 4)
mask_train = mask_train.reshape((634,64,64,4)) 
#I chose channels last here because to_categorical is outputing your classes last: (2596864,4)

#moving the channel:
mask_train = np.moveaxis(mask_train,-1,1)

, :

newMask = np.zeros((634,4,64,64))

for samp in range(len(mask_train)):
    im = mask_train[samp,0]
    for x in range(len(im)):
        row = im[x]
        for y in range(len(row)):
            y_val = row[y]
            newMask[samp,y_val,x,y] = 1