Keras: How to use fit_generator with multiple outputs of different types

Question

In a Keras model with a functional API, I need to call fit_generator to learn from advanced image data using ImageDataGenerator. The problem is that my model has two outputs: the mask I'm trying to predict, and the binary value. I obviously only want to increase the input and output of the mask, not the binary value. How can I achieve this?

Answer 1

The example below may be understandable! The 'dummy' model takes 1 input (image) and outputs 2 values. The model computes the MSE for each output.

x = Convolution2D(8, 5, 5, subsample=(1, 1))(image_input)
x = Activation('relu')(x)
x = Flatten()(x)
x = Dense(50, W_regularizer=l2(0.0001))(x)
x = Activation('relu')(x)

output1 = Dense(1, activation='linear', name='output1')(x)
output2 = Dense(1, activation='linear', name='output2')(x)

model = Model(input=image_input, output=[output1, output2])
model.compile(optimizer='adam', loss={'output1': 'mean_squared_error', 'output2': 'mean_squared_error'})

. x y, y = [y1, y2]

batch_generator(x, y, batch_size):
        ....transform images
        ....generate batch batch of size: batch_size 
        yield(X_batch, {'output1': y1, 'output2': y2} ))

, fit_generator()

    model.fit_generator(batch_generator(X_train, y_train, batch_size))

Answer 2

, , - :

generator = ImageDataGenerator(rotation_range=5.,
                                width_shift_range=0.1, 
                                height_shift_range=0.1, 
                                horizontal_flip=True,  
                                vertical_flip=True)

def generate_data_generator(generator, X, Y1, Y2):
    genX = generator.flow(X, seed=7)
    genY1 = generator.flow(Y1, seed=7)
    while True:
            Xi = genX.next()
            Yi1 = genY1.next()
            Yi2 = function(Y2)
            yield Xi, [Yi1, Yi2]

, , . (Y2). fit_generator():

model.fit_generator(generate_data_generator(generator, X, Y1, Y2),
                epochs=epochs)

Answer 3

The best way to achieve this is to create a new generator class that extends Keras, which analyzes the data, enlarging only the images and getting all the outputs.