Keras - Merge Dense Layer with 2D Convolution Level

Question

I want to create a custom layer that should fuse the output of a dense layer with the Convolution2D level.

The idea came from this article and here is the network:

the merge layer attempts to fuse the Convolution 2D tensor ( 256x28x28) with the dense tensor ( 256). here is the equation for it:

y_global => Dense layer output with shape 256 y_mid => Convolution2D layer output with shape 256x28x28

Here's a description of an article on the Fusion process:

As a result, I created a new custom layer, as shown below:

class FusionLayer(Layer):

    def __init__(self, output_dim, **kwargs):
        self.output_dim = output_dim
        super(FusionLayer, self).__init__(**kwargs)

    def build(self, input_shape):
        input_dim = input_shape[1][1]
        initial_weight_value = np.random.random((input_dim, self.output_dim))
        self.W = K.variable(initial_weight_value)
        self.b = K.zeros((input_dim,))
        self.trainable_weights = [self.W, self.b]

    def call(self, inputs, mask=None):
        y_global = inputs[0]
        y_mid = inputs[1]
        # the code below should be modified
        output = K.dot(K.concatenate([y_global, y_mid]), self.W)
        output += self.b
        return self.activation(output)

    def get_output_shape_for(self, input_shape):
        assert input_shape and len(input_shape) == 2
        return (input_shape[0], self.output_dim)

I think I used the methods correctly __init__and build, but I don’t know, how to concatenate y_global(256 dimesnions) with y-mid(dimensions 256x28x28) at a level callso that the output will be the same as the equation mentioned above.

How can I implement this equation in a method call?

Many thanks...

: ... , , ...

Answer 1

Keras Github, - , ... issue on github...

Answer 2

, - . :

• Flatten,
• Merge,
• Dense,

.