Cannot change activation in existing Keras model

Question

I have a regular VGG16 model with relu activations, i.e.

 def VGG_16(weights_path=None): model = Sequential() model.add(ZeroPadding2D((1, 1),input_shape=(3, 224, 224))) model.add(Convolution2D(64, 3, 3, activation='relu')) model.add(ZeroPadding2D((1, 1))) model.add(Convolution2D(64, 3, 3, activation='relu')) model.add(MaxPooling2D((2, 2), strides=(2, 2))) [...] model.add(Flatten()) model.add(Dense(4096, activation='relu')) model.add(Dropout(0.5)) model.add(Dense(4096, activation='relu')) model.add(Dropout(0.5)) model.add(Dense(1000, activation='softmax')) if weights_path: model.load_weights(weights_path) return model 

and I create it with existing weights and now I want to change all relu activations to softmax (not useful, I know)

 model = VGG_16('vgg16_weights.h5') sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True) softmax_act = keras.activations.softmax for (n, layer) in enumerate(model.layers): if 'activation' in layer.get_config() and layer.get_config()['activation'] == 'relu': print('replacing #{}: {}, {}'.format(n, layer, layer.activation)) layer.activation = softmax_act print('-> {}'.format(layer.activation)) model.compile(optimizer=sgd, loss='categorical_crossentropy') 

Note: model.compile is called after the changes, so the model must still be modifiable. I suppose.

However, despite the fact that debugging prints correctly say

 replacing #1: <keras.layers.convolutional.Convolution2D object at 0x7f7d7c497f50>, <function relu at 0x7f7dbe699a28> -> <function softmax at 0x7f7d7c4972d0> [...] 

actual results are identical with relu activation.
Why doesn't Keras use a modified activation feature?

Answer 1

you can use apply_modifications

 idx_of_layer_to_change = -1 model.layers[idx_of_layer_to_change].activation = activations.softmax model = utils.apply_modifications(model) 

Answer 2

Since the activation setting only in the keras layer does not actually change the schedule, we need to save the changed model and load it back:

 from keras import activations from keras.models import load_model model.layers[-1].activation = activations.example model.save(some_path) model = load_model(some_path)