How to prevent lazy coiled neural network?

Question

How to prevent lazy coiled neural network? I end up with "lazy CNN" after training with KERAS. Regardless of the input, the output is constant. What do you think is the problem?

I am trying to repeat the NVIDIA experiment to teach the ultimate lesson for self-driving cars in a document . Absolutely, I do not have a real machine, but Uatitys is a simulator . The simulator generates figures about the foreground of the car.

CNN gets a number, and it gives a rotation angle to keep the car on track. The rule of the game is to keep the car’s simulated mileage on the track safe. It is not very difficult.

The strange thing sometimes ends with a lazy CNN after training with KERAS, which gives constant control angles. The simulated car will leave the stunt, but the CNN output will not change. Especially the layer gets deeper, for example. CNN per document .

If I use CNN like this, I can get a useful model after training.

model = Sequential()
model.add(Lambda(lambda x: x/255.0 - 0.5, input_shape = (160,320,3)))
model.add(Cropping2D(cropping=((70,25),(0,0))))
model.add(Conv2D(24, 5, strides=(2, 2)))
model.add(Activation('relu'))
model.add(Conv2D(36, 5, strides=(2, 2)))
model.add(Activation('relu'))
model.add(Conv2D(48, 5, strides=(2, 2)))
model.add(Activation('relu'))
model.add(Flatten())
model.add(Dense(50))
model.add(Activation('sigmoid'))
model.add(Dense(10))
model.add(Activation('sigmoid'))
model.add(Dense(1))

But, if I use deeper CNN, I’m more likely to get lazy CNN. In particular, if I use CNN, which NVIDIA loves, I almost get lazy CNN after each training.

model = Sequential()
model.add(Lambda(lambda x: x/255.0 - 0.5, input_shape = (160,320,3)))
model.add(Cropping2D(cropping=((70,25),(0,0))))
model.add(Conv2D(24, 5, strides=(2, 2)))
model.add(Activation('relu'))
model.add(Conv2D(36, 5, strides=(2, 2)))
model.add(Activation('relu'))
model.add(Conv2D(48, 5, strides=(2, 2)))
model.add(Activation('relu'))
model.add(Conv2D(64, 3, strides=(1, 1)))
model.add(Activation('relu'))
model.add(Conv2D(64, 3, strides=(1, 1)))
model.add(Activation('relu'))
model.add(Flatten())
model.add(Dense(1164))
model.add(Activation('sigmoid'))
model.add(Dense(100))
model.add(Activation('sigmoid'))
model.add(Dense(50))
model.add(Activation('sigmoid'))
model.add(Dense(10))
model.add(Activation('sigmoid'))
model.add(Dense(1))

I use 'relu for convolution layers, and the activation function for a fully connected layer is' sigmoid. I am trying to change the activation function, but there is no effect.

. ​​ , CNN. , . , , ; . . 60% . , CNN . , . , , CNN -, . , . , CNN, , .

? , ? , CNN.

GitHub. .

Answer 1

, , GitHub . 90% .

, sigmoid . , , , .

, NVidia , , sigmoid ( ), relu. . , , . , , - :

model = Sequential()
model.add(Lambda(lambda x: x/255.0 - 0.5, input_shape = (160,320,3)))
model.add(Cropping2D(cropping=((70,25),(0,0))))
model.add(Conv2D(24, (5, 5), strides=(2, 2), activation="relu"))
model.add(Conv2D(36, (5, 5), strides=(2, 2), activation="relu"))
model.add(Conv2D(48, (5, 5), strides=(2, 2), activation="relu"))
model.add(Conv2D(64, (3, 3), strides=(1, 1), activation="relu"))
model.add(Conv2D(64, (3, 3), strides=(1, 1), activation="relu"))
model.add(Flatten())
model.add(Dense(1164, activation="relu"))
model.add(Dense(100, activation="relu"))
model.add(Dense(50, activation="relu"))
model.add(Dense(10, activation="relu"))
model.add(Dense(1))

NVidia .