Why does CNN binary Keras always predict 1?

Question

I want to build a binary classifier using Keras CNN. I have about 6,000 lines of input that look like this:

>> print(X_train[0]) [[[-1.06405307 -1.06685851 -1.05989663 -1.06273152] [-1.06295958 -1.06655996 -1.05969803 -1.06382503] [-1.06415248 -1.06735609 -1.05999593 -1.06302975] [-1.06295958 -1.06755513 -1.05949944 -1.06362621] [-1.06355603 -1.06636092 -1.05959873 -1.06173742] [-1.0619655 -1.06655996 -1.06039312 -1.06412326] [-1.06415248 -1.06725658 -1.05940014 -1.06322857] [-1.06345662 -1.06377347 -1.05890365 -1.06034568] [-1.06027557 -1.06019084 -1.05592469 -1.05537518] [-1.05550398 -1.06038988 -1.05225064 -1.05676692]]] >>> print(y_train[0]) [1] 

And then I will build CNN in this way:

 model = Sequential() model.add(Convolution1D(input_shape = (10, 4), nb_filter=16, filter_length=4, border_mode='same')) model.add(BatchNormalization()) model.add(LeakyReLU()) model.add(Dropout(0.2)) model.add(Convolution1D(nb_filter=8, filter_length=4, border_mode='same')) model.add(BatchNormalization()) model.add(LeakyReLU()) model.add(Dropout(0.2)) model.add(Flatten()) model.add(Dense(64)) model.add(BatchNormalization()) model.add(LeakyReLU()) model.add(Dense(1)) model.add(Activation('softmax')) reduce_lr = ReduceLROnPlateau(monitor='val_acc', factor=0.9, patience=30, min_lr=0.000001, verbose=0) model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy']) history = model.fit(X_train, y_train, nb_epoch = 100, batch_size = 128, verbose=0, validation_data=(X_test, y_test), callbacks=[reduce_lr], shuffle=True) y_pred = model.predict(X_test) 

But it returns the following:

 >> print(confusion_matrix(y_test, y_pred)) [[ 0 362] [ 0 608]] 

Why are all predictions the same? Why is CNN so bad? Here are the graphs of losses and acc:

Answer 1

It always predicts one due to output on your network. You have a dense layer with one neuron with Softmax activation. Softmax is normalized by the sum of the exponent of each output. Since there is one way out, the only way out is 1.0.

For the binary classifier, you can either use sigmoid activation with the loss of "binary_crossentropy", or put the two output blocks at the last level, continue using softmax and change the loss to categorical percents.