Loss, val_loss, acc and val_acc are not updated at all times

Question

I created an LSTM network for sequence classification (binary), where each sample has 25 timestamps and 4 functions. Below is my keras network topology:

Above, the activation level after the Dense layer uses the softmax function. I used binary_crossentropy for the loss function and Adam as an optimizer for compiling the keras model. Trained the model using batch_size = 256, shuffle = True and validation_split = 0.05. The following is a training journal:

Train on 618196 samples, validate on 32537 samples 2017-09-15 01:23:34.407434: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2017-09-15 01:23:34.407719: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties: name: GeForce GTX 1050 major: 6 minor: 1 memoryClockRate (GHz) 1.493 pciBusID 0000:01:00.0 Total memory: 3.95GiB Free memory: 3.47GiB 2017-09-15 01:23:34.407735: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0 2017-09-15 01:23:34.407757: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0: Y 2017-09-15 01:23:34.407764: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1050, pci bus id: 0000:01:00.0) 618196/618196 [==============================] - 139s - loss: 4.3489 - acc: 0.7302 - val_loss: 4.4316 - val_acc: 0.7251 Epoch 2/50 618196/618196 [==============================] - 132s - loss: 4.3489 - acc: 0.7302 - val_loss: 4.4316 - val_acc: 0.7251 Epoch 3/50 618196/618196 [==============================] - 134s - loss: 4.3489 - acc: 0.7302 - val_loss: 4.4316 - val_acc: 0.7251 Epoch 4/50 618196/618196 [==============================] - 133s - loss: 4.3489 - acc: 0.7302 - val_loss: 4.4316 - val_acc: 0.7251 Epoch 5/50 618196/618196 [==============================] - 132s - loss: 4.3489 - acc: 0.7302 - val_loss: 4.4316 - val_acc: 0.7251 Epoch 6/50 618196/618196 [==============================] - 132s - loss: 4.3489 - acc: 0.7302 - val_loss: 4.4316 - val_acc: 0.7251 Epoch 7/50 618196/618196 [==============================] - 132s - loss: 4.3489 - acc: 0.7302 - val_loss: 4.4316 - val_acc: 0.7251 Epoch 8/50 618196/618196 [==============================] - 132s - loss: 4.3489 - acc: 0.7302 - val_loss: 4.4316 - val_acc: 0.7251 ... and so on through 50 epochs with same numbers 

So far I have also tried to use the rmsprop, nadam and batch_size (s) 128, 512, 1024 optimizers, but the losses, val_loss, acc, val_acc always remained the same throughout all eras, providing accuracy in the range from 0.72 to 0.74 in every attempt of mine.

Answer 1

Activating softmax ensures that the sum of the outputs is 1. This is useful to ensure that only one class will be displayed among many classes .

Since you only have 1 way out (only one class), this is definitely a bad idea. You will likely get a result of 1 for all samples.

Use sigmoid instead. This goes well with binary_crossentropy .