I am trying to prepare a 2D convolutional LSTM to make categorical forecasts based on video data. However, my output level seems to run into a problem:
"ValueError: error while checking the target: it is expected that dense_1 has 5 dimensions, but received an array with the form (1, 1939, 9)"
My current model is based on the ConvLSTM2D example provided by Keras Team. I believe that the above error is the result of my misunderstanding of the example and its basic principles.
Data
I have an arbitrary number of videos, where each video contains an arbitrary number of frames. Each frame has a size of 135x240x1 (last color channels). This leads to an input form (None, None, 135, 240, 1), where the two "None" values represent the batch size and timestamps in that order. If I train on one video with 1052 frames, then my input form becomes (1, 1052, 135, 240, 1).
For each frame, the model should predict values between 0 and 1 through 9 classes. This means my output form (None, None, 9). If I train on one video with 1052 frames, then this form becomes (1, 1052, 9).
Model
Layer (type) Output Shape Param #
=================================================================
conv_lst_m2d_1 (ConvLSTM2D) (None, None, 135, 240, 40 59200
_________________________________________________________________
batch_normalization_1 (Batch (None, None, 135, 240, 40 160
_________________________________________________________________
conv_lst_m2d_2 (ConvLSTM2D) (None, None, 135, 240, 40 115360
_________________________________________________________________
batch_normalization_2 (Batch (None, None, 135, 240, 40 160
_________________________________________________________________
conv_lst_m2d_3 (ConvLSTM2D) (None, None, 135, 240, 40 115360
_________________________________________________________________
batch_normalization_3 (Batch (None, None, 135, 240, 40 160
_________________________________________________________________
dense_1 (Dense) (None, None, 135, 240, 9) 369
=================================================================
Total params: 290,769
Trainable params: 290,529
Non-trainable params: 240
Source
model = Sequential()
model.add(ConvLSTM2D(
filters=40,
kernel_size=(3, 3),
input_shape=(None, 135, 240, 1),
padding='same',
return_sequences=True))
model.add(BatchNormalization())
model.add(ConvLSTM2D(
filters=40,
kernel_size=(3, 3),
padding='same',
return_sequences=True))
model.add(BatchNormalization())
model.add(ConvLSTM2D(
filters=40,
kernel_size=(3, 3),
padding='same',
return_sequences=True))
model.add(BatchNormalization())
model.add(Dense(
units=classes,
activation='softmax'
))
model.compile(
loss='categorical_crossentropy',
optimizer='adadelta'
)
model.fit_generator(generator=training_sequence)
Traceback
Epoch 1/1
Traceback (most recent call last):
File ".\lstm.py", line 128, in <module>
main()
File ".\lstm.py", line 108, in main
model.fit_generator(generator=training_sequence)
File "C:\Users\matth\Anaconda3\envs\capstone-gpu\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "C:\Users\matth\Anaconda3\envs\capstone-gpu\lib\site-packages\keras\models.py", line 1253, in fit_generator
initial_epoch=initial_epoch)
File "C:\Users\matth\Anaconda3\envs\capstone-gpu\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "C:\Users\matth\Anaconda3\envs\capstone-gpu\lib\site-packages\keras\engine\training.py", line 2244, in fit_generator
class_weight=class_weight)
File "C:\Users\matth\Anaconda3\envs\capstone-gpu\lib\site-packages\keras\engine\training.py", line 1884, in train_on_batch
class_weight=class_weight)
File "C:\Users\matth\Anaconda3\envs\capstone-gpu\lib\site-packages\keras\engine\training.py", line 1487, in _standardize_user_data
exception_prefix='target')
File "C:\Users\matth\Anaconda3\envs\capstone-gpu\lib\site-packages\keras\engine\training.py", line 113, in _standardize_input_data
'with shape ' + str(data_shape))
ValueError: Error when checking target: expected dense_1 to have 5 dimensions, but got array with shape (1, 1939, 9)
, , 1, (1, 1389, 135, 240, 1). , , , Keras Sequence ( "training_sequence" ) .
, , BatchNormalization() Dense(). , , dense_1 ( ). , , - , , , .
3/27/2018
, , ConvLSTM2D, return_sequences False True. GlobalAveragePooling2D Dense. :
Layer (type) Output Shape Param #
=================================================================
conv_lst_m2d_1 (ConvLSTM2D) (None, None, 135, 240, 40 59200
_________________________________________________________________
batch_normalization_1 (Batch (None, None, 135, 240, 40 160
_________________________________________________________________
conv_lst_m2d_2 (ConvLSTM2D) (None, None, 135, 240, 40 115360
_________________________________________________________________
batch_normalization_2 (Batch (None, None, 135, 240, 40 160
_________________________________________________________________
conv_lst_m2d_3 (ConvLSTM2D) (None, 135, 240, 40) 115360
_________________________________________________________________
batch_normalization_3 (Batch (None, 135, 240, 40) 160
_________________________________________________________________
global_average_pooling2d_1 ( (None, 40) 0
_________________________________________________________________
dense_1 (Dense) (None, 9) 369
=================================================================
Total params: 290,769
Trainable params: 290,529
Non-trainable params: 240
:
Traceback (most recent call last):
File ".\lstm.py", line 131, in <module>
main()
File ".\lstm.py", line 111, in main
model.fit_generator(generator=training_sequence)
File "C:\Users\matth\Anaconda3\envs\capstone-gpu\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "C:\Users\matth\Anaconda3\envs\capstone-gpu\lib\site-packages\keras\models.py", line 1253, in fit_generator
initial_epoch=initial_epoch)
File "C:\Users\matth\Anaconda3\envs\capstone-gpu\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "C:\Users\matth\Anaconda3\envs\capstone-gpu\lib\site-packages\keras\engine\training.py", line 2244, in fit_generator
class_weight=class_weight)
File "C:\Users\matth\Anaconda3\envs\capstone-gpu\lib\site-packages\keras\engine\training.py", line 1884, in train_on_batch
class_weight=class_weight)
File "C:\Users\matth\Anaconda3\envs\capstone-gpu\lib\site-packages\keras\engine\training.py", line 1487, in _standardize_user_data
exception_prefix='target')
File "C:\Users\matth\Anaconda3\envs\capstone-gpu\lib\site-packages\keras\engine\training.py", line 113, in _standardize_input_data
'with shape ' + str(data_shape))
ValueError: Error when checking target: expected dense_1 to have 2 dimensions, but got array with shape (1, 1034, 9)
x y . x (1, 1034, 135, 240, 1), y (1, 1034, 9). . , y, x. , Dense . , .
3/28/2018
- . , , , , :
Layer (type) Output Shape Param #
=================================================================
conv_lst_m2d_1 (ConvLSTM2D) (None, None, 135, 240, 40 59200
_________________________________________________________________
batch_normalization_1 (Batch (None, None, 135, 240, 40 160
_________________________________________________________________
conv_lst_m2d_2 (ConvLSTM2D) (None, None, 135, 240, 40 115360
_________________________________________________________________
batch_normalization_2 (Batch (None, None, 135, 240, 40 160
_________________________________________________________________
conv_lst_m2d_3 (ConvLSTM2D) (None, None, 135, 240, 40 115360
_________________________________________________________________
batch_normalization_3 (Batch (None, None, 135, 240, 40 160
_________________________________________________________________
average_pooling3d_1 (Average (None, None, 1, 1, 40) 0
_________________________________________________________________
reshape_1 (Reshape) (None, None, 40) 0
_________________________________________________________________
dense_1 (Dense) (None, None, 9) 369
=================================================================
Total params: 290,769
Trainable params: 290,529
Non-trainable params: 240
, :
model = Sequential()
model.add(ConvLSTM2D(
filters=40,
kernel_size=(3, 3),
input_shape=(None, 135, 240, 1),
padding='same',
return_sequences=True))
model.add(BatchNormalization())
model.add(ConvLSTM2D(
filters=40,
kernel_size=(3, 3),
padding='same',
return_sequences=True))
model.add(BatchNormalization())
model.add(ConvLSTM2D(
filters=40,
kernel_size=(3, 3),
padding='same',
return_sequences=True))
model.add(BatchNormalization())
model.add(AveragePooling3D((1, 135, 240)))
model.add(Reshape((-1, 40)))
model.add(Dense(
units=9,
activation='sigmoid'))
model.compile(
loss='categorical_crossentropy',
optimizer='adadelta'
)