I am working on the problem of labeling sentences. I did the attachment and addition myself, and my inputs look like this:
X_i = [[0,1,1,0,2,3...], [0,1,1,0,2,3...], ..., [0,0,0,0,0...], [0,0,0,0,0...], ....]
For each word in the sentence, I want to predict one of four classes, so my desired result should look like this:
Y_i = [[1,0,0,0], [0,0,1,0], [0,1,0,0], ...]
My simple network architecture:
model = Sequential()
model.add(LSTM(input_shape = (emb,),input_dim=emb, output_dim=hidden, return_sequences=True))
model.add(TimeDistributedDense(output_dim=4))
model.add(Activation('softmax'))
model.compile(loss='binary_crossentropy', optimizer='adam')
model.fit(X_train, Y_train, batch_size=32, nb_epoch=3, validation_data=(X_test, Y_test), verbose=1, show_accuracy=True)
During training, he shows approximately 95%, but when I try to predict new offers using the prepared results of the model, this is really bad. It seems that the model just learned some classes for the first words and shows it every time. I think the problem could be:
Written by me padding (zero vectors at the end of the sentence), can this impair learning?
, ( , , Keras?)
, , - , .
- TimeDistributedDense softmax, , , , 100% .
- , !