Stateful LSTM: When is reset indicated?

Question

Given X with sizes (m samples, n sequences and k characters) and y labels with sizes (m samples, 0/1):

Suppose I want to train stateful LSTM (transition by definition keras, where "stateful = True" means that the states of the cells are not reset between the sequences in the sample - please correct me if I am wrong!), States that should be reset based on era or per sample?

Example:

for e in epoch:
    for m in X.shape[0]:          #for each sample
        for n in X.shape[1]:      #for each sequence
            #train_on_batch for model...
            #model.reset_states()  (1) I believe this is 'stateful = False'?
        #model.reset_states()      (2) wouldn't this make more sense?
    #model.reset_states()          (3) This is what I usually see...

In conclusion, I am not sure if reset indicates after each sequence or each era (after all m samples it is trained in X).

Advice is much appreciated.

Answer 1

stateful=True, reset . reset , stateful=False.

, :

for e in epoch:
    for m in X.shape[0]:          #for each sample
        for n in X.shape[1]:      #for each sequence

, X

 (m samples, n sequences, k features)

(batch size, number of timesteps, number of features)

, :

for n in X.shape[1]

,

for m in X.shape[0]

keras, ( , reset ). , reset , .

( ):

batch_size = 1
model = Sequential()
model.add(LSTM(16, batch_input_shape=(batch_size, X.shape[1], X.shape[2]), stateful=True))
model.add(Dense(y.shape[1], activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
for i in range(300):
    model.fit(X, y, epochs=1, batch_size=batch_size, verbose=2, shuffle=False)
    model.reset_states()