Tensorflow: show or save forgotten gate values ​​in LSTM

Question

I am using the LSTM model, which is used by default in the tensor stream. I would like to check or find out how to save or show shutter values ​​at each step, has anyone done this before, or at least something like this?

So far I have tried with tf.print, but many values ​​have appeared (even more than expected). I would try to build something with a tensor, but I think these gates are just variables and not extra layers that I can print (also call them inside the TF script)

Any help would be well received.

Answer 1

If you use a tf.rnn_cell.BasicLSTMCellvariable, you are looking for will have the following suffix to the name: <parent_variable_scope>/BasicLSTMCell/Linear/Matrix. This is a concatenated matrix for all four gates. Its first size coincides with the sum of the second dimensions of the input matrix and state matrix (or cell output, to be precise). The second dimension is 4 times the size of the cell.

Another additional variable <parent_variable_scope>/BasicLSTMCell/Linear/Bias, which is a vector of the same size as the second dimension of the above tensor (for obvious reasons).

You can get the parameters for the four gates using tf.split()size 1. Split matrices will be in order [input], [new input], [forget], [output]. I mean the code here is form rnn_cell.py.

, Cell, . , , , .

Edit:
​​ Matrix Bias