How to print LSTM gate values ​​in TensorFlow?

Question

I use TSTORFlow LSTM for the language model (I have a sequence of words and you want to predict the next word), and when I run the language model, I want to print the values ​​of forgotten, input, transform and output at each step. How to do it?

Checking the code at https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/rnn/python/ops/rnn_cell.py , I see that the class LayerNormBasicLSTMCellhas a method callthat contains i, j, f, owhich I want to print.

  def call(self, inputs, state):
    """LSTM cell with layer normalization and recurrent dropout."""
    c, h = state
    args = array_ops.concat([inputs, h], 1)
    concat = self._linear(args)

    i, j, f, o = array_ops.split(value=concat, num_or_size_splits=4, axis=1)
    if self._layer_norm:
      i = self._norm(i, "input")
      j = self._norm(j, "transform")
      f = self._norm(f, "forget")
      o = self._norm(o, "output")

    g = self._activation(j)
    if (not isinstance(self._keep_prob, float)) or self._keep_prob < 1:
      g = nn_ops.dropout(g, self._keep_prob, seed=self._seed)

    new_c = (c * math_ops.sigmoid(f + self._forget_bias)
             + math_ops.sigmoid(i) * g)
    if self._layer_norm:
      new_c = self._norm(new_c, "state")
    new_h = self._activation(new_c) * math_ops.sigmoid(o)

    new_state = core_rnn_cell.LSTMStateTuple(new_c, new_h)
    return new_h, new_state

However, is there an easy way to print these variables? Or do I need to basically recreate the corresponding lines of code in this method, in my script, where I run LTSM?

Answer 1

git. , c h ( y ). , .

: https://github.com/tensorflow/tensorflow/issues/5731

Answer 2

:

, , return new_h, new_state, i, j, f, o. , TensorFlow , , .
session.run(to_return, feed_dict) to_return :

output, state, i, j, f, o = lstm_cell(input, state)   
to_return = {
    "new_h": output, 
    "new_state": state,
    "i": i,
    "j": j,
    "f": f, 
    "o": o,
}

results = session.run(to_return, feed_dict) # get what you want from the 
# graph(which are tensors), resulting in results of a dictionary with values
# being numpy arrays. 

print results["i"] # you'll get a numpy array representing the i gate