TensorFlow: a simple recurrent neural network

Question

I built several neural networks with TensorFlow, for example, basic MLP and convolutional neural networks. Now I want to move on to repeating neural networks. However, I have no experience in natural language processing. Therefore, TensorFlow NLP tutorials for RNN are not easy to read for me (and also not very interesting).

Basically, I want to start with something simple, not LSTM.

How to build a simple recurrent neural network, such as an Elman network, in TensorFlow?

I could only find GRU or LSTM RNN examples for TensorFlow, mainly for NLP. Does anyone know of some simple neural network recurrence tutorials or examples for TensorFlow?

This figure shows the core Elman network, which is often simply called the SRN (Simple Recurrent Network):

Answer 1

One option is to use the built-in RNNCell located in tensorflow / python / ops / rnn_cell.py .

If you do not want to do this, you can create your own RNN. RNN will train using back propagation in time. Try deploying a network with a fixed number of steps, for example. consider input sequences ten in length. Then you can write a loop in python to do all matrix multiplications for each step of the network. Each time you can output the result from the previous step and combine it with the input of this step. It will not be too many lines of code to make it work.

Answer 2

Another option is to use the Tensorflow scan function to implement repetition. My result (which is a simple recurrent neural network / Elman network) can be found here: https://www.data-blogger.com/2017/05/17/elman-rnn-implementation-in-tensorflow/