Simple RNN on artificial data
Open the notebook 14_simple_rnn_tf1_no_solution.ipynb.
a) Forward pass with trained weights ($W$ and $b$): Complete the code for the single time step forward pass from the hidden state $h_0 \rightarrow h_1$.
b) Complete the forward pass for a sequence of time steps.
c) Use ($V$ and $bv$) to calculate the output probabilities for time step 1. Hint: You might want to use a softmax.
d) Try to understand the training code (Keras and/or TensorFlow).
e) Do the training with a hidden state of size of 4 and change it then to 2, how is the performance affected? What is the influence on $W$,$b$,$V$ and $bv$.Use the TensorFlow code, which currently runs much faster.