How is the forward pass for SimpleRNN computed? There are 3 kernels, an input? kernel of shape (input_dim, units), recurrent kernel of shape (units, units), and bias of shape (units, 1). I was not able to manually reproduce the final_state from calling the layer:
simple_rnn = tf.keras.layers.SimpleRNN(4, return_sequences=True, return_state=True)
whole_sequence_output, final_state = simple_rnn(inputs)