Question about stateful in RNN layers

General Discussion
, ,
#1

In tf.keras.layers.RNN, the documentation states:

Note on using statefulness in RNNs: You can set RNN layers to be ‘stateful’, which means that the states computed for the samples in one batch will be reused as initial states for the samples in the next batch. This assumes a one-to-one mapping between samples in different successive batches.

  1. Does this mean that if stateful = True, then states are not reset between epochs as well?

  2. If stateful = False, is the layer’s state being reset each time the layer is called?

1 Like
#3
  1. I think you’d have to test and find out, I can imagine arguments for either way.
  2. Yes, the state is zero-initialized, unless you capture the state, and pass it back in on the next call. See these two docs for examples of manual state management: