Using a recurrent model in reinforcement learning

Are there any good code examples of using a sequential model with the tf-agents package? I’ve got the DqnAgent to work with Atari Breakout but the model only takes a single frame from the environment. I’d like to use a recurrent or time-distributed model which takes a stack of frames as its input, but I’m finding it very hard to see how to do this with the DqnAgent class.

An approach I have seen is to get the environment to pass me a stack of frames as a single observation, but I would like the frame-stacking to be inside the agent rather than part of the environment.

Hey @ljbw_online.

I’m not aware of any blog/github with code ready to run, but this shall not be big issue using tf_agents.networks.Sequential  |  TensorFlow Agents.

The documentation reads:

Stateful Keras layers (e.g. LSTMCell, RNN, LSTM, TF-Agents DynamicUnroll) are all supported.

Did you start to write your code? Where did you get stuck?