Retrain TF-Agents model with changed parameters (Checkpointer or PolicySaver)

Hello everyone!

I am trying to load a deep q model and retrain it with different parameters (e.g. changed epsilon value). I am using a config file for setting various parameters of the environment and the agent.
What is the best way to go about this? Should i use Checkpointer, PolicySaver or something else entirely?

I would be very grateful for your help!
Thank you!


I’d have to experiment with this too. Since Checkpointer allows you to save and load not just the policy network state, but also the training state, I’d give that a go. In addition, it seems like it also caches into a replay buffer, which would be useful for sampling in DQN’s case. (tf_agents.utils.common.Checkpointer  |  TensorFlow Agents)

Hope this helps a little. cc @yablak @markdaoust

1 Like

@Sarah_Riedmann, spoke with tensorflow-agents team member and they advised PolicySaver - it’s TF-Agents specific for saving a policy (with the step, and other info etc). PolicySaver uses Checkpointer underneath. Hope this helps!

1 Like

Thank you @8bitmp3! I will experiment with PolicySaver.

1 Like

I have tried using PolicySaver and it works so far. However, I would like to set the loaded policy as the DQN agent’s policy. Is it possible to continue using the agent with the saved policy or do i have to use loaded_policy.action() manually?

Any help would be appreciated! Thank you.

1 Like