Following the tensorflow tutorial on deep reinforcement learning and DQN. Even after setting up the exact same libraries and running the same code, I am getting some error.
from tf_agents.replay_buffers import reverb_utils
....
rb_observer = reverb_utils.ReverbAddTrajectoryObserver(
replay_buffer.py_client,
table_name,
sequence_length=2) # This line is throwing error
This is the stacktrace
TypeError Traceback (most recent call last)
Input In [7], in <cell line: 23>()
15 reverb_server = reverb.Server([table])
17 replay_buffer = reverb_replay_buffer.ReverbReplayBuffer(
18 agent.collect_data_spec,
19 table_name=table_name,
20 sequence_length=2,
21 local_server=reverb_server)
---> 23 rb_observer = reverb_utils.ReverbAddTrajectoryObserver(
24 replay_buffer.py_client,
25 table_name,
26 sequence_length=2)
File ~/.local/lib/python3.8/site-packages/tf_agents/replay_buffers/reverb_utils.py:336, in ReverbAddTrajectoryObserver.__init__(self, py_client, table_name, sequence_length, stride_length, priority, pad_end_of_episodes, tile_end_of_episodes)
332 self._py_client = py_client
333 # TODO(b/153700282): Use a single writer with max_sequence_length=max(...)
334 # once Reverb Dataset with emit_timesteps=True returns properly shaped
335 # sequences.
--> 336 self._writer = py_client.trajectory_writer(
337 num_keep_alive_refs=sequence_length + 1, get_signature_timeout_ms=None)
338 self._cached_steps = 0
339 self._last_trajectory = None
TypeError: trajectory_writer() got an unexpected keyword argument 'get_signature_timeout_ms'
The stacktrace suggests the error is originating from the library itself. I am pretty new to DRL and tf-agents, so most of these classes and functions are total black-box to me. I am just trying to run and inspect them, but do not have nearly enough knowledge to go debug inside the library or even to know what my code (which is, actually, Google’s tutorial code) is doing wrong.
Even the linked google colab notebooks throw the same error. Seems like some library version/API issue, any idea? Without solving this, there is no way for me to proceed with the tutorial and run the rest of the code. 