Unexpected keyword argument error in tensorflow-agents replay buffers

Following the tensorflow tutorial on deep reinforcement learning and DQN. Even after setting up the exact same libraries and running the same code, I am getting some error.

from tf_agents.replay_buffers import reverb_utils
rb_observer = reverb_utils.ReverbAddTrajectoryObserver(
  sequence_length=2) # This line is throwing error 

This is the stacktrace

TypeError                                 Traceback (most recent call last)
Input In [7], in <cell line: 23>()
     15 reverb_server = reverb.Server([table])
     17 replay_buffer = reverb_replay_buffer.ReverbReplayBuffer(
     18     agent.collect_data_spec,
     19     table_name=table_name,
     20     sequence_length=2,
     21     local_server=reverb_server)
---> 23 rb_observer = reverb_utils.ReverbAddTrajectoryObserver(
     24   replay_buffer.py_client,
     25   table_name,
     26   sequence_length=2)

File ~/.local/lib/python3.8/site-packages/tf_agents/replay_buffers/reverb_utils.py:336, in ReverbAddTrajectoryObserver.__init__(self, py_client, table_name, sequence_length, stride_length, priority, pad_end_of_episodes, tile_end_of_episodes)
    332 self._py_client = py_client
    333 # TODO(b/153700282): Use a single writer with max_sequence_length=max(...)
    334 # once Reverb Dataset with emit_timesteps=True returns properly shaped
    335 # sequences.
--> 336 self._writer = py_client.trajectory_writer(
    337     num_keep_alive_refs=sequence_length + 1, get_signature_timeout_ms=None)
    338 self._cached_steps = 0
    339 self._last_trajectory = None

TypeError: trajectory_writer() got an unexpected keyword argument 'get_signature_timeout_ms'

The stacktrace suggests the error is originating from the library itself. I am pretty new to DRL and tf-agents, so most of these classes and functions are total black-box to me. I am just trying to run and inspect them, but do not have nearly enough knowledge to go debug inside the library or even to know what my code (which is, actually, Google’s tutorial code) is doing wrong.

Even the linked google colab notebooks throw the same error. Seems like some library version/API issue, any idea? Without solving this, there is no way for me to proceed with the tutorial and run the rest of the code. :worried:

Maybe @Wei_Wei may be able to help here

I had the same error, for me it was a compatibility problem between dm-reverb and tf-agents.
For me deinstalling tf-agents and dm-reverb and then reinstalling with pip install tf-agents[reverb] fixed the problem. My version are tf-agents=0.12.0, tensorflow=2.8.2 and dm-reverb=0.7.3