Lack of explanation of "Working with RNNs" might lead to misunderstading

General Discussion
#1

Hi, i’m working with the RNNs with nested inputs, and found what I was looking for in here.

And thankfully, someone already made an example of that cell, NestedCell which shows how to build a custom RNN cell. However, I don’t think this cell precisely describes nested input version of GRU or LSTM.

If you see the call method part of this cell, it just simply updates the states by adding the output of the current inputs. And it actually confused me how the original version of GRU or LSTM works. (And as it is widely known, states should be updated with the reset gates/forget gates/input gates and so on)

So I am writing this post to get sure I am following along that this NestedCell is not clearly the one I can use for my nested input case. This example is a great one of blueprint of the nested case though.

Thanks for any comment in advance. Please let me know if there is any other ready-made layer for nested inputs case.

#3

Hi @HyeonPhil_Youn, The code you mentioned serves as example for creating custom code for nested input/output. You have to make changes in the call methods to include gates as per your requirement. As far as I know I don’t think there is a ready-made layer for nested inputs. you have to write your custom code as per your requirement. Thank You.