When applying an LSTM layer to something of the shape (None, None, x), I get something of shape (None, y) where y is a parameter of the LSTM layer. It can “unrag” axis 1, which the documentation refers to as the “timesteps”.
The input shape of my layer is (None, None, None, x). I cannot reshape it to (None, None, x), because the tensor is ragged in axis 1, and I will lose shape information which I will need later. I want to end up with a shape (None, None, y), so I want to “unrag” axis 2, but keep axis 1 ragged. However, an LSTM can only accept a 3-dimensional input. How should I go about achieving this?