I have problem making an LSTM Regressor with Keras

It’s not a bug neither a documentation error, I’m just new and I seek for help. Hopefully this is the right place and I hope I’m not making noise for no reason.

I am using this architecture (a Masking Layer for varying trajectory lengths that are padded with 0s to maximum length trajectory followed by a LSTM with a dense layer afterwards that outputs 2 values) to build a regressor that predicts 2 values based on a trajectory.

    samples, timesteps, features = x_train.shape[0], x_train.shape[1], x_train.shape[2]
    model = Sequential()
    model.add(tf.keras.layers.Masking(mask_value=0., input_shape=(timesteps, features), name="mask"))
    model.add(LSTM(30, return_sequences=True, name="lstm1"))
    model.add(LSTM(30, return_sequences=False, name="lstm2"))
    model.add(Dense(20, activation='relu', name="dense1"))
    model.add(Dense(20, activation='relu', name="dense2"))
    model.add(Dense(2, activation='linear', name="output"))

My input data are of shape:

x_train (269, 527, 11) (269 trajectories of 527 timesteps of 11 features)
y_train (269, 2) (these 269 trajectories have 2 target values)
x_test (30, 527, 11) (--- same ---)
y_test (30, 2) (--- same ---)

I’ve preprocessed my data so as all my sequences have fixed length and smaller ones are filled with 0’s at missing timesteps. Thus, I’m using a Masking layer to skip these timesteps as they provide no information.

As expected, the output is of shape:

(30, 2)

But looking into it seems like its regressing the same values.

[[37.48257     0.7025466 ]
 [37.48258     0.70254654]
 [37.48257     0.70254654]
 [37.48257     0.7025466 ]
 [37.48258     0.70254654]
 [37.48258     0.70254654]
 [37.48258     0.70254654]
 [37.48258     0.7025465 ]
 [42.243515    0.6581909 ]
 [37.48258     0.70254654]
 [37.48257     0.70254654]
 [37.48258     0.70254654]
 [37.48261     0.7025462 ]
 [37.48257     0.7025466 ]
 [37.482582    0.70254654]
 [37.482567    0.70254654]
 [37.48257     0.7025466 ]
 [37.48258     0.70254654]
 [37.48258     0.70254654]
 [37.48257     0.7025466 ]
 [37.48258     0.70254654]
 [37.48258     0.70254654]
 [37.48258     0.70254654]
 [37.482567    0.7025465 ]
 [37.48261     0.7025462 ]
 [37.482574    0.7025466 ]
 [37.48261     0.7025462 ]
 [37.48261     0.70254624]
 [37.48258     0.70254654]
 [37.48261     0.7025462 ]]

below is the target values (y_test) obviously of shape (30,2)

[[70.   0.6]
 [40.   0.6]
 [ 6.   0.6]
 [94.   0.7]
 [50.   0.6]
 [60.   0.6]
 [16.   0.6]
 [76.   0.9]
 [92.   0.6]
 [32.   0.8]
 [22.   0.7]
 [70.   0.7]
 [36.   1. ]
 [64.   0.7]
 [ 0.   0.9]
 [82.   0.9]
 [38.   0.6]
 [54.   0.8]
 [28.   0.8]
 [62.   0.7]
 [12.   0.6]
 [72.   0.8]
 [66.   0.8]
 [ 2.   1. ]
 [98.   1. ]
 [20.   0.8]
 [82.   1. ]
 [38.   1. ]
 [68.   0.6]
 [62.   1. ]]

It’s like approaching the whole dataset as 1 data point or something like that. Can anybody with some experience find any obvious mistake here?

Appreciate any kind of help!

Hi @white_sirilo

Welcome to the TensorFlow Forum!

Could you please let us know if this issue still persists? If so, Please share the standalone code along with the dataset type and data preprocessing part to replicate and understand the issue. Thank you.