How can I use a Tensorflow LSTM model to predict a single number from sequence of numbers?

General Discussion
, ,
#1

I am trying to build a machine learning model which predicts a single number from a sequence of numbers.
Please fell free to have a look at this minimal example in Google Colab to understand what I am talking about.

You can imagine my dataset to look something like this:

Index x data y data
0 np.ndarray(shape (1209278,) ) numpy.float32
1 np.ndarray(shape (1211140,) ) numpy.float32
2 np.ndarray(shape (1418411,) ) numpy.float32
3 np.ndarray(shape (1077132,) ) numpy.float32
… … …

In a nutshell, my goal is the following: Predicting a single number from a sequence of numbers.

For example:

  • np.array([3.461, 3.478, 3.478, 3.485, 3.489, 3.489, 3.492]) => 3.281
  • np.array([3.469, 3.481, 3.481, 3.495, 3.495]) => 3.271
  • …

Additionally it is important to understand, that the lengths of my input sequences may vary and are not of the same shape.

I was able to train my model (as you can see in the minimal example) but apparently the desired output is nothing at all like the one I expected it to be. Right now I’m stuck with this problem. I was expecting it to be working just fine, because I saw the loss decreasing.

Apparently this is not the case. I would be very thankful if you could find the time to have a look at the minimal example I provided.

Thanks in advance!

#3

I believe you should simply do “prediction = model.predict(x_train)” for getting the prediction.

The input passed to model.predict() should be similar with x used for model.fit().
As you’re already aware, the input shape for your model should be [samples, time_steps, 1], but you’re calling predict by passing input with shape [time_steps,1,1] which is wrong.

1 Like
#5

Hi, thank you very much for your help!
I was thinking about using prediction = model.predict(x_train) directly where x_train is a numpy ndarray, so taking straightly from above dataset for example.
Sadly, this leads to a ValueError:

ValueError: Exception encountered when calling layer "sequential" (type Sequential).
Input 0 of layer "lstm" is incompatible with the layer: expected ndim=3, found ndim=1. Full shape received: (None,)
    
    Call arguments received:
      • inputs=tf.Tensor(shape=(None,), dtype=float32)
      • training=False
      • mask=None

Also this warning is displayed:

WARNING:tensorflow:Model was constructed with shape (None, None, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, None, 1), dtype=tf.float32, name='lstm_input'), name='lstm_input', description="created by layer 'lstm_input'"), but it was called on an input with incompatible shape (None,).

I think that is the case, because my x_predict is of shape (None,) because it is a sequence of numbers, but my LSTM network expects shapes of (None, None, 1) (see the Warning above).

You said:

The input passed to model.predict() should be similar with x used for model.fit().
As you’re already aware, the input shape for your model should be [samples, time_steps, 1], but you’re calling predict by passing input with shape [time_steps,1,1] which is wrong.

Which I totally get. But now my question ist:
How do I reshape my data to be in the neccessary shape for my LSTM network?

Thanks in advance! :slight_smile:

#7

No, your x_train is not a numpy ndarray. It is a RaggedTensor.

Using prediction = model.predict(x_train) directly works fine as shown in this notebook below. (see “Prediction” section)

https://colab.research.google.com/drive/1SISAIIG2IZ22MP_HuFs1WzbJyn8aasqS?usp=sharing

1 Like
#9

Thank you very much for your help!
Greatly appreciated :smiley: