Loss and val_loss are unreasonably HIGH - need ideas

General Discussion
#1

Hi, beginner here.

I followed the Basic regression: Predict fuel efficiency | TensorFlow Core and got an idea of how to build and test a model.

So I tried to design my own experiment.
the input data is of the simple form
x,y
0,5
1,7
2,9
3,11
(10,000 rows)
basically y = 2x + 5
the idea is to train the model to predict y (label); given feature x.
[I get it, it’s a function already, I’m doing this so I can better understand the model building and prediction process].

so after loading the data, building the model with only 2 layers, 1 layer = normalization layer, 2nd layer = dense layer (as suggested in the tutorial)

model.summary()
Model: “sequential”

Layer (type) Output Shape Param #

normalization_1 (Normaliza (None, 1) 3
tion)
dense (Dense) (None, 1) 2

Total params: 5 (24.00 Byte)
Trainable params: 2 (8.00 Byte)
Non-trainable params: 3 (16.00 Byte)

code for fitting the data:

–
history = a_model.fit(
train_features[‘x’],
train_labels,
epochs=100,
validation_split=0.2)

training progress:

Epoch 1/100 30/200 [===>…] - ETA: 0s - loss: 9917.3330
2024-02-26 12:50:35.272904: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:961] model_pruner failed: INVALID_ARGUMENT: Graph does not contain terminal node Adam/AssignAddVariableOp.
200/200 [==============================] - 1s 5ms/step - loss: 10058.8467 - val_loss: 9823.6104 Epoch 2/100 200/200 [==============================] - 1s 4ms/step - loss: 10058.6445 - val_loss: 9823.4121 Epoch 3/100 200/200 [==============================] - 1s 4ms/step - loss: 10058.4463 - val_loss: 9823.2129 Epoch 4/100 200/200 [==============================] - 1s 4ms/step - loss: 10058.2471 - val_loss: 9823.0127 Epoch 5/100 200/200 [==============================] - 1s 4ms/step - loss: 10058.0459 - val_loss: 9822.8135

as yon can see, the loss and validation loss is pretty HIGH, and I believe this is not going to produce a good model.

What am I doing wrong?

Thank you.

#3

Hi @Shiva_Subramanian, I have created 2 arrays x , y where x contains values (0,1,2,3,…) and y contains values equal to 2x+5 for the corresponding x.

x=[]
y=[]
for i in range(10000):
  x.append(i)
  y.append((2*i)+5)

and defined the model with 1 dense layer

model = tf.keras.Sequential([
    tf.keras.layers.Dense(units=1,input_shape=[1])
])

and trained the model for 100 epochs and got a very minimal loss.

image

Please refer to this gist for working code example. Thank You.