Remove keras loss averaging behavior

[Niccolo_Stefanini]

I am currently having some difficulties understanding this behaviour of Keras during training.

At every epoch the loss goes down very smoothly. Then at the next epoch first step the loss steps down abruptly. So on for every epoch, having a loss plot which looks like and exponential stair going down. (see picture)

I think this is due from the fact that keras is averaging the loss over the epoch automatically, but this looks very ugly!

How can I remove this behaviour? I don’t find it documented anywhere, but it’s the only explanation I can give to this plot as

• the model is not overfitting as validation metrics are going okay
• keras is doing this something similar for metrics already (https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/ProgbarLogger: ref stateful_metrics)

Thanks for helping

[Renu_Patel]

Hi @Niccolo_Stefanini

Welcome to the TensorFlow Forum!

This is completely depends on which loss function you have used during the model compilation. Model will compute the loss between predicted and actual value based on the defined loss function. Generally the loss averaging is done when the MeanAbsoluteError, MeanAbsolutePercentageError, MeanSquaredError and MeanSquaredLogarithmicError loss function has been used in the model.

We may need a reproducible code along with the model definition and model compilation code to provide you better understanding in this.

Meanwhile, please have a look at these various types of available Keras Loss functions and you can select and use these loss functions as per your requirements in your model. Thank you.