I am currently having some difficulties understanding this behaviour of Keras during training.
At every epoch the loss goes down very smoothly. Then at the next epoch first step the loss steps down abruptly. So on for every epoch, having a loss plot which looks like and exponential stair going down. (see picture)
I think this is due from the fact that keras is averaging the loss over the epoch automatically, but this looks very ugly!
How can I remove this behaviour? I don’t find it documented anywhere, but it’s the only explanation I can give to this plot as
- the model is not overfitting as validation metrics are going okay
- keras is doing this something similar for metrics already (https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/ProgbarLogger: ref stateful_metrics)
Thanks for helping