Remove keras loss averaging behavior

I am currently having some difficulties understanding this behaviour of Keras during training.

At every epoch the loss goes down very smoothly. Then at the next epoch first step the loss steps down abruptly. So on for every epoch, having a loss plot which looks like and exponential stair going down. (see picture)

I think this is due from the fact that keras is averaging the loss over the epoch automatically, but this looks very ugly!

How can I remove this behaviour? I don’t find it documented anywhere, but it’s the only explanation I can give to this plot as

Thanks for helping