Hi there
I have some issues with setting the learning rate for my model to predict time series data.
Problem (A): with the learning rate=1e-6, loss is higher and I have to reduce the prediction loss. If I tune the rate, loss result also changes again and get the desired result. But I am not sure how to find the best optimal learning rate for the SGD from the plot of the training result.
Problem (B): I got the prediction loss with learning rate 5e-6, it is a randomly chosen number and it worked. Nevertheless, it is not so clear how I can get 5e-6 from the curve . Please help me to clear my doubts.
This is the code how the training result curve was plotted
lrs = 1e-8 * (10 ** (np.arange(100) / 20))
plt.figure(figsize=(10, 6))
plt.grid(True)
plt.semilogx(lrs, history.history[...])
plt.tick_params(....')
plt.axis([1e-8, 1e-3, 0, 300])