Where can I find implemented examples of second order optimizers using tf.GradientTape.jacobian, tf.hessian or similar methods, if there are any? Thanks!
Abstract. Second-order optimization technique is the advances of first-order optimization in neural networks. It provides an addition curvature information of an objective function that adaptively estimate the step-length of optimization trajectory in training phase of neural network.
There is some second order optimizer in TFP with requests to exposed it as Keras optimizer: