Proximal-gradient type optimization for training

Keras
,
#1

I noticed that in Keras we can specify a constraint for model parameters (see the constraint argument of the Dense layer, for example). And the protocol the constraint object seems to implement is a Euclidean projection onto the constraint set.

Is it possible to use the same framework to implement a proximal-gradient type algorithm for training by implementing the regularizer’s proximal operator as a “constraint”?

1 Like
#3

Hi @alexshtf

Welcome to The TensorFlow Forum!

You can use TensorFlow Probability’s tfp.glm.fit_sparse API for proximal gradient descent. There are also some other APIs available for Hessian-informed proximal gradient descent such as
proximal_hessian_sparse_minimize() and proximal_hessian_sparse_one_step().

1 Like