Imagine a custom layer that has a weight vector that multiplies the input vector element-wise (no biases).
I need the weight vector to be such that the norm of the elements along axis=1 can be at max some value say, ‘max_value’.
tf.clip_by_norm(**kwargs) gives the norms to be exactly ‘max_value’ along axis = 1. But I expect the norms along axis = 1 to be anywhere between (0, ‘max_value’).
Any help?