How do I quantize the weights of neural networks?

General Discussion
, ,
#1

I have a 102 filters of size 32 x 32. How do I regularize the weights of the neural network to become {0, 1} or { -1, 1} . If weights is more than 0, then w = 1. Else, w = -1 or 0.

Dense(inputs=1024, units=102, activation= , kernel_initializer=tf.random_normal_initializer(stddev=0.05))

#3

you can have a look here Quantization aware training comprehensive guide  |  TensorFlow Model Optimization , you can write then a custom quantization algorithm that clamp the values of your weights on the desired range (also activations).

then you annotate the layers you want with the custom quantizer and perform Quantization Aware Training (QAT) as in the tutorial

1 Like