Code error using Gradient Tape

Hi all,
I tried to implement a very basic classification algorithm using tensorflow API
the steps are:

  1. creating synthetic data
  2. define the architecture prediction = tf.matmul(inpurs,W) + b
  3. iterate on training step

For some reason the GradientTape instance could not find W,b so I used local function variables
the code is:

import tensorflow as tf

W = tf.Variable(initial_value = tf.random.uniform((input_dims,output_dims)))
b = tf.Variable(initial_value = tf.random.uniform((output_dims,)))

def square_loss(preds,labels):
loss = tf.square(preds-labels)

return tf.reduce_mean(loss)

def training_step(data,labels,local_W,local_b):
with tf.GradientTape() as g:
predictions = tf.matmul(data,local_W)+local_b
loss = square_loss(predictions,labels)

grad_W,grad_b = g.gradient(loss, [local_W,local_b])
return loss,grad_W,grad_b

step_size = 0.02
loss_val = []
for i in range(40):
(loss,grad_W,grad_b) = training_step(data,targets,W,b)
W = tf.add(W,grad_W*(-step_size))
b = tf.add(b,grad_b*(-step_size))

What happens is at iteration 2 the gradients calculated by GradientTape instance are none type
I checked the loss function (on which gradient is calculated) and it has a valid number
Can someone please explain why the gradient calculation is not working?

Can you structure your code better. It’s partially marked with code and mostly not, there’s no indents, etc.

Also, this is an example of how I’ve used gradientTape successfully:

def step(real_x, real_y):
    with tf.GradientTape() as tape:
        # Make prediction
        pred_y = model(real_x.reshape((-1, 28, 28, 1)))
        # Calculate loss
        model_loss = tf.keras.losses.categorical_crossentropy(real_y, pred_y)
    # Calculate gradients
    model_gradients = tape.gradient(model_loss, model.trainable_variables)
    # Update model
    optimizer.apply_gradients(zip(model_gradients, model.trainable_variables))

This is from section 2.2 of tf.GradientTape Explained for Keras Users | by Sebastian Theiler | Analytics Vidhya | Medium

I didn’t see an optimiser.apply_gradients() call above, you seem to be trying to apply them manually.

So instead of using the optimizer I am just calculating the gradient and tweak the weights with a fixed learning rate
I saw this done in a book called Deep learning with python, it implements model fit basics without usage of keras (actually the writer was one of keras developers).
I will try to load my code better