Apply gradient method with mirrored strategy

Research & Models
#1

I have been trying to make a custom training function for my models and have stumbled upon an error that I can’t seem to fix nor understand. When trying to update my weights using the method optimize.apply_gradients, I get the following error :

“Type Error : tuple indices must be integers or slices, not MirroredVariable”

I assume that this is due to my models being originally created using a mirrored strategy to use multiple gpus and dispatch the work load. Is there a way to work around this issue ?

#3

Hi @Redo0606,

You can get the underlying variables from the MirroredVariable objects. Once you have the underlying variables, you can use the optimizer.apply_gradients() function to update them.

I hope this helps you!

Thanks.

#5

Hello,

Thank you for your response. But to rephrase my question : does the method optimizer.apply_gradients() work with MirroredVariables ? Or is there a way to use this function with MirroredVariables ?

Best Regards.

#7

Hi @Redo0606 ,

As per my experience, the method optimizer.apply_gradients() works with MirroredVariables.

Thanks.