Nones returned in tape.gradient in Conv-TasNet Implementation

beepbopboop · February 15, 2022, 3:03am

Hi all;

I am attempting to implement Conv-TasNet for a course project. My train_step code is at:

JustinGoheen/tf2-ConvTasNet/blob/87ce38922aa1e80aa7b0714cb52ace3a75a5a9eb/research/algorithm.py#L340

      
        
                    raise KeyError(
                        f"Expected one of (weights, variables, x, pred). Got: {self.apply_loss_to}"
                    )
                # ENDNOTE
            
            
    # Trackers
                self.mean_loss = tf.keras.metrics.Mean(name="mean-loss")
                self.cossim_loss = tf.keras.metrics.CosineSimilarity(name="cos-sim")
                self.rmse_loss = tf.keras.metrics.RootMeanSquaredError(name="rmse-loss")
            
            
def train_step(self, batch):
                """custom training
            
            
    # Note
            
            
        see https://keras.io/examples/audio/melgan_spectrogram_inversion/
                    for a guide on writing a custom train_step.
                """
                x_train, y_true = batch
            
            
    with tf.GradientTape(persistent=True) as tape:

I’ve read the docs concerning possible causes of having all Nones returned by tape.gradient; I’ve made sure to remove any NumPy code from my custom loss function, and am not quite sure what else may be causing the issue.

I don’t think the error is caused by the loss function; I ran a debugging session with a built-in loss function, and the same result (all Nones) occurs.

The trainable_variables are tracked by tape, meaning the loss should be applied to a list of Variables as intended; so this should work, at least according to my knowledge of TensorFlow and Keras – which only spans about 6 weeks at the time of this posting.

I used the MelGAN example as a guide for how to implement a similar model; a main difference I can see in their code and mine is, I am not using (what appears to be) functional API concepts in my ConvBlock. Could that be the issue?

Regarding system/env:

Mac M1 Max
Python 3.9.7
TensorFlow/Keras 2.7

Conda environment is managed with miniforge.

Renu_Patel · October 4, 2023, 9:43am

Hi @beepbopboop

Welcome to the TensorFlow Forum!

The code link mentioned in this issue is not accessible. Please provide minimal reproducible code to replicate the error and fix it. Thank you

beepbopboop · October 4, 2023, 1:07pm

Hi Renu! Apologies. This was so long ago and I have deleted the repo.

Tanget –– how do I update my username?

Renu_Patel · October 4, 2023, 1:15pm

You can check in your account summary - preferences - edit option.