Problem computing gradients for Grad-CAM visualization

General Discussion
, ,
#1

Hi,
I want to visualize a custom CNN (pre-trained feature extractor plus classification head finetuned on a new task) using Grad-CAM. I started with the example in Grad-CAM class activation visualization
Here is how the custom model looks like:

import tensorflow as tf
IMG_SHAPE = (299, 299, 3)
num_classes = 5

data_augmentation = tf.keras.Sequential([
tf.keras.layers.experimental.preprocessing.RandomFlip(‘horizontal’),
tf.keras.layers.experimental.preprocessing.RandomRotation(0.1) ])

base_model = tf.keras.applications.InceptionV3(input_shape=IMG_SHAPE,include_top=False,weights=‘imagenet’)
preprocess_input = tf.keras.applications.inception_v3.preprocess_input
global_avg_layer = tf.keras.layers.GlobalAveragePooling2D()
dense_layer = tf.keras.layers.Dense(1024, activation = ‘relu’)
predict_layer = tf.keras.layers.Dense(num_classes)

define model

inputs = tf.keras.Input(shape=IMG_SHAPE)
x = data_augmentation(inputs)
x = preprocess_input(x)
x = base_model(x, training=False)
x = global_avg_layer(x)
x = dense_layer(x)
x = tf.keras.layers.Dropout(0.2)(x)
outputs = predict_layer(x)
model = tf.keras.Model(inputs, outputs)

Then I need to calculate the output of the last conv layer and then compute the gradient of the top predicted class w.r.t. the activations of this last conv layer. However, using the following code to access the output of the last conv layer throws a “graph disconnected” error:

last_conv_layer = model.get_layer(‘inception_v3’).get_layer(‘conv2d_93’)
grad_model = tf.keras.models.Model(
[model.inputs], [last_conv_layer.output, model.output]
)

I don’t understand why the grad_model isn’t able to give me the outputs of the last conv layer. Can anyone please help me debug this?

1 Like
#2

Below may helps you ( please modify VGG and prediction layer).

You can set [data_augumentation] and [preprocess_input] to “input_tensor” option.
However it is not good to get a deterministic heatmap.

    
base_model = tf.keras.applications.VGG16(input_shape=(229,229,3),
                                       include_top=False,
                                       weights='imagenet',
                                       classes = len(class_names))

last_layer=base_model.layers[-1]
last_layer_outputshape=last_layer.output_shape
dense_unit_to_set=last_layer_outputshape[-1]

x2=base_model.output
x2 = tf.keras.layers.GlobalAveragePooling2D()(x2)
x2 = tf.keras.layers.Dropout(0.2)(x2)

x2=tf.keras.layers.Dense(dense_unit_to_set,activation='relu')(x2)

prediction_layer = tf.keras.layers.Dense(len(class_names),activation="softmax")#
x2 = prediction_layer(x2)

model = tf.keras.Model(inputs=base_model.input, outputs=x2)
    

base_learning_rate = 0.0001
batch_size= BATCH_SIZE
momentum_usage=0.99
initial_epochs = 50
sgd=tf.keras.optimizers.SGD(learning_rate=base_learning_rate,momentum=momentum_usage)

model.compile(optimizer=sgd,loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True),metrics=['accuracy'])