TCAV use in other applications beside image classification and text

Hi, I’m trying to use TCAV for Object detection and semantic segmentation models, but I am having a hard time figuring out how to do it. For me is especially hard on the Object detection models that TensorFlow Object Detection API provides. The SSDMetaArch architecture is a weird combination of Keras and Graphs. Hence I’m totally lost. If anyone has a clue what the right approach to this problem is, will be awesome :smile:. I really appreciate any help you can provide.

1 Like

Hi, to update my recent findings and issues.


  • tracing back to the objectdetection class, I found out how to get the prediction layer of the SSD.

  • Also, I successfully recreate the loss function for this layer that works in the same fashion as KerasWrapper

  • I copy the KerasWrapper class to build a new KerasWrapperSSD (the SSD layer are Keras layers).

  • This class is also on of the TCAV repository

  • I load the model as I did to found the model’s layers, no error in that, except that the model seems that is no loading the backbone or any other layer. Hence I cannot extract the bottlenecks.

  def import_keras_model(self, saved_path):
    """Loads keras model, fetching bottlenecks, inputs and outputs."""
    self.ends = {}
    PATH_TO_CFG=saved_path + "/pipeline.config"
    PATH_TO_CKPT = saved_path + "/checkpoint"
    configs = config_util.get_configs_from_pipeline_file(PATH_TO_CFG)
    model_config = configs['model']
    detection_model =, is_training=False)
    ckpt = tf.compat.v2.train.Checkpoint(model=detection_model)
    ckpt.restore(os.path.join(PATH_TO_CKPT, 'ckpt-0')).expect_partial()
    self.model = detection_model

If I the load model in this fashion on other script works, but as a class of TCAV, the model is just a simple shell
I don’t know what is missing or what I’m doing wrong. I really, really appreciate any help.

1 Like

I am facing with the same issue trying to load Object Detection model for tcav. Did you by any chance find solution to this issue ?