How to add Keras layer output to a custom loss component?

I have my old code written in Keras where I implemented a convolutional autoencoder. I wanted to add another loss component to its loss which is based on the output of a layer called ‘z’:
z_output = encoder.get_layer('z').output
and then use this output for another function from sklearn metrics:
silhouette_loss = - silhouette_coeff(z_output, names)

This function expects a 2D array as an input, as a result I am getting the error:
TypeError: You are passing KerasTensor(type_spec=TensorSpec(shape=(None, 32), dtype=tf.float32, name=None), name='z/add:0', description="created by layer 'z'"), an intermediate Keras symbolic input/output, to a TF API that does not allow registering custom dispatchers, such as tf.cond, tf.function, gradient tapes, or tf.map_fn. Keras Functional model construction only supports TF API calls that *do* support dispatching, such as tf.math.addortf.reshape. Other APIs cannot be called directly on symbolic Kerasinputs/outputs. You can work around this limitation by putting the operation in a custom Keras layer call and calling that layer on this symbolic input/output.

What can I do to use the output of layer ‘z’ for my custom loss component? My goal is to maximize the output of the silhouette_coeff() during model training. Thanks!