Cannot convert a symbolic Keras input/output to a numpy array

Research & Models
#1

Hi all,

I have a custom layer that inherits from tf.keras.layers.Layer. in that layer, I implemented the init function and the call function.

class MyDense(tf.keras.layers.Layer):
    def __init__(self, **kwargs):
            super(MyDense, self).__init__(**kwargs)
            }
    def call(self, inputs:List, **args, **kwargs) -> tf.Tensor:
            vec1 = inputs[0]
            vec2 = inputs[1]
            with tf.name_scope('bilinear'):
                return tfa.image.dense_image_warp(vec1, vec2)

I have an other layer which will contain 2 of MyDense layer.

The problem I am seeing is caused by the decorator @tf.function and hence I am getting that error.
did someone face this before and how did you solve it? after some tries, I think it could be a bug but I am not sure.

When I tried to implement that dense_image_warp in my files and remove that decorator I got an other error which is the following:

Could not build a TypeSpec for name: "tf.debugging.assert_equal/assert_equal_1/Assert/Assert"
op: "Assert"
input: "tf.debugging.assert_equal/assert_equal_1/All"
input: "tf.debugging.assert_equal/assert_equal_1/Assert/Assert/data_0"
input: "tf.debugging.assert_equal/assert_equal_1/Assert/Assert/data_1"
input: "tf.debugging.assert_equal/assert_equal_1/Assert/Assert/data_2"
input: "strided_slice"
input: "tf.debugging.assert_equal/assert_equal_1/Assert/Assert/data_4"
input: "tf.debugging.assert_equal/assert_equal_1/y"
attr {
  key: "summarize"
  value {
    i: 3
  }
}
attr {
  key: "T"
  value {
    list {
      type: DT_STRING
      type: DT_STRING
      type: DT_STRING
      type: DT_INT32
      type: DT_STRING
      type: DT_INT32
    }
  }
}
 of unsupported type <class 'tensorflow.python.framework.ops.Operation'>.

when I tried to use tf.config.run_functions_eagerly(True) I got the same error as above.

#3

@MLEnthusiastic based on my understanding, I would suggest:

  1. Looking into the implementation of tfa.image.dense_image_warp to ensure compatibility with the @tf.function decorator. Check for any specific considerations when using it within a graph mode.

  2. Make sure your TensorFlow version aligns with the TensorFlow Addons (tfa) library. Upgrading both to the latest versions might resolve any compatibility issues that could be causing the error.

  3. Experiment with modifications to your custom layer. Start by simplifying the layer to identify the root cause of the issue. Adjust how inputs are handled or make changes to the call method.

  4. Refer to the official documentation for tfa.image.dense_image_warp to see if there are any reported issues or specific usage considerations. Additionally, visit the TensorFlow Addons community forums for insights and potential workarounds.

Let me know if the problem persists and we can tag team on resolving this.

#5

Hi @BadarJaffer ,

So I am using TensorFlow 2.14.0 and TFA 0.22.0 ( I believe, they are compatible). So, I guess point 2 is checked. (I used TensorFlow 2.13.0 and TensorFlow 2.9.1 too).

I have other functions called by call(), I just simplified it because the only issue I am seeing is with tfa.image.dense_image_warp. The concept of my class is based on some operations (bilinear so I call tfa.image.dense_image_warp, other stuff I call my functions). NB: my functions were decorated with @tf.function and when I removed it from my functions the problem was solved, but still with tfa.image.dense_image_warp. The latter is calling other functions and when I tried to copy it in my own files with its functions without @tf.function I got the other error (Could not build a TypeSpec for name: “tf.debugging.assert_equal/assert_equal_1/Assert/Assert” …).

I really do not understand why I am getting this error:
Cannot convert a symbolic Keras input/output to a numpy array with tfa.image.dense_image_warp?

If you need any other relevant information, please let me know.
Thanks

#7

Hi @BadarJaffer ,

Any updates?

#9

@MLEnthusiastic Apologies for not getting back to you early.

It’s good to know that you’ve tried different versions of TensorFlow, and the issue persists in the current setup.

The fact that removing the @tf.function decorator from your custom functions resolves part of the issue suggests that there might be a compatibility or interaction problem with the dense_image_warp function in a graph mode.

Considering your class concept involves bilinear operations with tfa.image.dense_image_warp and other custom functions, and the error persists even when copying dense_image_warp to your files without @tf.function, here are some more specific suggestions:

  1. Data Type Consistency:
  • Ensure that the data types of inputs and outputs throughout your operations, especially within dense_image_warp, are consistent. Mismatched data types can lead to the “Cannot convert a symbolic Keras input/output to a numpy array” error.
  1. Graph Mode Compatibility:
  • Since removing @tf.function from your custom functions helped, you might want to investigate how dense_image_warp interacts with the graph mode. Consider trying to use tf.function specifically for dense_image_warp and see if that resolves the issue.
  1. Review TensorFlow Operations:
  • Examine the operations inside dense_image_warp and other functions for any TensorFlow operations that might conflict with graph mode. Ensure that all operations are compatible with the symbolic computation.
  1. Error Message Investigation:
  • The error message related to “tf.debugging.assert_equal/assert_equal_1/Assert/Assert” might indicate an issue with assertions. Check if there are any assert statements or debugging-related operations causing conflicts.

If the issue persists, sharing the relevant code snippet where dense_image_warp is used, along with its interactions, would be helpful for a more in-depth analysis.