AttributeError: 'KerasTensor' object has no attribute 'node'

Abhijeet · January 27, 2022, 11:11pm

How to get rid of this:
AttributeError: 'KerasTensor' object has no attribute 'node'

Was working fine with TF 2.4.1 but giving problem with TF 2.7.0

Script

types_inp = Input(shape=(self.len_of_types, ), dtype='int32') 
.......
out = Dense(self.num_ways, activation='softmax')(prob)
model = keras.Model(inputs=[types_inp,..],outputs=out]

Error:

AttributeError: 'KerasTensor' object has no attribute 'node'

Bhack · January 28, 2022, 12:02am

I don’t know if you could minimize a share few lines of standalone code to reproduce this (or a Colab).

In the meantime you could check if it could be related to:

github.com/tensorflow/tensorflow

[TF 2.4] KerasTensor breaks typing compatibility

opened 01:39PM - 05 Nov 20 UTC

lgeiger

type:bug comp:keras TF 2.4

**System information** - Have I written custom code (as opposed to using a stoc…k example script provided in TensorFlow): yes - OS Platform and Distribution (e.g., Linux Ubuntu 16.04): macOS - TensorFlow installed from (source or binary): binary - TensorFlow version (use command below): v2.4.0-rc0 and tf-nightly - Python version: 3.8 ## The Problem Since version 2.4 functional Keras models use `KerasTensor` instead of `tf.Tensor` as layer output types. Unfortunately `KerasTensor` doesn't subclass `tf.Tensor` which breaks `isinstance(x, tf.Tensor)` checks: https://github.com/tensorflow/tensorflow/blob/6d9e0887f6bce8fbeeb430364e520d05350d96d5/tensorflow/python/keras/engine/keras_tensor.py#L63 The release notes recommend to use `tf.is_tensor` instead. In my opinion this is not really Pythonic and breaks compatibility with the [TF Types RFC](https://github.com/tensorflow/community/blob/master/rfcs/20200211-tf-types.md) (/cc @mdanatg) wich even mentiones that `tf.is_tensor` is expected to be deprecated eventually. Concretely, switching from `isinstance(x, tf.Tensor)` to `tf.is_tensor` is also not possible in all cases. E.g. this breaks usage of static typecheckers like `pytype` or `typeguard`: A common pattern which can also be found in TensorFlow Addons (/cc @seanpmorgan) is the following: ```python from typeguard import typechecked import tensorflow as tf @typechecked def foo(x: tf.Tensor): print(x.dtype) foo(tf.keras.Input(shape=(32, 32, 3))) # Throws in TF 2.4 since `isintance` is used for typechecking ``` ## Possible solutions 1. Make `KerasTensor` a subclass of `tf.Tensor`. @mihaimaruseac @fchollet is there a reason why this isn't currently the case? 2. Make `KerasTensor` a subclass of [`types.Tensor`](https://github.com/tensorflow/tensorflow/blob/6d9e0887f6bce8fbeeb430364e520d05350d96d5/tensorflow/python/types/core.py#L40-L54). I don't see any disadvantage of doing this in general, but it wouldn't really fix this issue since `types.Tensor` is not exposed as part of the public API yet so users would need to rely on private TensorFlow APIs 3. In usercode this could be fixed by directly relying on `KerasTensor` to replace the usage of `tf.Tensor` with: ```python from typing import Union from tensorflow.python.keras.engine.keras_tensor import KerasTensor TensorType = Union[tf.Tensor, KerasTensor] ``` I do not think this is a proper solution since it requires users to rely on internal APIs and implementation details that might change in the future. I am currious to hear back from you on what the bestpractices for type checking of Tensors are, or whether I am just missing somthing obvious here.

Abhijeet · January 28, 2022, 3:26pm

I was trying to reproduce codes used in the above git repo, here is what I found.

python -c 'import tensorflow; print(tensorflow.__version__)'

> 2.4.1

>>> from keras.engine.keras_tensor import KerasTensor  # imported from keras-team/keras
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'keras'
>>> 
>>> from tensorflow.python.keras.engine.keras_tensor import KerasTensor as KerasTensorFromTF # This import should not exist anymore
2022-01-28 10:19:31.459850: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
>>> 
>>> assert KerasTensorFromTF == KerasTensor
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'KerasTensor' is not defined

>>> import tensorflow as tf
>>> 
>>> from tensorflow.keras.backend import is_keras_tensor
>>> from tensorflow.python.keras.backend import is_keras_tensor as is_keras_tensor_tf  # this import should not exist anymore
>>> 
>>> assert is_keras_tensor(tf.keras.Input([10]))
>>> assert is_keras_tensor_tf(tf.keras.Input([10]))

TF 2.4.1 has no issue

While TF V 2.7.0

$ python -c 'import tensorflow; print(tensorflow.__version__)'
2.7.0

>>> from keras.engine.keras_tensor import KerasTensor  # imported from keras-team/keras
>>> 
>>> from tensorflow.python.keras.engine.keras_tensor import KerasTensor as KerasTensorFromTF # This import should not exist anymore
>>> 
>>> assert KerasTensorFromTF == KerasTensor
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError

>>> import tensorflow as tf
>>> 
>>> from tensorflow.keras.backend import is_keras_tensor
>>> from tensorflow.python.keras.backend import is_keras_tensor as is_keras_tensor_tf  # this import should not exist anymore
>>> 
>>> assert is_keras_tensor(tf.keras.Input([10]))
>>> assert is_keras_tensor_tf(tf.keras.Input([10]))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/ccs/home/asd/.conda/envs/gdy_v2/lib/python3.9/site-packages/tensorflow/python/keras/backend.py", line 1281, in is_keras_tensor
    raise ValueError('Unexpectedly found an instance of type `' + str(type(x)) +
ValueError: Unexpectedly found an instance of type `<class 'keras.engine.keras_tensor.KerasTensor'>`. Expected a symbolic tensor instance.

Bhack · January 28, 2022, 6:34pm

In your last example I think that probably there is something in the namespace:

import tensorflow as tf
from tensorflow.keras.backend import is_keras_tensor
from tensorflow.python.keras.engine import keras_tensor

y=keras_tensor.KerasTensor(tf.TensorSpec(shape=[None,10], dtype=tf.int32))
x=tf.keras.Input([10])
print(str(type(y)))
print(str(type(x)))
assert(isinstance(y,keras_tensor.KerasTensor))
assert(isinstance(x,keras_tensor.KerasTensor))

<class 'tensorflow.python.keras.engine.keras_tensor.KerasTensor'>
<class 'keras.engine.keras_tensor.KerasTensor'>

---------------------------------------------------------------------------

AssertionError                            Traceback (most recent call last)

<ipython-input-47-051d59c54d14> in <module>()
      9 print(str(type(x)))
     10 assert(isinstance(y,keras_tensor.KerasTensor))
---> 11 assert(isinstance(x,keras_tensor.KerasTensor))
     12 

AssertionError:

/cc @Scott_Zhu is this expected or there is a small bug in the namespace?

Scott_Zhu · January 28, 2022, 6:53pm

The code under tensorflow.keras is legacy, and should not be used.

The correct way of importing keras code is always from “from tensorflow import keras” or “import tensorflow as tf; tf.keras”

Directly “import keras” will access keras package python code directly (not exactly same as the public Keras API), which might lead to method/class not found.

Abhijeet · January 28, 2022, 7:11pm

At the first question, I always use

import tensorflow as tf
from tensorflow import keras

Still, I was getting

AttributeError: 'KerasTensor' object has no attribute 'node'

Bhack · January 28, 2022, 7:20pm

Do you think that we need to add a deprecation in Doc? As it is still in the official doc without any claim:

https://www.tensorflow.org/api_docs/python/tf/keras/backend/is_keras_tensor

Scott_Zhu · January 28, 2022, 8:11pm

# tf nightly (should be same as 2.7)
import tensorflow as tf
from tensorflow import keras

x=tf.keras.Input([10])
dense = tf.keras.layers.Dense(1)
y = dense(x)

assert(keras.backend.is_keras_tensor(x))
assert(keras.backend.is_keras_tensor(y))

assert(tf.keras.backend.is_keras_tensor(x))
assert(tf.keras.backend.is_keras_tensor(y))

Note that tensorflow.python.keras is not a valid import, and it is accessing legacy code that is about to delete. You should never import that directly.

Bhack · January 28, 2022, 9:03pm

In the end you could only use something like:

import tensorflow as tf

x=tf.keras.Input([10])
print(str(type(x)))
tf.keras.backend.is_keras_tensor(x)

Scott_Zhu · January 28, 2022, 9:46pm

Correct. tf.keras.backend.is_keras_tensor() should be the only public API to use.

Abhijeet · January 28, 2022, 10:28pm

Scott_Zhu:

from tensorflow import keras

x=tf.keras.Input([10])
dense = tf.keras.layers.Dense(1)
y = dense(x)

assert(keras.backend.is_keras_tensor(x))
assert(keras.backend.is_keras_tensor(y))

assert(tf.keras.backend.is_keras_tensor(x))
assert(tf.keras.backend.is_keras_tensor(y))

I guess it was arising because

import tensorflow as tf
from tensorflow import keras
from tensorflow.python.keras.layers import Input, Dense,LSTM

now I moved to

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import Input, Dense, LSTM

no issue as of now with TF 2.7.0

Bhack · January 28, 2022, 10:48pm

from tensorflow.python.keras.layers import Input, Dense,LSTM

tensorflow.python.* is a private API namespace, but as you know it could not be strongly enforced in python.