Converting Keras Tensors to Tensorflow Tensors

General Discussion
, ,
#1

Hello, I’m trying to work with this project.

It works with TF 2.14, but not with 2.16.1. I need to get 2.16.1 to work because 1 of my GPUs only seems to work with 2.16.1.

The problems seem to come from converting Keras Tensors to TF. One problem is this line:

return inputs + self.pos_encoding[:, : tf.shape(inputs)[1], :]

It gives an error, I think because “inputs” is a Keras Tensor and the next term in the addition is TF.

Expected float32, but got SparseTensor(indices=Tensor(“Placeholder_1:0”, shape=(None, 3), dtype=int64), values=Tensor(“Placeholder:0”, shape=(None,), dtype=float32), dense_shape=Tensor(“PlaceholderWithDefault:0”, shape=(3,), dtype=int64)) of type ‘SparseTensor’.

If I switch the order of the addition to

return self.pos_encoding[:, : tf.shape(inputs)[1], :] + inputs

I get:

Expected float32 passed to parameter ‘y’ of op ‘AddV2’, got SparseTensor(indices=Tensor(“Placeholder_1:0”, shape=(None, 3), dtype=int64), values=Tensor(“Placeholder:0”, shape=(None,), dtype=float32), dense_shape=Tensor(“PlaceholderWithDefault:0”, shape=(3,), dtype=int64)) of type ‘SparseTensor’ instead. Error: Expected float32, but got SparseTensor(indices=Tensor(“Placeholder_1:0”, shape=(None, 3), dtype=int64), values=Tensor(“Placeholder:0”, shape=(None,), dtype=float32), dense_shape=Tensor(“PlaceholderWithDefault:0”, shape=(3,), dtype=int64)) of type ‘SparseTensor’.

If I comment out the PositionalEncoding layer, further down I get these types of errors, which I don’t know how to handle either. Anybody know what’s happening? Thanks!

ValueError: A KerasTensor cannot be used as input to a TensorFlow function. A KerasTensor is a symbolic placeholder for a shape and dtype, used when constructing Keras Functional models or Keras Functions. You can only use it as input to a Keras layer or a Keras operation (from the namespaces keras.layers and keras.operations). You are likely doing something like:

x = Input(...)
...
tf_fn(x)  # Invalid.

What you should do instead is wrap tf_fn in a layer:

class MyLayer(Layer):
    def call(self, x):
        return tf_fn(x)

x = MyLayer()(x)
#3

Hi @Uma_Jay, This error occurs due to passing sparse input to the model layers. only some Keras layers support sparse inputs. For example,

input = tf.keras.layers.Input(shape=(4), sparse=True)
output = tf.keras.layers.ReLU()(input)
model = tf.keras.Model(inputs=[input], outputs=[output])
print(model(np.array([[1, 0, 1, 0]])))

The above code gives
Failed to convert elements of SparseTensor(indices=Tensor("Placeholder_1:0", shape=(None, 2), dtype=int64), values=Tensor("Placeholder:0", shape=(None,), dtype=float32), dense_shape=Tensor("PlaceholderWithDefault:0", shape=(2,), dtype=int64)) to Tensor. Consider casting elements to a supported type. See https://www.tensorflow.org/api_docs/python/tf/dtypes for supported TF dtypes.

To overcome this error you can pass the sparse tensor to the dense layer and pass the dense output to the other model layers

input = tf.keras.layers.Input(shape=(4), sparse=True)
i=tf.keras.layers.Dense(4)(input)
output = tf.keras.layers.ReLU()(i)
model = tf.keras.Model(inputs=[input], outputs=[output])
print(model(np.array([[1, 0, 1, 0]])))

#output: tf.Tensor([[0.        0.        0.3541779 0.       ]], shape=(1, 4), dtype=float32)

Thank You.