Using Ragged Tensors in Conv2D

General Discussion
,
#1

I am trying to use Ragged Tensors for a Convolution model. The input tensors have a shape of (None, None, 8) excluding the batch size and ‘None’ represents the ragged dimension

Here is a simple model for taking in Ragged Tensor inputs. The tensors are of dtype ‘tf.float32’

m = Input(shape = (None, None, 8), ragged = True)
m1 = Conv2D(16, 3, strides = 1)(m)

Model(inputs = m, outputs = m1)

But it shows an error as below

TypeError                                 Traceback (most recent call last)
C:\Users\VAISHN~1\AppData\Local\Temp/ipykernel_15148/4293567980.py in 
      1 m = Input(shape = (None, None, 8), ragged = True)
----> 2 m1 = Conv2D(16, 3, strides = 1)(m)
      3 
      4 Model(inputs = m, outputs = m1)

~\AppData\Roaming\Python\Python38\site-packages\keras\utils\traceback_utils.py in error_handler(*args, **kwargs)
     68             # To get the full stack trace, call:
     69             # `tf.debugging.disable_traceback_filtering()`
---> 70             raise e.with_traceback(filtered_tb) from None
     71         finally:
     72             del filtered_tb

~\AppData\Roaming\Python\Python38\site-packages\tensorflow\python\framework\tensor_util.py in make_tensor_proto(values, dtype, shape, verify_shape, allow_broadcast)
    544       str_values = [compat.as_bytes(x) for x in proto_values]
    545     except TypeError:
--> 546       raise TypeError(f"Failed to convert elements of {values} to Tensor. "
    547                       "Consider casting elements to a supported type. See "
    548                       "https://www.tensorflow.org/api_docs/python/tf/dtypes "

TypeError: Exception encountered when calling layer "conv2d" (type Conv2D).

Failed to convert elements of tf.RaggedTensor(values=tf.RaggedTensor(values=Tensor("Placeholder:0", shape=(None, 8), dtype=float32), row_splits=Tensor("Placeholder_1:0", shape=(None,), dtype=int64)), row_splits=Tensor("Placeholder_2:0", shape=(None,), dtype=int64)) to Tensor. Consider casting elements to a supported type. See https://www.tensorflow.org/api_docs/python/tf/dtypes for supported TF dtypes.

Call arguments received by layer "conv2d" (type Conv2D):
  • inputs=tf.RaggedTensor(values=tf.RaggedTensor(values=Tensor("Placeholder:0", shape=(None, 8), dtype=float32), row_splits=Tensor("Placeholder_1:0", shape=(None,), dtype=int64)), row_splits=Tensor("Placeholder_2:0", shape=(None,), dtype=int64))

I also checked if Conv2D supports Ragged Tensors. Below is the class definition of Conv2D from tensorflow

class Conv2D(Conv):
  """2D convolution layer (e.g. spatial convolution over images).

  This layer creates a convolution kernel that is convolved
  with the layer input to produce a tensor of
  outputs. If `use_bias` is True,
  a bias vector is created and added to the outputs. Finally, if
  `activation` is not `None`, it is applied to the outputs as well.

  When using this layer as the first layer in a model,
  provide the keyword argument `input_shape`
  (tuple of integers or `None`, does not include the sample axis),
  e.g. `input_shape=(128, 128, 3)` for 128x128 RGB pictures
  in `data_format="channels_last"`. You can use `None` when
  a dimension has variable size.

Could someone please explain what the issue is? Ragged Tensors are compatible with other layers like Dense and LSTM but I am not sure why it shows this error in Conv2D

#3

@Vaishnav_Bhaskaran,

The Conv2D layer expects the input to be a fixed shape. To fix this, you can use the to_tensor() method to convert the ragged tensor to a dense tensor.

import tensorflow as tf
from tensorflow.keras.layers import Input, Conv2D
from tensorflow.keras.models import Model

m = Input(shape=(None, None, 8), ragged=True)
m1 = m.to_tensor()
m2 = Conv2D(16, 3, strides=1)(m1)
model = Model(inputs=m, outputs=m2)

When using None in the input_shape argument of Conv2D, it indicates that a particular dimension can have a variable size, but it still assumes a fixed shape within each input example.

Thank you!

1 Like
#4

@chunduriv,

Thank you for your answer!

Yes. This makes the Conv2D layer take in tensors with different dimensions. But the to_tensor() adds a default_value while converting it into a normal tensor. My intention to use ragged tensors was to avoid padding the regular tensors as it affects the performance. Is there a way to avoid the padding?

#6

@Vaishnav_Bhaskaran,

You can use `None` whena dimension has variable size.

If we didn’t specify the shape (i.e None) then it will infer the shape from input image.

My intention to use ragged tensors was to avoid padding the regular tensors as it affects the performance. Is there a way to avoid the padding?

No, since it expects fixed length input.

Thank you!

#8

@chunduriv,

okay, got it. Thank you for your help!