Auto shard policy for tf.keras.utils.Sequence

Hello there,

So I am feeding a Sequence object to a recommender model. During training, I am getting the message:

AUTO sharding policy will apply DATA sharding policy as it failed to apply FILE sharding policy because of the following reason: Did not find a shardable source, walked to a node which is not a dataset: name: "FlatMapDataset/_2

My question is how can I apply or disable the sharding policy? I understand that you have to create an options object and apply it to a object, but I am not sure how does this work with a Sequence object instead.

Any help is greatly appreciated. Thanks.

Hi @Andres_Toledo, You can disable the sharding using

options =
options.experimental_distribute.auto_shard_policy =

For more details please refer to this documentation. Thank You.