Does cuDNN's 2**31 elements limit apply to TF 2.5.0?

General Discussion
,
#1

Hi there,

TensorFlow 2.5.0 is based on cuDNN 8.x, which no longer has the 2 billion elements per tensor limitation of cuDNN 7.x:

https://docs.nvidia.com/deeplearning/cudnn/release-notes/rel_8.html

I would like to know: does TensorFlow have a limitation of its own, independently of cuDNN, limiting the number of elements in a tensor?

Thank you,
OD

1 Like
#2

Hi @odesm

Welcome to the TensorFlow Forum!

Yes, TensorFlow 2.5.0 and later versions which are based on cuDNN 8.x does not have the 2 billion elements per tensor limitation that was present in cuDNN 7.x. Now, you can use TensorFlow to train and run models with tensors that contain more than 2 billion elements which will improve the memory efficiency as well as improve the performance.