Tensorflow 2 models on Jetson Nano

Hi,

I’m currently working on an AI project using an NVIDIA Jetson Nano (4GB) and TensorFlow 2 where we were planning on using a Faster R-CNN Inception ResNet V2 640x640 model. We tried using TF-TRT to reduce the network, but it seems to be too big to fit in, the vRAM memory is not big enough and using Swap doesn’t solve the issue.

We have done several tests and, for the moment, the heaviest network from the TensorFlow Model Zoo we managed to get working is the SSD MobileNet V2 FPNLite 640x640.

I’ve been searching for a list of networks that have been tested on this device for TF2, but I can’t seem to find it. I know of the existence of this list, but it is for TF1 and doesn’t involve the TF2 Model Zoo models.

Has anyone tried to get something bigger running on the Jetson? Which network and how? Is there any official documentation on which models are feasible for only 4GB?

Also, I’d like to understand why I can execute these models on my CPU but not on the GPU, as the system runs out of (RAM/vRAM) memory. I’m no expert and it seems a bit weird.

Thank you in advance.

The Nano is not designed to run large neural networks. I repeat: execute. The work must do this upstream and think lite

You can find model memory with:

You can try with smaller model and explore some model optimizations:

You can also play with the TF-TRT memory managment params:
https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#mem-manage

1 Like