There’s an existing thread on this: https://github.com/tensorflow/tensorflow/issues/53529.
Following the suggestions, I spun up a Workbench instance on GCP with TensorFlow 2.6 Enterprise which comes with TensorRT:
>>> tf.sysconfig.get_build_info()
OrderedDict([('cpu_compiler', '/usr/bin/gcc-5'),
('cuda_compute_capabilities',
['compute_37',
'compute_60',
'compute_61',
'compute_70',
'compute_75',
'compute_80']),
('cuda_version', '11.0'),
('cudnn_version', '8'),
('is_cuda_build', True),
('is_rocm_build', False),
('is_tensorrt_build', True)])
I tried the following code:
import tensorflow as tf
resnet = tf.keras.applications.ResNet50(weights="imagenet", include_top=True)
resnet.save("resnet")
TENSORRT_MODEL_DIR = f"tensorrt-resnet"
params = tf.experimental.tensorrt.ConversionParams(
precision_mode='FP16'
)
converter = tf.experimental.tensorrt.Converter(
input_saved_model_dir="resnet",
conversion_params=params
)
converter.convert()
converter.save(TENSORRT_MODEL_DIR)
The kernel restarts automatically when it tries to create the Converter object. I have tried with tensorflow.python.compiler.tensorrt too (as shown here) but it didn’t help.
Any help?
@Sayak_Paul,
Recent tensorflow versions are build with TensorRT.
Tensorflow-2.11:
import tensorflow as tf
print("TensorFlow version:", tf.__version__)
import tensorflow.compiler as tf_cc
linked_trt_ver=tf_cc.tf2tensorrt._pywrap_py_utils.get_linked_tensorrt_version()
print(f"Linked TRT ver: {linked_trt_ver}")
Output:
TensorFlow version: 2.11.0
Linked TRT ver: (7, 2, 2)
Tensorflow-2.12:
import tensorflow as tf
print("TensorFlow version:", tf.__version__)
import tensorflow.compiler as tf_cc
linked_trt_ver=tf_cc.tf2tensorrt._pywrap_py_utils.get_linked_tensorrt_version()
print(f"Linked TRT ver: {linked_trt_ver}")
Output:
TensorFlow version: 2.12.0
Linked TRT ver: (8, 4, 3)
Could you please try with the latest version and let us know if the issue persists?
Thank you!
When I install TensorRT by deb file. Because my CUDA version is 11.8, I install TensorRT 8.5.1. Install step is
os=“ubuntuxx04” tag=“8.x.x-cuda-x.x” sudo dpkg -i nv-tensorrt-local-repo-${os}-${tag}_1.0-1_amd64.deb sudo cp /var/nv-tensorrt-local-repo-${os}-${tag}/*-keyring.gpg /usr/share/keyrings/ sudo apt-get update sudo apt-get install tensorrt
but after sudo apt-get install tensorrt, output is:
The following additional packages will be installed:
cuda-cccl-12-1 cuda-cccl-12-2 cuda-cudart-12-1 cuda-cudart-12-2
cuda-cudart-dev-12-1 cuda-cudart-dev-12-2 cuda-driver-dev-12-1
cuda-driver-dev-12-2 cuda-nvcc-12-1 cuda-toolkit-12-1-config-common
cuda-toolkit-12-2-config-common cuda-toolkit-12-config-common
cuda-toolkit-config-common libcublas-12-2 libcublas-dev-12-2 libcudnn8
libcudnn8-dev libnvinfer-bin libnvinfer-dev libnvinfer-dispatch-dev
libnvinfer-dispatch8 libnvinfer-headers-dev libnvinfer-headers-plugin-dev
libnvinfer-lean-dev libnvinfer-lean8 libnvinfer-plugin-dev
libnvinfer-plugin8 libnvinfer-samples libnvinfer-vc-plugin-dev
libnvinfer-vc-plugin8 libnvinfer8 libnvonnxparsers-dev libnvonnxparsers8
libnvparsers-dev libnvparsers8
The following NEW packages will be installed:
cuda-cccl-12-1 cuda-cccl-12-2 cuda-cudart-12-1 cuda-cudart-12-2
cuda-cudart-dev-12-1 cuda-cudart-dev-12-2 cuda-driver-dev-12-1
cuda-driver-dev-12-2 cuda-nvcc-12-1 cuda-toolkit-12-1-config-common
cuda-toolkit-12-2-config-common cuda-toolkit-12-config-common
cuda-toolkit-config-common libcublas-12-2 libcublas-dev-12-2 libcudnn8
libcudnn8-dev libnvinfer-bin libnvinfer-dev libnvinfer-dispatch-dev
libnvinfer-dispatch8 libnvinfer-headers-dev libnvinfer-headers-plugin-dev
libnvinfer-lean-dev libnvinfer-lean8 libnvinfer-plugin-dev
libnvinfer-plugin8 libnvinfer-samples libnvinfer-vc-plugin-dev
libnvinfer-vc-plugin8 libnvinfer8 libnvonnxparsers-dev libnvonnxparsers8
libnvparsers-dev libnvparsers8 tensorrt
so if I install this TensorRT version, my CUDA upgrade from 11.8 to 12.1?