No OpKernel was registered to support Op ‘MaxPool3D’

General Discussion
, ,
#1

Hi Experts,

I’m a researcher in the filed of neuroscience using MRI techniques.
I’m new to pyhton and tensorflow.

I have the same question raised by Dr. Roger ( Tensorflow question (MaxPool3d, MaxPool3D, MaxPooling3D)!)). I have the same trouble " No OpKernel was registered to support Op ‘MaxPool3D’".

In my understanding (ModelCheckpoint callback fails when mixed precision is enabled in TF 2.11.0 · Issue #349 · keras-team/tf-keras · GitHub), bfloat 16 did not work for bfloat 16 in the tf.keras.layers.MaxPooling3D one year ago. But the keras team now fixed this issue. Is that correct? Please tell me if my understanding is correct or not. If this issue is fixed, any other problems??

I’m using python3.10 on mac terminal and installed the tensorflow 2.12.0.
Mac OS is ventura 13.6.1.

The code history as follows:

input the folliwing:
python3 /Users/username/Downloads/SHIVA_PVS/predict_one_file.py -i /Users/username/mri/young_healthy/MRI_resliced.nii -m /Users/username/Downloads/SHIVA_PVS/PVS/v1/T1.PVS/20211030-162753_Unet3Dv2-10.7.2-1.8-T1.VRS_fold_1x6_pi_fold_0_model.h5 -b /Users/username/mri/young_healthy/brain_reslice.nii -o /Users/username/mri/young_healthy/pv_MRI_resliced --verbose --gpu 0

terminal returns:
2023-11-16 07:55:41.891278: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Trying to run inference on GPU 0
WARNING:tensorflow:Mixed precision compatibility check (mixed_float16): WARNING
The dtype policy mixed_float16 may run slowly because this machine does not have a GPU. Only Nvidia GPUs with compute capability of at least 7.0 run quickly with mixed_float16.
If you will use compatible GPU(s) not attached to this host, e.g. by running a multi-worker model, you can ignore this warning. This message will only be logged once
INFO : Predicting fold : 20211030-162753_Unet3Dv2-10.7.2-1.8-T1.VRS_fold_1x6_pi_fold_0_model
Traceback (most recent call last):
File “/Users/username/Downloads/SHIVA_PVS/predict_one_file.py”, line 125, in
prediction = model.predict(
File “/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py”, line 70, in error_handler
raise e.with_traceback(filtered_tb) from None
File “/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tensorflow/python/eager/execute.py”, line 53, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: No OpKernel was registered to support Op ‘MaxPool3D’ used by {{node model/Enc_Max_D7/MaxPool3D}} with these attrs: [T=DT_HALF, data_format=“NDHWC”, ksize=[1, 2, 2, 2, 1], strides=[1, 2, 2, 2, 1], padding=“VALID”]
Registered devices: [CPU]
Registered kernels:
device=‘XLA_CPU_JIT’; T in [DT_FLOAT, DT_BFLOAT16, DT_HALF]
device=‘CPU’; T in [DT_FLOAT]
device=‘CPU’; T in [DT_BFLOAT16]

#2

@mitsu regarding your inquiry about the “No OpKernel” error with MaxPool3D and bfloat16 compatibility, your understanding is indeed accurate. The reported issue from a year ago related to bfloat16 in MaxPooling3D has been addressed by the Keras team, and this fix is integrated into TensorFlow 2.12.0.

However, it’s noteworthy that the mixed_float16 warning implies a potential performance concern due to your machine lacking a GPU fully supporting this dtype policy. If applicable, this might impact the execution speed.

Examining the traceback, the absence of a MaxPooling3D kernel for bfloat16 on the CPU is apparent. A meticulous review of your TensorFlow version, machine specifications, and model architecture is advised.

  1. TensorFlow Version: Confirm that you are indeed utilizing TensorFlow 2.12.0.
  2. Machine Compatibility: Ensure that your machine, particularly the GPU, is compatible with mixed_float16, especially considering the absence of a GPU on your system.
  3. Model Architecture: Thoroughly inspect your model architecture, particularly the MaxPooling3D layers, for any intricacies that might contribute to the encountered issue.

Should the matter persist, it is advisable to consider seeking assistance on the TensorFlow GitHub issues page. The community there is adept at troubleshooting and might offer valuable insights specific to your configuration.

#4

Hi Badar,

Thnk you very much for your suggestions!
I condirmed again that TensorFlow 2.12.0. is implmented in phython3.10.13 in my machine.

I’m interested in the following comment: "Examining the traceback, the absence of a MaxPooling3D kernel for bfloat16 on the CPU is apparent. ". Does that mean that the tensorflow.keras DID NOT call a MaxPooling3D kernel for bfloat in my environment, although I installed tensorflow 2.12.0?

I will adress to the point 3 raised by you, and I will share my situations later.

As you advised, I will ask helps on the TensorFlow GitHub issues page. We will see.

Thanks,

Mitsu