I’m training a neural network with Keras, and trying to use RandomCrop layer. I’m using a dynamic sized dataset (varying resolution), but I’ve found it’s not currently the cause of this issue.
When I run model.fit(), after a short while, I receive the above mentioned error INVALID_ARGUMENT: required broadcastable shapes. I am able to get a summary of my model, so it’s not some mismatch there.
My model works fine when I remove this layer, but I need it to reduce the size of my inputs (hence using RandomCrop).
full traceback + tensorflow status
2022-03-23 13:27:28.772937: W tensorflow/core/framework/op_kernel.cc:1733] INVALID_ARGUMENT: required broadcastable shapes
Traceback (most recent call last):
File "c:\Users\samue\Desktop\rcrop\main.py", line 37, in <module>
conv_model.fit(
File "C:\Users\samue\AppData\Roaming\Python\Python310\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "C:\Users\samue\AppData\Roaming\Python\Python310\site-packages\tensorflow\python\eager\execute.py", line 54, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:
Detected at node 'mean_squared_error/SquaredDifference' defined at (most recent call last):
File "C:\Program Files\Python310\lib\threading.py", line 966, in _bootstrap
self._bootstrap_inner()
File "C:\Program Files\Python310\lib\threading.py", line 1009, in _bootstrap_inner
self.run()
File "C:\Users\samue\AppData\Roaming\Python\Python310\site-packages\keras\engine\training.py", line 1000, in run_step
outputs = model.train_step(data)
File "C:\Users\samue\AppData\Roaming\Python\Python310\site-packages\keras\engine\training.py", line 860, in train_step
loss = self.compute_loss(x, y, y_pred, sample_weight)
File "C:\Users\samue\AppData\Roaming\Python\Python310\site-packages\keras\engine\training.py", line 918, in compute_loss
return self.compiled_loss(
File "C:\Users\samue\AppData\Roaming\Python\Python310\site-packages\keras\engine\compile_utils.py", line 201, in __call__
loss_value = loss_obj(y_t, y_p, sample_weight=sw)
File "C:\Users\samue\AppData\Roaming\Python\Python310\site-packages\keras\losses.py", line 141, in __call__
losses = call_fn(y_true, y_pred)
File "C:\Users\samue\AppData\Roaming\Python\Python310\site-packages\keras\losses.py", line 245, in call
return ag_fn(y_true, y_pred, **self._fn_kwargs)
File "C:\Users\samue\AppData\Roaming\Python\Python310\site-packages\keras\losses.py", line 1329, in mean_squared_error
return backend.mean(tf.math.squared_difference(y_pred, y_true), axis=-1)
Node: 'mean_squared_error/SquaredDifference'
Detected at node 'mean_squared_error/SquaredDifference' defined at (most recent call last):
File "C:\Program Files\Python310\lib\threading.py", line 966, in _bootstrap
self._bootstrap_inner()
File "C:\Program Files\Python310\lib\threading.py", line 1009, in _bootstrap_inner
self.run()
File "C:\Users\samue\AppData\Roaming\Python\Python310\site-packages\keras\engine\training.py", line 1000, in run_step
outputs = model.train_step(data)
File "C:\Users\samue\AppData\Roaming\Python\Python310\site-packages\keras\engine\training.py", line 860, in train_step
loss = self.compute_loss(x, y, y_pred, sample_weight)
File "C:\Users\samue\AppData\Roaming\Python\Python310\site-packages\keras\engine\training.py", line 918, in compute_loss
return self.compiled_loss(
File "C:\Users\samue\AppData\Roaming\Python\Python310\site-packages\keras\engine\compile_utils.py", line 201, in __call__
loss_value = loss_obj(y_t, y_p, sample_weight=sw)
File "C:\Users\samue\AppData\Roaming\Python\Python310\site-packages\keras\losses.py", line 141, in __call__
losses = call_fn(y_true, y_pred)
File "C:\Users\samue\AppData\Roaming\Python\Python310\site-packages\keras\losses.py", line 245, in call
return ag_fn(y_true, y_pred, **self._fn_kwargs)
File "C:\Users\samue\AppData\Roaming\Python\Python310\site-packages\keras\losses.py", line 1329, in mean_squared_error
return backend.mean(tf.math.squared_difference(y_pred, y_true), axis=-1)
Node: 'mean_squared_error/SquaredDifference'
2 root error(s) found.
(0) INVALID_ARGUMENT: required broadcastable shapes
[[{{node mean_squared_error/SquaredDifference}}]]
[[div_no_nan/ReadVariableOp/_84]]
(1) INVALID_ARGUMENT: required broadcastable shapes
[[{{node mean_squared_error/SquaredDifference}}]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_1308]
How to reproduce
I’ve create a minimal reproducible example, with only two images with resolution of [10, 10] both saved as .png with rgb colorspace.
Running main.py loads these images and tries to start training (failing with an error).
When I exclude the RandomCrop layer, it works just fine.
folder structure
/main_folder
--main.py
--/data
--001.png
--002.png
main.py
import cv2, os
import keras
import tensorflow as tf
from keras import layers
strategy = tf.distribute.MirroredStrategy()
with strategy.scope():
input_layer = keras.Input(shape=(None, None, 3))
cropped = layers.RandomCrop(32, 32)(input_layer)
out = layers.Conv2D(3, (3, 3), activation='sigmoid', padding='same')(cropped)
conv_model = keras.Model(input_layer, out)
conv_model.compile(
optimizer='adam',
loss=tf.keras.losses.MeanSquaredError()
)
conv_model.summary()
path = "data"
data = [cv2.imread(os.path.join(path, f)) / 255 for f in os.listdir(os.path.join(path))]
def data_generator():
for i in range(len(data)):
yield data[i], data[i]
dataset = tf.data.Dataset.from_generator(
data_generator,
output_types=(tf.float32, tf.float32),
output_shapes=((None, None, 3), (None, None, 3))
).batch(1)
conv_model.fit(
dataset,
epochs=1,
validation_data=dataset
)