Model not trainable when using keras.layers.Resizing

Keras
, ,
#1

Hi,
I recently found the keras.layers.Resizing method and tried to use it as an alternative to the Upsampling1D method. Unfortunately it does not seem to work… When I train the model, the loss stays constantly at 100%. I have checked the outputs of the layers: the Resizing-layer is doing the exact same as the Upsampling1D-layer, but still I cannot train the model. Does anybody know what is causing this problem? I don’t really see why it shouldn’t work.

I use tensorflow 2.9.1.

Thanks and best regards!

#4

Upsampling layers will repeat values in the smaller layer to reach the target size whereas Resizing layers allows the user to specify how the pixels in the larger image are interpolated. Thank you.

#5

Resizing is differentiable, so that’s not the problem:

import tensorflow as tf

x = tf.Variable(tf.zeros([3, 10,12,7]), dtype=tf.float32)

resizer = tf.keras.layers.Resizing(23, 17)
with tf.GradientTape() as tape:
  y = resizer(x)
  result = tf.reduce_sum(y)

tape.gradient(result, x)
#6

Note that int ops are not differentiable, are you casting your image to int at some point?

#8

Hi thanks for the answer! The inputs were of type float, so this should not have been a problem. However, do you know if keras.layers.Resizing has problems handling tf.float64? If so, this actually could have caused the problem…
But I also did not receive any warnings or errors.

#9

Just try it in the Colab. float64 works fine. There’s some other error in there.

#10

I think I figured out the problem. I used the Resizing-operation within a model-API consisting of Conv1D-layers… But, the output of a Conv1D-Layer has no second dimension. Therefore, keras seemed to have mixed the batch-dimension into the output which obviously resulted in a diverging training… Am I right about the assumption that keras.layers.Resizing does not work if the input to the operation is the output of a Conv1D layer?

#12

The answer is always just try it and see what happens:

resizer = tf.keras.layers.Resizing(23, 17)

x = tf.random.normal(shape = [3,5,7])
resizer(x).shape

TensorShape([23, 17, 7])

It looks like it interprets (batch, sequence, channels) as (height, width, channels) !

I’ve never had to resize a sequence… but if you want it to work with a resize layer you need to do a:

!pip install einops

resizer = tf.keras.layers.Resizing(1, 17)

x = tf.random.normal(shape = [3,5,7])
x = einops.rearrange(x, 'b s c -> b 1 s c')
x = resizer(x)
x = einops.rearrange(x, 'b 1 s c -> b s c')
1 Like
#14

Thank you very much for your help and explanation!