ValueError: Input 0 of layer "sequential_1" is incompatible with the layer: expected shape=(None, 6, 216, 1), found shape=(None, 216, 1)

Sankalp_20 · February 16, 2023, 1:29pm

Hey Tensorflow community

I’m new to this ML package and am stuck on a bug related to the shapes of my tensors.

I’m trying to create a CNN model for speech emotion classification but I am getting the error at the model.fit stage.

The shapes of the 2 tensors being used are:
x_testcnn shape: (9, 216, 1)
y_test shape: (9, 4)

The full error that I’m getting is:

ValueError                                Traceback (most recent call last)
<ipython-input-58-bd5fc6744319> in <module>
----> 1 cnnhistory = model.fit(x_traincnn, y_train, batch_size=16, epochs=700, validation_data=(x_testcnn, y_test))

1 frames
/usr/local/lib/python3.8/dist-packages/keras/engine/training.py in tf__train_function(iterator)
     13                 try:
     14                     do_return = True
---> 15                     retval_ = ag__.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope)
     16                 except:
     17                     do_return = False

ValueError: in user code:

    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1249, in train_function  *
        return step_function(self, iterator)
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1233, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1222, in run_step  **
        outputs = model.train_step(data)
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1023, in train_step
        y_pred = self(x, training=True)
    File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 70, in error_handler
        raise e.with_traceback(filtered_tb) from None
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/input_spec.py", line 295, in assert_input_compatibility
        raise ValueError(

    ValueError: Input 0 of layer "sequential_1" is incompatible with the layer: expected shape=(None, 6, 216, 1), found shape=(None, 216, 1)

Any help would be appreciated!

Thanks a lot

Mah_Neh · February 16, 2023, 1:45pm

can you include the net definition (you can just print the summary)

Sankalp_20 · February 16, 2023, 1:55pm

Hi @Mah_Neh

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 conv1d (Conv1D)             (None, 216, 256)          1536      
                                                                 
 activation (Activation)     (None, 216, 256)          0         
                                                                 
 conv1d_1 (Conv1D)           (None, 216, 128)          163968    
                                                                 
 activation_1 (Activation)   (None, 216, 128)          0         
                                                                 
 dropout (Dropout)           (None, 216, 128)          0         
                                                                 
 max_pooling1d (MaxPooling1D  (None, 27, 128)          0         
 )                                                               
                                                                 
 conv1d_2 (Conv1D)           (None, 27, 128)           82048     
                                                                 
 activation_2 (Activation)   (None, 27, 128)           0         
                                                                 
 conv1d_3 (Conv1D)           (None, 27, 128)           82048     
                                                                 
 activation_3 (Activation)   (None, 27, 128)           0         
                                                                 
 flatten (Flatten)           (None, 3456)              0         
                                                                 
 dense (Dense)               (None, 10)                34570     
                                                                 
 activation_4 (Activation)   (None, 10)                0         
                                                                 
=================================================================
Total params: 364,170
Trainable params: 364,170
Non-trainable params: 0

Is this what you wanted?

Mah_Neh · February 16, 2023, 2:39pm

Yes but I wonder why there isnt an input shape, can you add also how you defined the model? Or a link to whatever you use

Sankalp_20 · February 16, 2023, 3:02pm

This is how I defined the model:
model.compile(loss='categorical_crossentropy', optimizer=opt,metrics=['accuracy'])

And this is the link to my GitHub repo where the code is. You can go through the code if you’d like.

Mah_Neh · February 16, 2023, 3:38pm

From a quick look, it seems as if the labels you have in cell 57 are 10, but then you pass 5 labels, I think you have repeated labels somewhere, do you notice the same? Or maybe I didnt understand @Sankalp_20

Sankalp_20 · February 16, 2023, 4:57pm

Thanks for the reply @Mah_Neh

I see I have 10 labels in cell 57 but could you elaborate a little more on the 5 label part. I’m not quite sure what to do here.

from tensorflow import keras
#from keras import optimizers

model = Sequential()


model.add(Conv1D(256, 5,padding='same',
                 input_shape=(216,1)))
model.add(Activation('relu'))
model.add(Conv1D(128, 5,padding='same'))
model.add(Activation('relu'))
model.add(Dropout(0.1))
model.add(MaxPooling1D(pool_size=(8)))
model.add(Conv1D(128, 5,padding='same',))
model.add(Activation('relu'))
#model.add(Conv1D(128, 5,padding='same',))
#model.add(Activation('relu'))
#model.add(Conv1D(128, 5,padding='same',))
#model.add(Activation('relu'))
#model.add(Dropout(0.2))
model.add(Conv1D(128, 5,padding='same',))
model.add(Activation('relu'))
model.add(Flatten())
model.add(Dense(10))
model.add(Activation('softmax'))
#opt = keras.optimizers.RMSprop(lr=0.00001, decay=1e-6)
opt = keras.optimizers.legacy.Adam(lr=0.00001, decay=1e-6)

I suppose you are referring to the "5"s I have entered as the argument for Conv1D?

If so, I tried changing that to 10 but I am still getting the error.

Would be great if you could suggest a change

Mah_Neh · February 16, 2023, 6:54pm

I will think about it, I tried to run it but I do not have the dataset, I am not sure if there is a secure way to share this. I am also learning and trying to solve some problems, understanding the dimensions is essential. What I would to is try to remove stuff, to scope the error.

For example, does the same happen is the evaluation data is removed from the fit fn (to know if the problem is already present in the training data.)

I thought that maybe the labels should actually be five rather than the input 10, for some reason.

Sankalp_20 · February 18, 2023, 9:38am

Here is a link to the audio files I’m using to train the model. (I’m using the RAVDESS dataset)
https://drive.google.com/drive/folders/1TFDdJM14ze4qnRFh9nDLIpnodOYjLBrU?usp=sharing

tagoma · February 18, 2023, 11:39am

Can you show the full code, please or at least the line with the .fit() command as we may be missing some bits here?

If you are a starter with Tensorflow/ML/NN, and if I may advise, you should:

It is critical you understand tensors shape, e.g. (batch, timesteps, input_features) 텐서 소개 | TensorFlow Core as hinted by @Mah_Neh
You also need to understand the layers you use: data that get in, operations within that layer and output
Add one layer another. check if this works. If it does, add the next layer, etc…
Use the same import(s) as in keras/Tensorflow documentation e.g. from keras import layers, and when using a layer layers.[that layer]
Use variables as much as possible instead of hard coding tensors shapes

I mean otherwise you will get stuck all the time whenever you create a new model.

Thank you.

Mah_Neh · February 18, 2023, 11:45am

Great then I will reproduce it from scratch in colab

Sankalp_20 · February 18, 2023, 12:37pm

Thanks @tagoma,

I’ll keep these things in mind as I go forward on my path to learn Tensorflow in more detail.

To answer the first part of your question, this is the model and .fit() command I used:

model = Sequential()


model.add(Conv1D(256, 5,padding='same',
                 input_shape=(216,1)))
model.add(Activation('relu'))
model.add(Conv1D(128, 5,padding='same'))
model.add(Activation('relu'))
model.add(Dropout(0.1))
model.add(MaxPooling1D(pool_size=(8)))
model.add(Conv1D(128, 5,padding='same',))
model.add(Activation('relu'))
#model.add(Conv1D(128, 5,padding='same',))
#model.add(Activation('relu'))
#model.add(Conv1D(128, 5,padding='same',))
#model.add(Activation('relu'))
#model.add(Dropout(0.2))
model.add(Conv1D(128, 5,padding='same',))
model.add(Activation('relu'))
model.add(Flatten())
model.add(Dense(10))
model.add(Activation('softmax'))
#opt = keras.optimizers.RMSprop(lr=0.00001, decay=1e-6)
opt = keras.optimizers.legacy.Adam(lr=0.00001, decay=1e-6)

cnnhistory = model.fit(x_traincnn, y_train, batch_size=16, epochs=700, validation_data=(x_testcnn, y_test))

Let me know if there is some other code snippet that you require and I’ll add it

Thanks in advance

tagoma · February 18, 2023, 6:15pm

I’m a little confused to be honest. I went through your Colab now (using Actor_01/03-01-01-01-01-01-01.wav as you did) and the tensors shapes and errors messages are often different from what you wrote down here, and also from what was in your Colab as I opened it the first time and what I get when I ran the Colab cells. The shape of your training and test datasets also don’t look quite consistent with one another. Can you please check?

Mah_Neh · February 18, 2023, 6:46pm

Not sure how this works really because I cant see a single file. NVM I found the files elsewhere

Mah_Neh · February 18, 2023, 10:08pm

Took me a while to figure this out but the main error is that you have 5 labels and your output layer has 10 units.

So replace model.add(Dense(10)) with model.add(Dense(5))

I actually re wrote the whole notebook apart from parts I didnt get and think your idea is fantastic.

Comments

Basically you are representing sound as an image, and trying to find patterns on it. I am a newcomer as well.

Now you pass 32,216 but as far as I can see this consumes all your samples,

What I did is to reshape each sample into an image of 27x8 (216) just to see what happens.

Here is the code.

Sankalp_20 · February 19, 2023, 6:20am

Thanks a lot for reproducing the issue @Mah_Neh

I ran the notebook you provided but I am facing the same error with the shapes again. I’m not sure why’s that happening.

The shapes I’m using:
X_test: (6, 27, 8)
Y_test: (6, 4)

This shouldn’t really throw an error considering I reshaped the sample like you described.

The full error I’m getting:

ValueError: in user code:

    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1820, in test_function  *
        return step_function(self, iterator)
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1804, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1792, in run_step  **
        outputs = model.test_step(data)
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1758, in test_step
        self.compute_loss(x, y, y_pred, sample_weight)
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/training.py", line 1082, in compute_loss
        return self.compiled_loss(
    File "/usr/local/lib/python3.8/dist-packages/keras/engine/compile_utils.py", line 265, in __call__
        loss_value = loss_obj(y_t, y_p, sample_weight=sw)
    File "/usr/local/lib/python3.8/dist-packages/keras/losses.py", line 152, in __call__
        losses = call_fn(y_true, y_pred)
    File "/usr/local/lib/python3.8/dist-packages/keras/losses.py", line 284, in call  **
        return ag_fn(y_true, y_pred, **self._fn_kwargs)
    File "/usr/local/lib/python3.8/dist-packages/keras/losses.py", line 2004, in categorical_crossentropy
        return backend.categorical_crossentropy(
    File "/usr/local/lib/python3.8/dist-packages/keras/backend.py", line 5532, in categorical_crossentropy
        target.shape.assert_is_compatible_with(output.shape)

    ValueError: Shapes (None, 4) and (None, 5) are incompatible

Mah_Neh · February 19, 2023, 10:30am

Can you try again Google Colab.

It would be nice if you can explain the main idea of your Notebook, so that we can also understand your goal. I think there may be something missing yet, like, is representing just a series of values (intensities of sound) in time, as a matrix instead, i.e just splitting the array, enough for sentient classification?

So we would ideally understand well how that process is thought out. Currently I think it is not enough. You will notice, for example, that the validation loss barely changes, which is also an indication of something missing…

@Sankalp_20

Sankalp_20 · February 20, 2023, 10:17am

@Mah_Neh Thanks a lot for all the efforts you put in! Your notebook worked for me! Although the data is underfitting right now but I will fix that

To answer your question on what I am trying to achieve from this notebook: I’m trying to create an audio classifier that I’m training on the RAVDESS dataset. I then need to use that model to classify a bunch of audio files that I’ll feed it

Mah_Neh · February 20, 2023, 10:23am

Great, could be nice to see your results in a while, looks like an interesting project.

Good luck!