Multi-channel CNN - ValueError: Output tensors of a Functional model must be the output of a TensorFlow `Layer`

Keras
#1

I am trying to construct a multi-channel convolutional neural network. I have two tensors (i.e., training_x1 and training_x2), each including float values with (476, 47, 3, 1) dimension. I build the multi-channel NN as follows (a minimal example):

import numpy as np    
from tensorflow import keras
from keras.models import Model
from keras.layers import Dense, Conv2D, MaxPooling2D, Flatten, Dropout, concatenate, AveragePooling2D, Input


training_x1 = np.zeros((476, 47, 3, 1))  # just as an example
training_x2 = np.zeros((476, 47, 3, 1))  # just as an example

inputs1 = Input(shape=(training_x1.shape[0], ))
conv1 = Conv2D(filters=32, kernel_size=3, activation='relu')(training_x1)
drop1 = Dropout(0.5)(conv1)
pool1 = MaxPooling2D(pool_size=2)(drop1)
flat1 = Flatten()(pool1)

# channel 2
inputs2 = Input(shape=(training_x2.shape[0], ))
conv2 = Conv2D(filters=32, kernel_size=3, activation='relu')(training_x2)
drop2 = Dropout(0.5)(conv2)
pool2 = MaxPooling2D(pool_size=2)(drop2)
flat2 = Flatten()(pool2)

# merge
merged = concatenate([flat1, flat2])

# interpretation
dense1 = Dense(10, activation='relu')(merged)
outputs = Dense(1, activation='linear')(dense1)
model = Model(inputs=[inputs1, inputs2], outputs=outputs)

However, my model fails with error ValueError: Output tensors of a Functional model must be the output of a TensorFlow Layer (thus holding past layer metadata). What mistake am I making?

#3

Hi @Amir, You are getting this error due to the outputs of the model is not a layer instance. While creating the model you have to provide the input shape only, but for convolution layer you are passing the values due to which the outputs has values. The below code will not generate the error because i am not passing any values to the model while defining model

inputs1 = Input(shape=(28,28,1))
conv1 = Conv2D(filters=32, kernel_size=3, activation='relu')(inputs1)
pool1 = MaxPooling2D(pool_size=2)(conv1)
drop1 = Dropout(0.5)(drop1)
flat1 = Flatten()(pool1)

# channel 2
inputs2 = Input(shape=(28,28,1))
conv2 = Conv2D(filters=32, kernel_size=3, activation='relu')(inputs2)
pool2 = MaxPooling2D(pool_size=2)(conv2)
drop2 = Dropout(0.5)(drop2)
flat2 = Flatten()(pool2)

# merge
merged = Concatenate()([flat1, flat2])

# interpretation
dense1 = Dense(10, activation='relu')(merged)
outputs = Dense(1, activation='linear')(dense1)

model = Model(inputs=[inputs1, inputs2], outputs=outputs)

The values to model are passed during the forward pass.For working code example please refer to this gist.Thank You.

1 Like
#4

Thanks for your reply, @Kiran_Sai_Ramineni! I copied & pasted your MWE but it unfortunately does not work for me! drop1 and drop2 seem to be unresolved references. Here is the error: drop1 = Dropout(0.5)(drop1) NameError: name ‘drop1’ is not defined

I made the following modification and it seems to be working (i.e., creates the model). However, I cannot fit the model! Here is my minimal code:

from tensorflow import keras
from keras.models import Model
from keras.layers import Dense, Conv2D, MaxPooling2D, Flatten, Dropout, concatenate, Input

training_x1 = np.zeros((476, 47, 3)) # channel 1
training_x2 = np.zeros((476, 47, 3)) # channel 2
validation_x1 = np.zeros((100, 47, 3)) # channel 1
validation_x2 = np.zeros((100, 47, 3)) # channel 2
training_y = np.zeros((476, 1))
validation_y = np.zeros((100, 1))

# channel 1
inputs1 = Input(shape=(training_x1.shape[0], training_x1.shape[1], training_x1.shape[2]))
conv1 = Conv2D(filters=32, kernel_size=3, activation='relu')(inputs1)
pool1 = MaxPooling2D(pool_size=2, padding='valid')(conv1)
drop1 = Dropout(0.5)(pool1)
flat1 = Flatten()(drop1)

# channel 2
inputs2 = Input(shape=(training_x2.shape[0], training_x2.shape[1], training_x2.shape[2]))
conv2 = Conv2D(filters=32, kernel_size=3, activation='relu')(inputs2)
drop2 = Dropout(0.5)(conv2)
pool2 = MaxPooling2D(pool_size=2, padding='valid')(drop2)
flat2 = Flatten()(pool2)

# merge
merged = concatenate([flat1, flat2])

# interpretation
dense1 = Dense(10, activation='relu')(merged)
outputs = Dense(1, activation='linear')(dense1)
model = Model(inputs=[inputs1, inputs2], outputs=outputs)

# compile the model
model.compile(loss='mse',
              optimizer='rmsprop',
              metrics='accuracy')

# fit the model
history = model.fit([training_x1, training_x2], training_y,
                    validation_data=([validation_x1, validation_x2], validation_y),
                    epochs=100)

I face this error: ValueError: Input 0 of layer “model” is incompatible with the layer: expected shape=(None, 476, 47, 3), found shape=(None, 47, 3)

I am trying different data input settings, but I am unable to find the appropriate one. Please let me know how I should reshape/pass data to make this algorithm works. I appreciate your help, @Kiran_Sai_Ramineni.

#6

I have edited the code this will work fine

inputs1 = Input(shape=(28,28,1))
conv1 = Conv2D(filters=32, kernel_size=3, activation='relu')(inputs1)
pool1 = MaxPooling2D(pool_size=2)(conv1)
drop1 = Dropout(0.5)(pool1)
flat1 = Flatten()(drop1)

# channel 2

inputs2 = Input(shape=(28,28,1))
conv2 = Conv2D(filters=32, kernel_size=3, activation='relu')(inputs2)
pool2 = MaxPooling2D(pool_size=2)(conv2)
drop2 = Dropout(0.5)(pool2)
flat2 = Flatten()(drop2)

# merge
merged = concatenate(([flat1, flat2]))

# interpretation
dense1 = Dense(10, activation='relu')(merged)
outputs = Dense(1, activation='linear')(dense1)
model = Model(inputs=[inputs1, inputs2], outputs=outputs)
1 Like
#7

Hi @Amir ,

You are getting this error because you are passing an incorrect shape to the model. In this gist i have executed the above code with error. Thank You.

1 Like
#8

Thanks, @Kiran_Sai_Ramineni. I think there is a misunderstanding.

In the above example ((476, 47, 3) and (100, 47, 3) for training and validation, respectively), my data includes two channels, each channel with tensors of 47 by 3. For each channel, I have 476 tensors for the training set and 100 tensors for the validation set. And there are 476 and 100 output values for the training and validation sets, respectively. Your code does not seem to respect these dimensions though. For example, you are defining the training response variable as training_y = np.zeros((1,)) while it should have 476 data points. Thanks for your help.

#9

Hi @Amir, Then you can use training_y = np.zeros((1,476)). Thank You.

1 Like
#10

It is now working. Thanks, @Kiran_Sai_Ramineni. One last question. What if the number of data points are not identical for training and validation sets (unequal values causes problem)? It seems you are assuming they are both identical (both have 476 tensors).

#11

Hi @Amir, there is a convolution layer in the model so the training and validation data should be of the same shape and fixed length. The number of samples (batch size) can vary between training and validation data. Thank You.