How to properly add a residual connection to a Keras Model?

General Discussion
, ,
#1

I would like to get the recipe for a model that contains residual connections and I want to change dimensions of some layers (thus, I barely can use clone_model() method) to re-assemble this model. How to do it?

I have tried to read all the residual connections:

non_seq_layers = {layer.name: l[0].name for layer in model.layers if isinstance(l:=layer._inbound_nodes[0].inbound_layers, list) and l}

l[1] is always just previous layer, l[0] shows residual connection.

Then I tried to assemble model (without any shape modifications):

def recompose(model, non_seq_layers):
    new_model = Model()
    previous_layer = model.input
    for layer in model.layers:
        print(layer.name)
        if layer.name in non_seq_layers:
            layer = copy_layer(layer)
            previous_layer = layer([previous_layer, model.get_layer(non_seq_layers[layer.name]).output])
        else:
            layer = copy_layer(layer)
            previous_layer = layer(previous_layer)
        
        
    new_model = Model(model.input, previous_layer)
    return new_model

Everything works fine untill I the conv3_block1_add layer (Resnet50) where I got error:

ValueError: Inputs have incompatible shapes. Received shapes (14, 14, 512) and (28, 28, 512)

Is it a proper way to manage such task? How to repair it? Perhaps sth with encoding residual connections (non seq layers ) may be wrong.

#3

Hi @Igor_Ratajczyk.
I guess it would be easier to help if you shared your whole snippet code.
The error message you get is quite clear. There is a shape mismatch in the model you created using recompose function. Did you explore the architecture of that new model, e.g. using the summary() method?

#4

@tagoma ,surely, here is entire code:

from keras.applications.resnet import ResNet50
model = ResNet50(weights='imagenet')
non_seq_layers = {layer.name: l[0].name for layer in model.layers if isinstance(l:=layer._inbound_nodes[0].inbound_layers, list) and l}

def copy_layer(layer):
    if isinstance(layer, Conv2D):
        return Conv2D(layer.filters, layer.kernel_size, layer.strides, layer.padding, layer.data_format, layer.dilation_rate, layer.groups, layer.activation, layer.use_bias, layer.kernel_initializer, layer.bias_initializer, layer.kernel_regularizer, layer.bias_regularizer, layer.activity_regularizer, layer.kernel_constraint, layer.bias_constraint, name=layer.name)
    else:
        return layer.__class__.from_config(layer.get_config())

def recompose(model, non_seq_layers):
    new_model = Model()
    previous_layer = model.input
    for layer in model.layers:
        try:
            # print(layer.name)
            if layer.name in non_seq_layers:
                # layer = copy_layer(layer)
                previous_layer = layer([previous_layer, model.get_layer(non_seq_layers[layer.name]).output])
            else:
                layer = copy_layer(layer)
                previous_layer = layer(previous_layer)
        except ValueError as e:
            print(f'Error in {layer.name} : {e}')

        
        
    new_model = Model(model.input, previous_layer)
    return new_model

recomposed = recompose(model, non_seq_layers)

I am at the begining of implementing the more advanced algorithm, but as long as plain decompose-recompose funtion does not work i cannot do any more advanced weights manipulation

#5

Can you please also show your import commands, e.g. is Model() a Tensorflow/Keras model object? Also beware possible issues using layer or other such variable names.
Thank you.

#6

@tagoma

from keras.applications.resnet import ResNet50

from keras.models import Model

from keras.layers import Dense, GlobalAveragePooling2D, Add, Conv2D