Get_weights, set_weights

General Discussion
, , ,
#1

I everybody,
I want to average the weights of three identical model trained with different training sets and that create a new model with averagerd weights coming from these three models.
My models have the same structure.
I’ve tried with this code but seems the weights of the new model that i want create remanins equal to the last model that i fit.
What i should do in order to make it works?
Thanks

def load_all_models():
  all_models = list()
  for i in range(3):
    filename = 'Conv_autoencoder_'+str(n)+'_layer_.h5'
    model = load_model(filename)
    all_models.append(model)
    print('Loaded model %s' % filename)
  return all_models
	
# create a model from the weights of multiple models
def model_weight_ensemble(members, weights):
    n_layers = len(members[0].get_weights())
    avg_model_weights = list()
    for layer in range(n_layers):
      layer_weights = np.array([model.get_weights()[layer] for model in members])
      # weighted average of weights for this layer
      avg_layer_weights = np.average(layer_weights, axis=0, weights=weights)
      # store average layer weights
      avg_model_weights.append(avg_layer_weights)
	
    # create a new model with the same structure
    model = clone_model(members[1])
    # set the weights in the new
    model.set_weights(avg_model_weights)
    model.compile(optimizer='SGD', loss='mean_squared_error')
    return model

members = load_all_models()
print('Loaded %d models' % len(members))
n_models = len(members)
weights = [1 for i in range(1, n_models+1)]
autoencoder_global = model_weight_ensemble(members,weights)

print(autoencoder_global.get_weights()[12])
print('------------------------------------------------------------------')
#2

Hi @Luca_Zanetti, To average the weights of the 3 models having the same architecture but trained on different data, first you have to get the model weights using get_weights( ) method and then you can average the weights, For example, i have extracted 2 model weights and assigned to a variable

Handwritten_model=model.get_weights()
fashionMnist_model=model1.get_weights()

averaged_weights = []
for weight1, weight2 in zip(Handwritten_model, fashionMnist_model):
    averaged_weight = (weight1 + weight2) / 2
    averaged_weights.append(averaged_weight)

To assign averaged to the new model you can use set_weights( ) method

avg_weight_model.set_weights(averaged_weights)

In the similar way you can get the avg weights for the 3 models. Please refer to this gist for working code example. Thank You.