Hw does fit function handles two loss functions

Research & Models
,
#1

Hi,
I am training a model, which has two stages, the stage 1 output is fed to stage 2.out_1 and out_2 are the outputs of stage1 and stage 2 respectively.
i had complied my model using 2 different loss functions. and a single optimizer ,i want to know how does the fit() function handles this? I mean after the calculating the loss how the backpropagation and optimizer step is taken?

Thanks in Advance.

#3

Hi @PALLALA_ARUNKUMAR, If the model has multiple outputs then you can use different losses on each output by passing a dictionary or a list of losses. For example, If your output is named, you can use a dictionary mapping the names to the corresponding losses:

x = Input((10,))
out1 = Dense(10, activation='softmax', name='Layer1')(x)
out2 = Dense(10, name='layer2')(x)
model = Model(x, [out1, out2])
model.compile(loss={'Layer1': 'sparse_categorical_crossentropy', 'layer': 'mae'},
              optimizer='adam')

If your output was not named then you can pass the losses as list like

model.compile(loss=['sparse_categorical_crossentropy', 'mae'], optimizer='adam')

The losses mentioned in the list will be applied to model output in the corresponding order. Thank You.

1 Like
#4

Hi @Kiran_Sai_Ramineni ,
Thanks for response, My question is how two loss function are handled by fit function.(My assumption is loss 1 will effect the stage 1 and loss 2 effects the complete model , because stage 1 output is fed to stage 2) what happens internally after the two loss function are calculated. i mean they are brackpropagated individually or the losses are combined and backpropagated? just wanted to know how fit function works internally.
if the losses the backpropagated individually then when to take the optimizer step?

1 Like