Efficientdet vs ResNet: Resnet outperforms Efficientdet-B3

So I tried to swap out ResNet for Efficientdet-B3 in the Eager Few Shot OD Training TF2 tutorial.
Now, based on all the positive feedback Efficientdet got I am very surprised that ResNet outperformed Efficientdet on this tutorial. In total Efficientdet got trained on 1700 batches in the tutorial, while I ran ResNet through the standard batch size of 100.

Efficientdet-B3 for the last 1000 batches I run of a total of 1700:

batch 950 of 1000, loss=0.21693243
batch 955 of 1000, loss=0.18070191
batch 960 of 1000, loss=0.1715184
batch 965 of 1000, loss=0.23656633
batch 970 of 1000, loss=0.16813375
batch 975 of 1000, loss=0.23602965
batch 980 of 1000, loss=0.14852181
batch 985 of 1000, loss=0.18400437
batch 990 of 1000, loss=0.22741726
batch 995 of 1000, loss=0.20477971
Done fine-tuning!

ResNet for 100 batches:

batch 0 of 100, loss=1.1079819
batch 10 of 100, loss=0.07644452
batch 20 of 100, loss=0.08746071
batch 30 of 100, loss=0.019333005
batch 40 of 100, loss=0.0071129226
batch 50 of 100, loss=0.00465827
batch 60 of 100, loss=0.0041421074
batch 70 of 100, loss=0.0026128457
batch 80 of 100, loss=0.0023376464
batch 90 of 100, loss=0.002139934
Done fine-tuning!

Why does Efficientdet need so much more training time than ResNet, is it due to that the number of parameters is only about 12 mill for Efficientdet-B3 (the one I tested) and about 25 mill for the ResNet50? Or are their other reasons?
The end result (the .gif at the end of the tutorial) also shows a huge different in accuracy, where ResNet performs much better.

Thanks for any input!

hello, when you mean “outperform” are you referring to the evaluation/test metrics? you should always monitor the gap between the training and validation losses, they should behave similar and then evaluate to see the final performance on a test set. Otherwise ResNet might be just overfitting the data and that’s why you get a extremely small loss, are these losses you posted training losses?

1 Like

Thank you for your reply.
This is based on the loss during training and the final .gif that is rendered after predictions has been performed on the test images.

You would need to compare the evaluation metrics on a larger set of test/evaluation examples to conclude which model is better, just by the training losses I can infer ResNet is overfitting, but the final conclusion always comes from evaluating the model on a large set of test cases


Thank you again!

The result for EfficientDet after 200 batches is like this when running through the test:

Efficientdet test result (gif)

and for ResNet with 200 batches:

ResNet test result (gif)

So it does not seem to be overfitting (?) as ResNet has high accuracy on the test data.
I am just trying to understand why there is such huge different on the result for the models :face_with_monocle:


What’s your Efficientdet train script? Provide a colab link if it’s possible.
Batch size=4 is too small for efficientdet