Identity Mappings in ResNet/ ResnetV2

General Discussion
, ,
#1

Hi everybody,

I’d like to discuss the correctness of the ResNet and ResNetV2 implementations, recently
I noticed a few flaws in the Keras applications implementation, starting with this. Now I want to discuss the identity mappings, it seems like they make a huge difference, according to the ResnetV2 paper.

The way ResNetV1 is implemented, the skip connections are not Identity at all, there are BN and activations in the way, which defeats the idea in which ResNets were designed.

ResNetV2 tries to correct those issues, but I still see some MaxPooling layers in the skip connections.

I have compared my results training a binary classifier with ResNetV1, ResNetV2 and a modified ResNetV1 with all identity mappings. The modified ResNetV1 is providing me the best results. My dataset contains images with a high number of channels, which impedes the usage of Imagenet weights.

See results here (scroll image to the right to see ResnetV2): ResNet_test/networks_precision_recall.png at main · ririya/ResNet_test · GitHub

#2

Sorry all, my current forum member level does not allow me to embed pictures and only allows me to share two urls, that’s why I had to create a single image.

#3

I think this is a good observation. Even though your points are great but I would still want to try those modifications on a dataset of natural images just to confirm the hunches. Why don’t you discuss this in the repository:

/cc: @fchollet