ResNet-RS rewritten in Tensorlfow/Keras

sebastian-sz · November 11, 2021, 12:07pm

Hello Community!

I’m sharing a personal project of mine, which was to rewrite ResNet-RS models from TPUEstimator to Tensorflow/Keras.

Features:
Automatic weights download.
Transfer learning possible.
pip install directly from GitHub.
keras.applications like usage.
Use like any other Tensorflow/Keras model!

Updating the ResNet-* weights

opened 01:41PM - 12 Dec 21 UTC

closed 06:02AM - 11 Apr 22 UTC

sayakpaul

type:feature Contributions welcome

If you open a GitHub issue, here is our policy: It must be a bug, a feature r…equest, or a significant problem with the documentation (for small docs fixes please send a PR instead). The form below must be filled out. **Here's why we have that policy:**. Keras developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow. **System information**. TensorFlow version (you are using): 2.7.0 Are you willing to contribute it (Yes/No) : Currently no **Describe the feature and the current behavior/state**. ResNets are arguably one of the most influential architectures in deep learning. Today, they are used in different capacities. For example, sometimes they act as strong baselines, sometimes they are used as backbones. Since their inception, their performance on ImageNet-1k, in particular, has improved quite a lot. I think it's time the ResNets under `tf.keras.applications` were updated to facilitate these changes. **Will this change the current api? How?** ResNet-RS (https://arxiv.org/abs/2103.07579) introduces slight architectural changes to the vanilla ResNet architecture (https://arxiv.org/abs/1512.03385). So, yes, there will be changes to the current implementation of ResNets (among other things) we have under `tf.keras.applications`. We could call it `tf.keras.applications.ResNet50RS`, for example. Following summarizes the performance benefits that ResNet-RS introduces to the final ImageNet-1k performance (measured on the `val` set): ![image](https://user-images.githubusercontent.com/22957388/145714574-b057a95e-4d6c-48ba-b7ea-67e953e83f29.png) <sub><a href=https://github.com/tensorflow/tpu/tree/master/models/official/resnet/resnet_rs#imagenet-checkpoints>Source</a></sub> **Who will benefit from this feature?** Keras users that use ResNets from `tf.keras.applications` for building downstream applications. **[Contributing](https://github.com/keras-team/keras/blob/master/CONTRIBUTING.md)** - Do you want to contribute a PR? (yes/no): Currently no - If yes, please read [this page](https://github.com/keras-team/keras/blob/master/CONTRIBUTING.md) for instructions - Briefly describe your candidate solution(if contributing):

innat · January 8, 2022, 4:21pm

@sebastian-sz cc. @Sayak_Paul
Amazing job. There’s an option to contribute to the core API of these model families (as mentioned by brother sayak). If you consider that would be great, like you did for EfficientNet-V2

sebastian-sz · January 9, 2022, 3:35pm

@Sayak_Paul @innat
Thank you for thinking of me.

Do you have information if it would be possible to simply, directly use (linted + slightly modified) my implementation’s model file?

If yes I could probably submit a PR by the end of this month.

Sayak_Paul · January 10, 2022, 1:44am

Thanks for considering!

I quickly took a look at your implementation and it seems very structured. I think barring a few formatting and Keras-specific nits, you’d be well on your way.

sebastian-sz · January 10, 2022, 7:08am

Thanks! I will let you know, how things go.

Bhack · January 10, 2022, 9:31am

Please check

sebastian-sz · January 10, 2022, 12:36pm

@Bhack
Do you mean that the PR, regarding this model family, should be opened in keras-cv instead of keras? Or that ResNetRS models should be retrained from scratch with components created in keras-cv?

The repository I created is simply a port of model architecture + weights. I did not include training code (but also none of the models under keras.applications do).
As for training components like dropblock / drop connect: they are present in the original repository but upon closer inspection their parameters are always either 0 or null, so I decided not to include them.

Bhack · January 10, 2022, 3:06pm

Yes there we will/would have all the reusable components to build and train these models.

sebastian-sz · January 11, 2022, 9:03am

In this case should we restrain from adding these models to keras.applications ?

Bhack · January 11, 2022, 7:49pm

I think that adding a model in keras.applications, model garden or TF HUB has different scope requirements.