hub.KerasLayer object does not allow modification of existing layer parameters. This is debilitating to users who would like to use tfhub models in their own architecture with minor changes. Some specific use cases are mentioned below
Residual networks (2D and 3D) can be dilated appropriately to increase the size of feature maps and they have been known to benefit performance in tasks such as semantic segmentation and image classification such as here.
Removing temporal pooling in 3D CNNs. In many applications such as tracking and action detection, it is of interest to remove temporal pooling to keep same number of frames in input and output. This also requires modification of layer parameters.
Access to intermediate activations is also important for explanability issues. This requires accessing intermediate layers which is currently not supported.
Access to intermediate layers is also important to utilize multiple layers of a CNN directly in cases such as FPN and in some object detectors such as MSCNN.
I therefore believe that usability of tfhub models would increase tremendously if hub.KerasLayer object is allowed to be modified. Is there any plan to consider this because it seems pretty important to improve the outreach of tfhub.
Given the structure of SavedModel, I don’t think this request is address-able in general. TF Hub only can expose elements that a publisher decided to provide callable end-points for.
For the modification that you are looking for working directly with code like the ones from Model Garden ( blog post is the way to go.
Maybe my understanding is wrong but here is what I observe about the situation.
TensorFlow website states that TF2 SavedModelFormat is the preferred way to publish models on TFHub. Now the problem is that one can load a TF2 SavedModel and modify its layers. But on TFHub it is not possible because it appears that the whole model is further wrapped around a hub.KerasLayer class. This seems to be a consistency issue if I am not wrong.
The current scenario prevents researchers like me ( I know so many others with similar concerns ) from adopting pre-trained TFHub models and using them in novel settings for experimentation.
Due to points 1) and 2) on many occassions researchers are unfortunately compelled to rewrite their prototype codes in other frameworks where the model sharing ecosystem is more consistent and pre-trained models are easily available.
Especially because of point 1), I think that revamping the hub.KerasLayer class could be a good idea as it will allow people not only to use pretrained models in simplistic settings of standard finetuning and feature extraction, but would also allow people like me to use them in more research settings.
Nothing specific, if you see the comment I pointed, it was related to improve TFHub and model garden interaction and coordination when the Model Garden redesign was proposed in that RFC
I guess, as of today, if you want to make changes to the model, you’ll need it’s source code and work from the checkpoint to load the weights. Model Garden has both.
Yes exactly, my point at that time was to think about if and how we could broke this rigid barrier between inference/fine-tuning and model garden full model control.
I still think that we could have some evolutionary step in this direction in one of the next roadmaps.