Hello everyone!
I am in the process of experimenting with different Tensorflow Serving batching configurations and I am looking for a way to speed up my experiments. Currently, I have a model running in a pod within a Kubernetes cluster, and each time I want to change the values set in the batching.config file I have to edit the file in cloud storage and restart the pods so that they pull the updated version.
After reading some of the docs I am under the impression that this can be done programmatically (on the fly, without having to restart the pod) with python, however, I cannot figure out how to do that.
I tried to read through the source code but don’t really understand how to translate what I see in the source code into python and more so how to actually achieve what I am trying to do.
Any help would be greatly appreciated!