In our scenario, there is only one process to serve the request, and the process will be closed only when an exception occurs. Each model inference algorithm is complied into a so file and loaded into this process. Multiple model inference algorithms will be loaded in this process. Each so file contain a tensorflow static library, the relationship is described as flows.
In the service, the so file of different model inference algorithms (model _ 1.so…) will often be loaded and closed through dlopen and dlclose. Because some static variables in tensorflow are created through the “new” method , as shown below. When closing the “so” with dlclose，these static variables will not be released， which will cause memory leaks.
What is the purpose of tensorflow designed like this？Will it be optimized for this problem in the future.