I’m trying to deploy my image classification model on streamlit but I encounter this warning message when I trying to make inferences using the app.
The first image is just showing information about your platform. You can pretty much safely ignore them.
About the second image…Your inference function is a tf.function which constructs, optimizes, and compiles a static graph at runtime (by “tracing” the annotated function, outside the scope of this answer). The warning is telling you that each call to your predict function has to reconstruct, optimize, and compile a new graph. That ends up being really expensive, and you miss out on the performance benefits of tf.function.
To break it down, here’s what it’s telling you could be happening:
- creating @tf.function repeatedly in a loop
while True:
my_predict_function = lambda x: x * x
my_tf_predict_function = tf.function(func=my_predict_function) # uh oh, this gets called every loop iteration
my_tf_predict_function(tf.random.uniform((2, 2)))
If you’re not defining / redefining your model or predict function in a loop, this is unlikely to be the case.
- passing tensors with different shapes
my_predict_function = lambda x: x * x
my_tf_predict_function = tf.function(func=my_predict_function)
my_tf_predict_function(tf.random.uniform((2, 2)))
my_tf_predict_function(tf.random.uniform((3, 2))) # different shapes, our graph was previously optimized on a shape of (2, 2)
If you’re passing images of different dimensions, this is likely the case.
- passing Python objects instead of tensors
my_predict_function = lambda x, flag: x * x if flag else x
my_tf_predict_function = tf.function(func=my_predict_function)
input = tf.random.uniform((2, 2))
my_tf_predict_function(input, True) # the graph compiled is x * x
my_tf_predict_function(input, False) # we have to recompile because it takes a different path based on flag!
Notice how the graph changes based on flag which is a Python object and not a tensor. So even though the shapes are the same, the graph is different. If you control your model with some hyperparameters which are Python scalars or other Python objects, then it will need to recompile the graph every time you change a parameter.
1 Like
Wow thanks, you explained it well.