Error when trying to run inference using TFlite

General Discussion
, ,
#1

I’ve custom trained ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8_3 with my data and exported the model and I’m able to do inferencing . im using tensorflow 2.6 and now I want to convert the model to tflite to run it on mobile .I’ve used the code below to convert the model to tflite and the code run and a tflite model is saved successfully .

converter=tf.lite.TFLiteConverter.from_saved_model(PATH_TO_SAVED_MODE1L)
    converter.optimizations=[tf.lite.Optimize.DEFAULT]
    converter.target_spec.supported_ops=[tf.lite.OpsSet.TFLITE_BUILTINS,tf.lite.OpsSet.SELECT_TF_OPS]
    tflite_model=converter.convert()
    with open('C:/Users/CX/Desktop/model.tflite', 'wb') as f:
      f.write(tflite_model)

Now im trying to test with some with images using the code below :

interpreter = tf.lite.Interpreter(model_path="C:/Users/CX/OneDrive/Desktop/model.tflite")
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
path=r"C:\t3"
for file in pathlib.Path(path).iterdir():

    img = cv2.imread(r"{}".format(file.resolve()))
     
    img1 = img.reshape(1,img.shape[0],img.shape[1],3) 

    interpreter.set_tensor(input_details[0]['shape_signature'],img1)

    interpreter.invoke()

Im getting the error :

TypeError: SetTensor(): incompatible function arguments. The following argument types are supported:
    1. (self: tensorflow.lite.python.interpreter_wrapper._pywrap_tensorflow_interpreter_wrapper.InterpreterWrapper, arg0: int, arg1: handle) -> object

Invoked with: <tensorflow.lite.python.interpreter_wrapper._pywrap_tensorflow_interpreter_wrapper.InterpreterWrapper object at 0x000001E4D88E38F0>, array([ 1, -1, -1,  3])

here is the details from the interpreter.get_input_details() fucntion :

[{'name': 'serving_default_input_tensor:0', 'index': 0, 'shape': array([1, 1, 1, 3]), 'shape_signature': array([ 1, -1, -1,  3]), 'dtype': <class 'numpy.uint8'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}]


and interpreter.get_output_details()
[{'name': 'StatefulPartitionedCall:6', 'index': 462, 'shape': array([    1, 12804,     4]), 'shape_signature': array([    1, 12804,     4]), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}, {'name': 'StatefulPartitionedCall:0', 'index': 1594, 'shape': array([1, 1]), 'shape_signature': array([ 1, -1]), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}, {'name': 'StatefulPartitionedCall:5', 'index': 1559, 'shape': array([1]), 'shape_signature': array([1]), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}, {'name': 'StatefulPartitionedCall:7', 'index': 482, 'shape': array([    1, 12804,    56]), 'shape_signature': array([    1, 12804,    56]), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}, {'name': 'StatefulPartitionedCall:1', 'index': 1647, 'shape': array([1, 1, 1]), 'shape_signature': array([ 1, -1, -1]), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}, {'name': 'StatefulPartitionedCall:2', 'index': 1629, 'shape': array([1, 1]), 'shape_signature': array([ 1, -1]), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}, {'name': 'StatefulPartitionedCall:4', 'index': 1576, 'shape': array([1, 1]), 'shape_signature': array([ 1, -1]), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}, {'name': 'StatefulPartitionedCall:3', 'index': 1612, 'shape': array([1, 1, 1]), 'shape_signature': array([ 1, -1, -1]), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}]

Any help really appreciated

#3

Hi @ayadtariq

Change

interpreter.set_tensor(input_details[0]['shape_signature'],img1)

to

interpreter.set_tensor(input_details[0]['index'],img1)

and see if this specific error disappears.

#5

Thank you for the help .I tried your suggestion and Im getting the error:
Cannot set tensor: Dimension mismatch. Got 59 but expected 1 for dimension 1 of input 0.

#6

From what you have stated above you are able to do inference… So for img1 use the same variable that you have used before. The input shape has to be the same with the TensorFlow model. Nothing changes during the conversion (with these parameters).

#8

The code never reaches interpreter.invoke() ,it always fails at interpreter.set_tensor no matter what argument I use

#10

is this issue solved or not?