Convert efficientDet Lite 2 to mlmodel

General Discussion
,
#1

Hello,
has anyone tried to convert the SavedModel or tflite Model of EfficientDet-Lite-2 found on tfhub https://tfhub.dev/tensorflow/efficientdet/lite2/detection/1 to MlModel?

on coremltools, it says they accept the path to savedModel directory as an input. But I am having unsupported input type.

import coremltools as ct

mlmodel = ct.convert("efficientdet_lite2_detection_1")

I get this error:

.
.
.
    inputs.append(TensorType(name=inp, shape=shape, dtype=dtype))
  File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/coremltools/converters/mil/input_types.py", line 215, in __init__
    raise TypeError("dtype={} is unsupported for inputs/outputs of the model".format(dtype))
TypeError: dtype=<class 'coremltools.converters.mil.mil.types.type_int.make_int.<locals>.int'> is unsupported for inputs/outputs of the model

I think it’s due to “uint8” input type, and they dont support it.
I try again with ImageType like this:

model = ct.convert(
    "efficientdet_lite2_detection_1",
    inputs=[ct.ImageType(name="images", shape = (1, 448, 448, 3), scale=1 / 255.0, bias=[0, 0, 0], channel_first=False)],
)

I got this error:

.
.
.
  File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/coremltools/converters/mil/frontend/tensorflow/converter.py", line 374, in convert_main_graph
    func_inputs[input_type.name] = mb.placeholder(
  File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/coremltools/converters/mil/mil/builder.py", line 189, in placeholder
    return Placeholder(shape, dtype, allow_rank0_input=allow_rank0_input)
  File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/coremltools/converters/mil/mil/program.py", line 163, in __init__
    raise ValueError('Rank-0 (input {}) is unsupported'.format(name))
ValueError: Rank-0 (input None) is unsupported

This is probably due to input shape:

import tensorflow as tf
model = tf.saved_model.load("efficientdet_lite2_detection_1")
signature = model.signatures
input_tensors = signature["serving_default"].inputs
output_tensors = signature["serving_default"].outputs
print(input_tensors[0])
>>> Tensor("images:0", shape=(None, None, None, 3), dtype=uint8)

I tried to change input shape:

signature["serving_default"].inputs[0].shape = (1, 448, 448, 3)

>>>
Traceback (most recent call last):
  File "/Users/smurf/Desktop/repositories/playground/convert_coreml.py", line 11, in <module>
    signature["serving_default"].inputs[0].shape = (1, 448, 448, 3)
AttributeError: can't set attribute

Any ideas to solve this problem ?

Thank you

1 Like
#3

@ilyas_aroui,

Please refer to Convert Efficientnet to Core ML may help you.

If you still have issue, we request you post on Core ML Forum for quick assistance.

Thank you!

#4

Thank you! I actually I saw that post and I guess this is an issue with the saved Detection model more than it is with coremltools.

For example:

This Classification model has no issue:

import tensorflow_hub as hub
import tensorflow as tf
m = tf.keras.Sequential([
    hub.KerasLayer("https://tfhub.dev/tensorflow/efficientnet/b0/classification/1")
])
m.build([1, 256, 256, 3])

or with different batch_size and/or different Height and Width

import tensorflow_hub as hub
import tensorflow as tf
m = tf.keras.Sequential([
    hub.KerasLayer("https://tfhub.dev/tensorflow/efficientnet/b0/classification/1")
])
m.build([4, 224, 224, 3])
m.summary()

BUT the EfficientDetLite2

import tensorflow_hub as hub
import tensorflow as tf
m = tf.keras.Sequential([
    hub.KerasLayer("https://tfhub.dev/tensorflow/efficientdet/lite2/detection/1")
])
m.build([1, 256, 256, 3])
m.summary()

throws this error:

Traceback (most recent call last):
  File "/Users/smurf/Desktop/repositories/playground/convert_coreml.py", line 6, in <module>
    m.build([1, 256, 256, 3])
  File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/keras/engine/sequential.py", line 349, in build
    super(Sequential, self).build(input_shape)
  File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/keras/engine/training.py", line 440, in build
    self.call(x, **kwargs)
  File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/keras/engine/sequential.py", line 388, in call
    outputs = layer(inputs, **kwargs)
  File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/tensorflow/python/autograph/impl/api.py", line 692, in wrapper
    raise e.ag_error_metadata.to_exception(e)
ValueError: Exception encountered when calling layer "keras_layer" (type KerasLayer).

in user code:

    File "/Users/smurf/miniconda3/envs/coreml_conv/lib/python3.9/site-packages/tensorflow_hub/keras_layer.py", line 229, in call  *
        result = f()

    ValueError: Could not find matching concrete function to call loaded from the SavedModel. Got:
      Positional arguments (1 total):
        * Tensor("imgs:0", shape=(1, 256, 256, 3), dtype=float32)
      Keyword arguments: {}
    
     Expected these arguments to match one of the following 1 option(s):
    
    Option 1:
      Positional arguments (1 total):
        * TensorSpec(shape=(None, None, None, 3), dtype=tf.uint8, name='images')
      Keyword arguments: {}


Call arguments received:
  • inputs=tf.Tensor(shape=(1, 256, 256, 3), dtype=float32)
  • training=None

Any idea how I can go about this ?
Thank you

1 Like
#6

I found the error in efficientDet conversion. There are two inputs names: images and StatefulPartitionedCall/loop_body/PlaceholderWithDefault the second one is crashing when coremltools tries to initiate a placeholder for it.

can you please explain what does this input stand for ? can i skip it ?

#8

I have tried the conversion with my model optimization sequence. It is a bit tricky, but once through the tool I created, the tflite model can be converted to tflite with some optimized structure.

tflite → onnx → saved_model → coreml

# tflite to ONNX
python3 -m tf2onnx.convert \
--opset 12 \
--tflite lite-model_efficientdet_lite2_detection_default_1.tflite \
--output efficientdet_lite2_detection_1.onnx \
--dequantize \
--inputs-as-nchw serving_default_images:0

# Graph Optimization
onnxsim efficientdet_lite2_detection_1.onnx efficientdet_lite2_detection_1.onnx
onnxsim efficientdet_lite2_detection_1.onnx efficientdet_lite2_detection_1.onnx
onnxsim efficientdet_lite2_detection_1.onnx efficientdet_lite2_detection_1.onnx

# Separate uint8 input OP and Cast
sne4onnx \
-if efficientdet_lite2_detection_1.onnx \
-ion serving_default_images:0_dequant_Cast__355:0 \
-oon StatefulPartitionedCall:0 StatefulPartitionedCall:1 StatefulPartitionedCall:2 StatefulPartitionedCall:3 \
-of efficientdet_lite2_detection_1_inp_float32.onnx

# onnx to saved_model
onnx2tf -i efficientdet_lite2_detection_1_inp_float32.onnx -osd

# saved_model_cli
saved_model_cli show --dir saved_model/ --all

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['__saved_model_init_op']:
    The given SavedModel SignatureDef contains the following input(s):
    The given SavedModel SignatureDef contains the following output(s):
    outputs['__saved_model_init_op'] tensor_info:
        dtype: DT_INVALID
        shape: unknown_rank
        name: NoOp
    Method name is:

signature_def['serving_default']:
    The given SavedModel SignatureDef contains the following input(s):
    inputs['serving_default_images_0_dequant_Cast__355_0'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 448, 448, 3)
        name: serving_default_serving_default_images_0_dequant_Cast__355_0:0
    The given SavedModel SignatureDef contains the following output(s):
    outputs['tf.cast_92'] tensor_info:
        dtype: DT_FLOAT
        shape: (1)
        name: PartitionedCall:0
    outputs['tf.compat.v1.pad_185'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 25)
        name: PartitionedCall:1
    outputs['tf.compat.v1.pad_186'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 25, 4)
        name: PartitionedCall:2
    outputs['tf.compat.v1.pad_187'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 25)
        name: PartitionedCall:3
    Method name is: tensorflow/serving/predict

The error message you posted no longer appears, instead the error message “Conversion for TF op ‘NonMaxSuppressionV4’ not implemented.” appears. Unfortunately, CoreML does not appear to support NMS.

import coremltools as ct
mlmodel = ct.convert("saved_model")

Conversion for TF op 'NonMaxSuppressionV4' not implemented.

name: "PartitionedCall/model/nms_layer/non_max_suppression/NonMaxSuppressionV4"
op: "NonMaxSuppressionV4"
input: "PartitionedCall/model/tf.compat.v1.squeeze_12/Squeeze"
input: "PartitionedCall/model/tf.compat.v1.squeeze_14/Squeeze"
input: "PartitionedCall/model/tf.where/SelectV2"
input: "PartitionedCall/model/nms_layer/non_max_suppression/iou_threshold"
input: "PartitionedCall/model/nms_layer/non_max_suppression/score_threshold"
attr {
    key: "T"
    value {
        type: DT_FLOAT
    }
}
attr {
    key: "T_threshold"
    value {
        type: DT_FLOAT
    }
}
attr {
    key: "pad_to_max_output_size"
    value {
        b: true
    }
}
File "/home/xxxx/work/coreml/convert_to_coreml.py", line 3, in <module>
    mlmodel = ct.convert("saved_model")
NotImplementedError: Conversion for TF op 'NonMaxSuppressionV4' not implemented.

name: "PartitionedCall/model/nms_layer/non_max_suppression/NonMaxSuppressionV4"
op: "NonMaxSuppressionV4"
input: "PartitionedCall/model/tf.compat.v1.squeeze_12/Squeeze"
input: "PartitionedCall/model/tf.compat.v1.squeeze_14/Squeeze"
input: "PartitionedCall/model/tf.where/SelectV2"
input: "PartitionedCall/model/nms_layer/non_max_suppression/iou_threshold"
input: "PartitionedCall/model/nms_layer/non_max_suppression/score_threshold"
attr {
    key: "T"
    value {
        type: DT_FLOAT
    }
}
attr {
    key: "T_threshold"
    value {
        type: DT_FLOAT
    }
}
attr {
    key: "pad_to_max_output_size"
    value {
        b: true
    }
}
#10

I have the same problem. Can you help ?