TFX Object Detection

@tensorflow_team

I’ve been reading up on TFX and wanted to make a pipeline for object detection, using an “off-the-shelf” model from Model Hub or the object detection API. I’ve also had a look at the TF Lite model builder.

But (as far as I can see) all of these options has a snag that makes it unusable in the context of a TFX pipeline.

The closest I’ve gotten is with a Efficientdet feature-vector from Model Hub:

My question:
Is there an easy way to retrain a Tensorflow object detection model in a TFX pipeline at this moment in time?

regards,
Anders

Here is the sample example on image classification for TFX.

Train and serve a TensorFlow model with TensorFlow Serving

Yes, I’m aware of that. This question is about object detection.

Which part of object detection is sufficiently different from image classification that you’re having problems?

Hi @Robert_Crowe,
Thanks for reaching out on this issue.

My main problem is that I have not succeeded in taking a pre-built model from either Tensorflow Hub or Tensorflow Object Detection API, or anyother place.

As stated on another thread on this forum:

the Object Detection fine tuning story on TFHub is not perfect. Most of the OD models that were published don’t have the proper api to be fine tuned easily (like the image classification ones you mentioned).

As you know the TFX trainer module expects the function _build_keras_model() to return a compiled Keras model.

Model HUB approach

If I’m not mistaken, I can’t use tf.keras.Sequential() (as in the above tutorial) because there is more than one output from object detection models. The class, and the box.

I tried a naive approach with the functional API:

    img_inputs = keras.Input(shape=IMAGE_SIZE + (3,))
    hub_model = hub.KerasLayer(model_handle, trainable=do_fine_tuning)
    outputs = hub_model(img_inputs)
    model = keras.Model(inputs=img_inputs, outputs=outputs, name="obj_det_model")

    loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
    model.compile(optimizer="adam", loss=loss_fn, metrics=["accuracy"])

This got me a bit further, but will not work because of the loss function. I need a combination of the Huber and Focal functions. But implementing the loss function is a bit over my level. Even if I felt comfortable doing it, it’s not quite an “of-the-shelf” solution.

Object Detection API Approach

The Tensorflow object detection API has some very good notebook tutorials but the model that is built is of type object_detection.meta_arcitectures.ssd_mera_arch.<MODEL>Arch. I have not found a way to get from that to a Keras model.

Trying to load a saved model using tf.keras.models.load_model() gives me a <tensorflow.python.saved_model.load.Loader._recreate_base_user_object.<locals>._UserObject not a keras model.

Related(?) Github issue github.com/tensorflow/tensorflow/issues/33870

Training from scratch

I’m currently following the instructions on training a model from scratch with the object detection API.

Recap

So I have tried quite a few things. I don’t see an easy way to take a predefined object detection model and use it in a TFX pipeline. I would be happy to be proven wrong with an example.

Regards,
Anders

Update

I finished training a version of Centerdet with the Object Detection API. It produces checkpoints, no saved model. I don’t know how to convert from checkpoints to the TF >2.5 version of saved_model.

AutoML

I’ve also tried the AutoML repository. But it also produces the old version of saved_model. (there is no keras_metadata.pb file)

After training, are you calling model.save()?

AutoML uses the TF v1 method b = tf.saved_model.Builder(output_dir)

https://github.com/google/automl/blob/39c39e5b635f73e86082bff3deddefecfbc0a123/efficientdet/model_inspect.py#L338

Object Detection API
Also seems to use the deprecated tf.saved_model.save

github.com/tensorflow/models/research/object_detection/exporter_lib_v2.py#L278

Model Hub

I have not come so far the I could call model.save() on a trained model.

Hello. I’m also interested in integrate Object Detection API in TFX.
More in general, is it possible using tfx trainer with custom trainer loop where there is not a compiled keras model?? ( Writing a training loop from scratch | TensorFlow Core)

Thanks!!