TFX Model Monitoring

General Discussion
,
#1

Hello,

I hope everyone is having a good day. I have been experimenting on TFX and running pipelines in Kubeflow trying to build an end-to-end solution.

After the model deployment phase, I have been wondering if there is a supported tool for model monitoring in the production environment to check whether a model is performing as expected/ what are the responses to the inference requests and if they are true in nature… I’m looking for a deployment solution… keeping eye on the running model especially in the kubeflow context.

I would love to hear from the team.

Thank You.

2 Likes
#2

Hi Muhammad,

It depends on where you’re deploying to. For TFlite and TFJS deployments model monitoring is very different. For TensorFlow Serving it would mostly be a matter of logging and then analyzing the log data for drift and skew. To detect concept drift you will need labeling of samples of prediction requests. For Vertex AI there is a preview release of model monitoring support.

thanks,

Robert

3 Likes
#3

Thank you for the reply.

In the TensorFlow Serving context, besides the GCP Vertex AI solution. Is there any open-source on-premise solution available for logging and analyzing the model in production?

and what are the other methods to drive model metrics besides labelling and is there any open-source solution available for that as well?

Waiting for your reply.

Thank you.

2 Likes