Deploy model endpoint in a TFX pipeline

Sayak_Paul · July 23, 2021, 5:45am

Referring to the official Vertex AI x TFX example:

The tutorial shows how to push the trained model to a GCS bucket as the last step. What if I wanted to also deploy it to Vertex AI with an endpoint enabled?

Any resources?

Bhack · July 23, 2021, 12:23pm

If you want to serve a custom model there Is:

https://codelabs.developers.google.com/vertex_custom_training_prediction#0

Sayak_Paul · July 23, 2021, 12:50pm

It’s out of scope for my query. I can easily run a CustomTrainingJob on Vertex AI and directly deploy that.

But my query involves that from a pipeline which is different than what you referenced.

Divvya_Saxena · July 24, 2021, 2:59pm

@Sayak_Paul , That was by mistake. Not meant for your post.

Bhack · July 24, 2021, 3:29pm

Do you want to do this directly from a TFX pipeline?

Sayak_Paul · July 24, 2021, 3:54pm

After my model is pushed from the Pusher component of TFX, I want to be able to deploy it on Vertex AI.

Please follow the tutorial I mentioned in the initial post. You’ll see that the entire TFX Pipeline is orchestrated as a Vertex AI pipeline.

Ccing @Robert_Crowe if he has any suggestions.

Bhack · July 24, 2021, 4:24pm

Mhh I don’t think it is available off the shelf, but let’s wait for Robert.

Pobably a custom component is required, something like:

github.com

GoogleCloudPlatform/mlops-with-vertex-ai/blob/main/src/tfx_pipelines/components.py#L73-L116

      
        
            
            

            
@component
            def vertex_model_uploader(
                project: Parameter[str],
                region: Parameter[str],
                model_display_name: Parameter[str],
                pushed_model_location: Parameter[str],
                serving_image_uri: Parameter[str],
                model_blessing: InputArtifact[ModelBlessing],
                uploaded_model: OutputArtifact[UploadedModel],
                explanation_config: Parameter[str]="",
                labels: Parameter[str]="",
            ):
            
            
    vertex_ai.init(project=project, location=region)
                
                blessing = artifact_utils.get_single_instance([model_blessing])
                if not model_utils.is_model_blessed(blessing):
                    logging.info(f"Model is not uploaded to Vertex AI because it was not blessed by the evaluator.")

This file has been truncated. show original

Sayak_Paul · July 25, 2021, 8:43am

I agree. This repository is gold, thanks for mentioning it here. It’s pretty recent as well.

Robert_Crowe · July 26, 2021, 5:39pm

Unfortunately the official Vertex Pusher component is still being developed, but if you push to a file system destination then you can script deployment to Vertex Prediction.

Sayak_Paul · July 27, 2021, 1:32am

Thanks, @Robert_Crowe. Do you have a code example that I could refer to for supporting this?

Bhack · July 27, 2021, 9:09am

Isn’t it already in the mentioned repo (step 7)?

github.com

GoogleCloudPlatform/mlops-with-vertex-ai/blob/main/03-training-formalization.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "33b0bbba",
   "metadata": {},
   "source": [
    "# 03 - TFX Interactive Training Pipeline Execution\n",
    "\n",
    "The purpose of this notebook is to interactively run the following `TFX` pipeline steps:\n",
    "1. Receive hyperparameters using `hyperparam_gen` custom Python component.\n",
    "2. Extract data from BigQuery using `BigQueryExampleGen` component.\n",
    "3. Validate the raw data using `StatisticsGen` and `ExampleValidator` components.\n",
    "4. Process the data using `Transform` component.\n",
    "5. Train a custom model using `Trainer` component.\n",
    "7. Evaluate and Validate the custom model using `ModelEvaluator` component.\n",
    "7. Save the blessed to model registry location using `Pusher` component.\n",
    "8. Upload the model to Vertex AI using `vertex_model_pusher` custom Python component\n",
    "\n",
    "The custom components are implemented in the [tfx_pipeline/components.py](tfx_pipeline/components) module."

This file has been truncated. show original

Sayak_Paul · July 27, 2021, 9:21am

Step 9 I guess: “Upload the model to Vertex AI using vertex_model_pusher custom Python component”. I will try this one out and see.

Bhack · July 27, 2021, 10:26am

it was 7 but it was update some days ago now it is on point 8.

Sayak_Paul · July 27, 2021, 10:54am

But: there’s a duplicate number.

Bhack · July 27, 2021, 10:58am

Yes but I don’t mean notebook cells number I meant 8 (it was 7) as 8. Model Upload to AI Platform

Bhack · July 27, 2021, 10:59am

P.s. 7. Model pushing is Robert:

but if you push to a file system destination …

Sayak_Paul · August 16, 2021, 4:18am

Getting back to the solution that actually worked:

github.com

sayakpaul/Dual-Deployments-on-Vertex-AI/blob/main/notebooks/Custom_Model_TFX.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Custom_Model_TFX",
      "provenance": [],
      "collapsed_sections": [],
      "machine_shape": "hm",
      "include_colab_link": true
    },
    "environment": {
      "name": "tf2-gpu.2-4.mnightly-2021-02-02-debian-10-test",
      "type": "gcloud",
      "uri": "gcr.io/deeplearning-platform-release/tf2-gpu.2-4:mnightly-2021-02-02-debian-10-test"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"

This file has been truncated. show original