Official tensorflow serving tutorial is not working in colab

Nitish_Pandey_NSL · December 19, 2022, 2:13am

The official tensorflow serving simple tutorial (link - Train and serve a TensorFlow model with TensorFlow Serving | TFX) is not working in colab when I use the run in colab option.

The part of code which is throwing error is this:

headers = {"content-type": "application/json"}
json_response = requests.post('http://localhost:8501/v1/models/fashion_model:predict', data=data, headers=headers)
predictions = json.loads(json_response.text)['predictions']

The exact error it is giving is this:

ConnectionError: HTTPConnectionPool(host=‘localhost’, port=8501): Max retries exceeded with url: /v1/models/fashion_model:predict (Caused by NewConnectionError(’<urllib3.connection.HTTPConnection object at 0x7fc6c23768b0>: Failed to establish a new connection: [Errno 111] Connection refused’))

Please tell me what should I do? Also, please fix this in the official tutorial notebook.

#serving #tfserving #tensorflow_serving #tf-serving #tensorflow-serving #tf_serving #tfs #tensorflow #tf

chunduriv · December 19, 2022, 6:58am

@Nitish_Pandey_NSL,

Welcome to the Tensorflow Forum!

As a workaround, downgrade tensorflow-model-server from 2.11.0 to 2.8.0.

In Install TensorFlow Serving section of the notebook, please uncomment !{SUDO_IF_NEEDED} apt-get install tensorflow-model-server line and add below lines.

Code to install tensorflow-model-server 2.8.0:

!wget 'http://storage.googleapis.com/tensorflow-serving-apt/pool/tensorflow-model-server-2.8.0/t/tensorflow-model-server/tensorflow-model-server_2.8.0_all.deb'
!dpkg -i tensorflow-model-server_2.8.0_all.deb
!pip3 install tensorflow-serving-api==2.8.0

Please refer to the working gist. Thank you!

markdaoust · December 19, 2022, 5:09pm

If you found a fix, please submit a PR to the TFX repo to fix this for everyone.

github.com

tensorflow/tfx/blob/master/docs/tutorials/serving/rest_simple.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MhoQ0WE77laV"
      },
      "source": [
        "##### Copyright 2020 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "_ckMIh7O7s6D"
      },
      "outputs": [],
      "source": [

This file has been truncated. show original

chunduriv · December 19, 2022, 5:49pm

Thank you Mark.

Submitted `Rest_Simple` tutorial is not working in Colab by chunduriv · Pull Request #5602 · tensorflow/tfx · GitHub

Nitish_Pandey_NSL · December 19, 2022, 5:51pm

Thank you very much @chunduriv for providing a workaround.