Cannot get to work Tensorflow Serving REST API

General Discussion
#1

I am using a custom model based on faster_rcnn_resnet152_v1_640x640. After following many tutorials on implementing Tensorflow serving through docker with rest api, I am unable to get it to work.

Meanwhile, the model is working very well on my local machine. I am using this object detection script https://github.com/TannerGilbert/Tensorflow-Object-Detection-with-Tensorflow-2.0/blob/master/detect_from_image.py

My first issue was with the utf-8 format, which I solved by changing to uint8 that is expected by my model.

Then I fixed the input, which is “input_tensor” for my model.

I am using this code for my request:

import json
import numpy as np
import base64
import cv2

# Replace this with the actual image path you want to test
image_path = 'test_images/test_image/58111313621316938095299609.jpg'

# Read and preprocess the image
image_content = cv2.imread(image_path,1).astype('uint8').tolist()

# Prepare the JSON request with the signature name
data = {
    "signature_name": "serving_default",
    "instances": [{"input_tensor": image_content}]  # Adjust the input key based on your model's signature
}
# Replace these labels with your actual labels
labels = ['Like_post', 'Like_photo', 'Like_num', 'Like_redky']

# Send the inference request to TensorFlow Serving
url = 'http://localhost:8501/v1/models/v1:predict'  # Replace 'model' with the actual model name and version
headers = {"content-type": "application/json"}
response = requests.post(url, data=json.dumps(data), headers=headers)

# Process the response
if response.status_code == 200:
    predictions = response.json()['predictions'][0]
    predicted_class_idx = np.argmax(predictions)
    predicted_label = labels[predicted_class_idx]
    print("Predicted Label:", predicted_label)
    print("Class Probabilities:", predictions)
else:
    print("Error: Unable to get predictions. Status code:", response.status_code)
    print("Response content:", response.content)

This is part my output:

Predicted Label: Like_post
Class Probabilities: {'num_detections': 300.0, 'raw_detection_boxes': [[0.71807313, 0.049279619, 0.741069555, 0.115024686], [0.711260557, 0.0462932028, 0.743643343, 0.108250089], [0.708367467, 0.0556791946, 0.734233856, 0.112597443], [0.113099433, 0.0460921936, 0.145879179, 0.109014511],...

No matter which image I try, it always predicts my first label. As far as I understand, the script is taking the wrong key from the output thus incorrectly predicting the label. I am unable to find any information on how to solve this issue for my model, so I would be very glad if someone helped me out with this.

Also looks like I’m not supposed to get 300 num_detections, so I’d be happy to get any help regarding this as well.