Error: The shape of dict['input_tensor'] provided in model.execute(dict) must be [1,-1,-1,3]

General Discussion
, ,
#6

Was the original Python model Keras saved model (.h5 file) or regular TF savedModel (typically .pb file)?

For Keras saved model you would use:

tf.loadLayersModel(MODEL_URL);

For TF SavedModel you would use:

tf.loadGraphModel(MODEL_URL);

Looking at your error message though in the first instance it seems your passing a tensor of the wrong shape. Check the tensor of the data you are sending into the model and figure out why it is 4 instead of 3 in that last dimension. What image did you load? PNG? What exactly is your input_tensor? Do you have this on a Glitch or Codpen somewhere for me to see running?

#7

It’s TF savedModel.
using loadGraphModel()
loading png image

    const image = fs.readFileSync(imageFile); 
    let decodedImage = tfnode.node.decodeImage(image);
    let inputTensor = decodedImage.expandDims();
   model.predict()

the repo is here GitHub - playground/tfjs-object-detection

#8

Can you try with JPG? I think it may be because PNG has 4 channels, JPG will have 3. IF that is the case then you simply need to force the channels to be 3 and ignore the alpha channel for transparency on PNGs:

There is parameter to force it to be 3 under channels which is optional parameter you can specify.

1 Like
#9

with jpg image, I get Error: The dtype of dict[‘input_tensor’] provided in model.execute(dict) must be int32, but was float32

#11

In that case you will need to convert the resulting tensor to be integer values and not floating point.

You can use something like:

tf.cast(data, 'int32')

However before doing that inspect the resulting Tensor to see what the values are after the image read. Depending how it decoded if it decided to take RGB values and normalize them eg instead of numbers from 0-255 it gives numbers 0 -1 as a floating point then you would want to convert back to whole number by multiplying by 255 first. Else 0.1239803 would just become 0 in a cast to integer which is not what you want!

However if you are using the PNG method I listed above it automatically returns an int32 tensor so it is probably better to do that if you are not using that method and you know your images are always PNG.

#12

Now, it’s getting Image:

  kept: false,
  isDisposedInternal: false,
  shape: [ 1, 600, 800, 3 ],
  dtype: 'int32',
  size: 1440000,
  strides: [ 1440000, 2400, 3 ],
  dataId: {},
  id: 918,
  rankType: '4',
  scopeId: 2
}
Error: This execution contains the node 'StatefulPartitionedCall/map/while/exit/_727', which has the dynamic op 'Exit'. Please use model.executeAsync() instead. Alternatively, to avoid the dynamic ops, specify the inputs [StatefulPartitionedCall/map/TensorArrayV2Stack_1/TensorListStack]

If use loadSavedModel(savedModel) without the conversion, it works fine for both jpg and png images.

Tensorflow.js video is wrong size
#13

Did you try what it recommended? Eg dont use predict but use executeAsync() instead?

#14

Actually, I have tried that earlier and this is what throws with model.executeAsync

with jpg image

Image: 1440000 bytes with shape: Tensor {
  kept: false,
  isDisposedInternal: false,
  shape: [ 1, 600, 800, 3 ],
  dtype: 'int32',
  size: 1440000,
  strides: [ 1440000, 2400, 3 ],
  dataId: {},
  id: 918,
  rankType: '4',
  scopeId: 2
}

Error: Invalid TF_Status: 3
Message: In[0] and In[1] has different ndims: [1,8,8,64,2] vs. [2,1]

With png image

Error: The shape of dict['input_tensor'] provided in model.execute(dict) must be [1,-1,-1,3], but was [1,600,800,4]
#16

Hello again! Sorry for the slight delay here. I have been discussing with our software engineers on the team. It seems we may have found a bug here and they would like you to submit an issue on the TFJS github which you can find here: GitHub - tensorflow/tfjs: A WebGL accelerated JavaScript library for training and deploying ML models.

Once you have submitted a bug, feel free to let me know the link and I can also send that link to the SWE directly who is looking into this to kick that process off.

Thanks for your patients here and for being part of the community :slight_smile:

#17

Hi @Jason,

Thanks for the update, I have submitted a bug here Error: The shape of dict[‘input_tensor’] provided in model.execute(dict) must be [1,-1,-1,3] · Issue #5366 · tensorflow/tfjs · GitHub.

Also this might be a regression bug with the recent release of tfjs-node v3.8.0, I have posted that here Failed to find bogomips warning · Issue #38260 · tensorflow/tensorflow · GitHub

A different question, is there an example of training savedmodel using nodejs you can point me to, similar to this example mnist-node

#19

This is the only codelab for Node that we have right now:

If you do get this working it may be a good blog writeup if you are interested!

#21

Ok, thanks, will take a look and see.

#22

Hi @Jason. Another question if I may, the reason I’m asking for a Nodejs version for training models is that my environment that I have been using to train models on my Mac for some reason in the last couple of weeks it seems to be acting up and not producing the proper result like it did before. Not sure what got changed, the trained savedmodel instead of recognizing the proper objects, it’s producing many random bounding boxes. In last couple of days I have been trying to dockerize the training pipeline using Ubuntu 20.04, 21.04 with different python version 3.7, 3.8 and 3.9 but the training keeps failing with the same errors

WARNING:tensorflow:Gradients do not exist for variables ['top_bn/gamma:0', 'top_bn/beta:0'] when minimizing the loss.
W0723 03:05:32.545922 140238382945856 utils.py:78] Gradients do not exist for variables ['top_bn/gamma:0', 'top_bn/beta:0'] when minimizing the loss.
WARNING:tensorflow:Gradients do not exist for variables ['top_bn/gamma:0', 'top_bn/beta:0'] when minimizing the loss.
W0723 03:05:44.013801 140238382945856 utils.py:78] Gradients do not exist for variables ['top_bn/gamma:0', 'top_bn/beta:0'] when minimizing the loss.
Killed

    at ChildProcess.exithandler (node:child_process:397:12)
    at ChildProcess.emit (node:events:394:28)
    at maybeClose (node:internal/child_process:1067:16)
    at Socket.<anonymous> (node:internal/child_process:453:11)
    at Socket.emit (node:events:394:28)
    at Pipe.<anonymous> (node:net:662:12) {
  killed: false,
  code: 137,
  signal: null,
  cmd: 'python /server/models/research/object_detection/model_main_tf2.py --pipeline_config_path=/server/data-set/ssd_efficientdet_d0_512x512_coco17_tpu-8.config --model_dir=/server/data-set/training --alsologtostderr'

Any idea or suggestions?

#24

Did anyone ever find a solution to this problem? I have tried so many possible solutions posted on stackoverflow but with no success at all. I have tried both jpg and png.

const tf = require('@tensorflow/tfjs-node')
const fs = require('fs');

 var path = 'webcams/Screenshot_1.png';
const buf = fs.readFileSync(path);
const input = tf.node.decodePng(buf)

async function load(img){
  const model = await tf.loadGraphModel('file://webcam_tfjs_model/webcam_tfjs_model/model.json')

    // const tensor = tf.tensor4d(Array.from(img.dataSync()), [1,-1,-1, 3]);
   // if I use the above line I get : Error: Tensor must have a shape comprised of positive integers but got shape [1,-1,-1,3].

  const predictions = await model.executeAsync(img.expandDims())
  console.log('Predictions: ', predictions);
}

load(input)

// Error: The shape of dict['input_tensor'] provided in model.execute(dict) must be [1,-1,-1,3], but was [1112,1975,4]
#25

It seems it wants JPG not PNG with 4 channels and in a batch of one? Did you try with JPG and then use .expandDims() to create batch of 1 that would give you the [1,1112,1975,3] shape that would then be valid?

#27

Hi Jason, thanks that indeed got rid of the error. But I do not know how to decipher the the array it has given me. I was expecting bounding box coordinates, but I am not sure if they are hidden in here or not…

Predictions:  [
  Tensor {
    kept: false,
    isDisposedInternal: false,
    shape: [ 1, 30 ],
    dtype: 'float32',
    size: 30,
    strides: [ 30 ],
    dataId: {},
    id: 2283,
    rankType: '2',
    scopeId: 2506
  },
  Tensor {
    kept: false,
    isDisposedInternal: false,
    shape: [ 1, 30 ],
    dtype: 'float32',
    size: 30,
    strides: [ 30 ],
    dataId: {},
    id: 2175,
    rankType: '2',
    scopeId: 2376
  },
  Tensor {
    kept: false,
    isDisposedInternal: false,
    shape: [ 1, 30, 4 ],
    dtype: 'float32',
    size: 120,
    strides: [ 120, 4 ],
    dataId: {},
    id: 2139,
    rankType: '3',
    scopeId: 2334
  },
  Tensor {
    kept: false,
    isDisposedInternal: false,
    shape: [ 1, 30 ],
    dtype: 'float32',
    size: 30,
    strides: [ 30 ],
    dataId: {},
    id: 2330,
    rankType: '2',
    scopeId: 2561
  },
  Tensor {
    kept: false,
    isDisposedInternal: false,
    shape: [ 1, 300, 2 ],
    dtype: 'float32',
    size: 600,
    strides: [ 600, 2 ],
    dataId: {},
    id: 2071,
    rankType: '3',
    scopeId: 2241
  },
  Tensor {
    kept: false,
    isDisposedInternal: false,
    shape: [ 1, 30, 2 ],
    dtype: 'float32',
    size: 60,
    strides: [ 60, 2 ],
    dataId: {},
    id: 2237,
    rankType: '3',
    scopeId: 2453
  },
  Tensor {
    kept: false,
    isDisposedInternal: false,
    shape: [ 1 ],
    dtype: 'float32',
    size: 1,
    strides: [],
    dataId: {},
    id: 2085,
    rankType: '1',
    scopeId: 2265
  },
  Tensor {
    kept: false,
    isDisposedInternal: false,
    shape: [ 1, 300, 4 ],
    dtype: 'float32',
    size: 1200,
    strides: [ 1200, 4 ],
    dataId: {},
    id: 2072,
    rankType: '3',
    scopeId: 2243
  }
]
#28

It seems you are printing the raw tensors, you need to call .arraySync() or .dataSync() on the Tensor returned to synchronously grab the actual data the tensor contains. Alternatively you can use .array() or .data() for async (more efficient). You can learn about all of these things in my free course starting on Google Developers: https://goo.gle/learn-WebML if you are new to working with TFJS API.

#29

I tried to do it as in your 3.6.2 video but it won’t let me do it like you have. Firstly I can only use executeAsync not predict. And secondly I can’t call data.Sync or arrraySync on the result. I get 8 Tensor results and I can call it on each of those but have no idea what to make of any of the data. I figure my output is somewhat similar to TensorFlow Hub however its different enough that I cannot use this as a guide. I have pretty much given up at this point. I just wanted to get the bounding box of a webcam within a screenshot of a live stream but have wasted days going down the rabbit hole. I tried my best to shoehorn this solution to mine but it couldn’t javascript - How to get bounding box from object detection of TensorFlowJS? - Stack Overflow .

#30

Can you show me your updated code that you tried?

#32

In the end I tried the cocssd library but got the same error as I was getting manually trying to use the code from the stack overflow.

const tf = require('@tensorflow/tfjs-node')
const cocoSsd = require('@tensorflow-models/coco-ssd');
const fs = require('fs');

 var path = './webcams/test.jpg';
const buf = fs.readFileSync(path);
const input = tf.node.decodeJpeg(buf)

async function load(img){

  
   // I also tried the cocossd version and got the same error
  
  cocoSsd.load({
      base: 'lite_mobilenet_v2',
      modelUrl: 'file://webcam_tfjs_model/webcam_tfjs_model/model.json'
    }
  ).then(model => {
    // detect objects in the image.
    model.detect(img).then(predictions => {
      console.log('Predictions: ', predictions);
    });
  });
}

load(input)