TensorFlow Lite C API - Problem with input and output dimensions

, , ,
#1

Hello,

I am working on a neural network built with tensorflow. The format is .tflite. the model input dimensions are [1,30,1] / the ouput ones are [1,30].

I am aiming to feed a float input[30] in the model, and retrieve an inference output[30] with a C program

The thing is that i can’t figure out what is the problem as the code is crashing each time when copying the output tensor.

Here is the code I wrote :

     float input[30];
 float output[30];

     // resize input tensor
 int inputTensorSize = 30;
 int inputDims[3] = { 1, inputTensorSize, 1 };

 TfLiteInterpreterResizeInputTensor(IA_config0->interpreter_direct, 0, inputDims, 3);

 // re-allocate tensors
 TfLiteInterpreterAllocateTensors(IA_config0->interpreter_direct);

 // input buffer to input tensor
 IA_config0->input_tensor_direct = TfLiteInterpreterGetInputTensor(IA_config0->interpreter_direct, 0);

 ///////////////////////////////////////////////////////////////////

 statut = TfLiteTensorCopyFromBuffer(IA_config0->input_tensor_direct, input, inputTensorSize * sizeof(float));

 // Execute inference.
 statut = TfLiteInterpreterInvoke(IA_config0->interpreter_direct);

 // Extract the output tensor data.

 statut = TfLiteTensorCopyToBuffer(IA_config0->output_tensor_direct, output, 
     TfLiteTensorByteSize(IA_config0->output_tensor_direct));

IA_config0 is initialized before with a function :

IA_config IA_config0;

IA_config0.interpreter_direct = nullptr;
IA_config0.options_direct = nullptr;
IA_config0.model_direct = nullptr;
IA_config0.input_tensor_direct = nullptr;
IA_config0.output_tensor_direct = nullptr;

bool m_modelQuantized = false;

TfLiteDelegate* m_xnnpack_delegate;

std::string filename(name);

const char* filename0 = filename.c_str();

IA_config0.model_direct = TfLiteModelCreateFromFile(filename0);

IA_config0.options_direct = TfLiteInterpreterOptionsCreate();
TfLiteInterpreterOptionsSetNumThreads(IA_config0.options_direct, 1);

// Create the interpreter.
IA_config0.interpreter_direct = TfLiteInterpreterCreate(IA_config0.model_direct, IA_config0.options_direct);

IA_config0.input_tensor_direct = TfLiteInterpreterGetInputTensor(IA_config0.interpreter_direct, 0);

IA_config0.output_tensor_direct = TfLiteInterpreterGetOutputTensor(IA_config0.interpreter_direct, 0);

If the someone has ever faced this kind of issue, I would be really grateful if he could help me solving this out.

Thanks in advance,

Laurick

#3

It looks like you’re on the right track with your TensorFlow Lite C API usage, but there are a few key points to consider that might be causing the crash, especially around tensor resizing, input/output copying, and ensuring the correct dimensions and data types.

1.	Tensor Resizing and Allocation:
•	Ensure that TfLiteInterpreterResizeInputTensor and TfLiteInterpreterAllocateTensors are called before setting the input tensor data. This looks correct in your snippet, but ensure these calls are successful. Check the return status of these functions to ensure they’re kTfLiteOk.
2.	Setting Input Tensor Data:
•	Your approach to copy data into the input tensor seems correct. However, ensure that the input array is properly initialized with the data you want to feed into the model.
3.	Extracting Output Tensor Data:
•	You need to ensure that output_tensor_direct is correctly pointing to the output tensor before you try to copy data from it. It seems you’re setting it up before inference, but ensure it’s still valid post-inference. Models can sometimes change tensor allocations during TfLiteInterpreterInvoke, so it’s a good practice to retrieve the output tensor pointer after the inference call.
4.	Model Quantization:
•	You mentioned a variable bool m_modelQuantized. If your model is quantized, the way you handle input and output tensors will be different (e.g., you might be dealing with uint8_t or int8_t instead of float). Since your input and output arrays are float, make sure your model is not quantized or handle the quantization/dequantization process accordingly.
5.	Error Checking:
•	Check the status (statut) after important calls like TfLiteTensorCopyFromBuffer, TfLiteInterpreterInvoke, and TfLiteTensorCopyToBuffer to ensure they’re completing successfully (kTfLiteOk).
6.	Debugging Tips:
•	Use debugging tools or print statements to check the status codes of TensorFlow Lite functions.
•	Ensure that the dimensions of the input and output tensors match what your model expects. Mismatches in dimensions can cause crashes.
•	Check for null pointers or invalid references, especially with IA_config0->output_tensor_direct before and after the inference call.
7.	Memory Management:
•	Make sure that all dynamically allocated memory (if any) is managed properly. TensorFlow Lite C API handles most of the memory management internally, but if you’re allocating any additional resources, ensure they’re released appropriately.

If after checking these points you’re still facing issues, it might be helpful to isolate the problem by simplifying the code to just the model loading, resizing, and a single inference step, and then gradually add complexity to identify where the issue lies.

#4

Hi,

Thank you for your answer

The problem was coming from the model creation with the sizes of input and output. So, it was related to numbers 2 and 3 you wrote above

ps : finally, I am performing inference with ONNX which is easier to use and allows more complexe models to be read

Laurick

1 Like