Decision Forests issue with C_API

General Discussion
, , ,
#1

Hi,
I am developing a decision forest model using TF-DF. I trained the model in Python and saved it using SavedModel. Then I want to use it in my program that is written in C. found that for this task I need to load the decision forest inference.so file from the Python package.

After that in my program, I load the inference.so file using TF_LoadLibrary. Then I load the model using TF_LoadSessionFromSavedModel.

Here is the code

#include <stdio.h>
#include <tensorflow/c/c_api.h>

int main() {
  TF_Graph *Graph = TF_NewGraph();
  TF_Status *Status = TF_NewStatus();
  TF_SessionOptions *SessionOpts = TF_NewSessionOptions();
  TF_Buffer *RunOpts = NULL;
  TF_Library *library;

  library = TF_LoadLibrary("/home/user/.local/lib/python3.7/site-packages/tensorflow_decision_forests/tensorflow/ops/inference/inference.so",
                              Status);

  const char *saved_model_dir = "randomforests-model/";
  const char *tags = "serve";
  int ntags = 1;

  TF_Session *Session = TF_LoadSessionFromSavedModel(
      SessionOpts, RunOpts, saved_model_dir, &tags, ntags, Graph, NULL, Status);

  printf("status: %s\n", TF_Message(Status));

  if(TF_GetCode(Status) == TF_OK) {
    printf("loaded\n");
  }else{
    printf("not loaded\n");
  }

  return 0;
}

Output

$ gcc -g main.c -ltensorflow -o main.out
user@debian:/home/code/tftest$ ./main.out 
Hello from TensorFlow C library version 2.7.0-dev20211101
2022-01-22 19:39:28.539621: I tensorflow/cc/saved_model/reader.cc:43] Reading SavedModel from: randomforests-mtproto-model-001028/
2022-01-22 19:39:28.547223: I tensorflow/cc/saved_model/reader.cc:107] Reading meta graph with tags { serve }
2022-01-22 19:39:28.547792: I tensorflow/cc/saved_model/reader.cc:148] Reading SavedModel debug info (if present) from: randomforests-mtproto-model-001028/
2022-01-22 19:39:28.548298: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-01-22 19:39:28.598841: I tensorflow/cc/saved_model/loader.cc:210] Restoring SavedModel bundle.
2022-01-22 19:39:28.743885: I tensorflow/cc/saved_model/loader.cc:194] Running initialization op on SavedModel bundle at path: randomforests-mtproto-model-001028/
[INFO kernel.cc:1153] Loading model from path
[INFO decision_forest.cc:617] Model loaded with 300 root(s), 618972 node(s), and 28 input feature(s).
[INFO abstract_model.cc:1063] Engine "RandomForestOptPred" built
[INFO kernel.cc:1001] Use fast generic engine
2022-01-22 19:39:30.922861: I tensorflow/cc/saved_model/loader.cc:283] SavedModel load for tags { serve }; Status: success: OK. Took 2383248 microseconds.

status: No shape inference function exists for op 'SimpleMLLoadModelFromPathWithHandle', did you forget to define it?
not loaded

The problem is that the function output is telling that the model is loading with Status: success: OK but the Status variable is not equal to TF_OK and the related message is No shape inference function exists for op 'SimpleMLLoadModelFromPathWithHandle', did you forget to define it?

So how can I load the model in the right way?

I also asked this question from Stackoverflow but didn’t get any answer.

Thanks.

#3

@Mathieu might be able to help

#5

hi @hpirlo,

Thanks for reporting your issue. I thought that after loading inference.so would be all that one needed. We’ll investigate and figure it out – and maybe add a documentation page.

Now, if you want to do inference it in C/C++, we recommend using directly the TF-DF C++ library, called Yggdrasil DF. It’s very simple to use, and it reads directly the model from the TensorFlow SavedModel’s assets sub-directory, see doc here, and here a doc on how to do inference using the C++ library.

The reason for that is that TensorFlow is a much larger machinery, and the decision forest models are significantly faster (in one of our 70-trees GBDT model we had inference in 700 nanoseconds!) and lighter without the TF framework.

We’ll follow up on the issue anyway – it should just work using TF library also. And let us know if you bump with any issues with YDF, if try that route.

2 Likes
#7

Hi @Jan,

It would be helpful if there was a serving mechanism that did not rely on using Yggrdrasil directly. I’m currently working on trying to serve a tfdf model using the Tensorflow Java SDK and am running into the same issue.