Unknown layer ERROR in importing Keras model to TensorFlow JS

General Discussion
, ,
#1

Hello,

I’m pretty new to TF and just started by working around this tutorial: “pose classification in keras”.
I am using my own dataset for training/testing and I adjusted the model config :

model.compile(

    optimizer='adam',

    loss='categorical_crossentropy',

    metrics=['accuracy']

)

# Add a checkpoint callback to store the checkpoint that has the highest

# validation accuracy.

checkpoint_path = "weights.best.hdf5"

checkpoint = keras.callbacks.ModelCheckpoint(checkpoint_path,

                             monitor='val_accuracy',

                             verbose=1,

                             save_best_only=True,

                             mode='max')

earlystopping = keras.callbacks.EarlyStopping(monitor='val_accuracy', 

                                              patience=20)

# Start training

history = model.fit(X_train, y_train,

                    epochs=200,

                    batch_size=16,

                    validation_data=(X_val, y_val),

                    callbacks=[checkpoint, earlystopping])

tfjs.converters.save_keras_model(model,'./path')

when importing the tfjs converted model with this command:

model = await tf.loadLayersModel('./path');

I get the following error :

Error: Unknown layer: SlicingOpLambda. This may be due to one of the following reasons:
1. The layer is defined in Python, in which case it needs to be ported to TensorFlow.js or your JavaScript code.
2. The custom layer is defined in JavaScript, but is not registered properly with tf.serialization.registerClass().

I’m not post processing or rescaling the layers. If you know what the issue might be and have any suggestions, I’ll much appreciate it.

Thanks,
Parvin

#3

It seems you are using some op contained in a layer that is not supported in TensorFlow.js. I do not use Python so maybe someone who is a TF Python expert can tell you what layer is using that SlicingOpLambda and you can remove it and just use regular layers that are more common that are known to convert. After all we have many pose estimation models in TFJS that do work (MoveNet, PoseNet etc) so it seems this just needs to be changed.

Looking at your layers in the example code though you appear to have:

# Define the model
inputs = tf.keras.Input(shape=(51))
embedding = landmarks_to_embedding(inputs)

layer = keras.layers.Dense(128, activation=tf.nn.relu6)(embedding)
layer = keras.layers.Dropout(0.5)(layer)
layer = keras.layers.Dense(64, activation=tf.nn.relu6)(layer)
layer = keras.layers.Dropout(0.5)(layer)
outputs = keras.layers.Dense(len(class_names), activation="softmax")(layer)

model = keras.Model(inputs, outputs)
model.summary()

Tensorflow.js definitely supports layers for dense and dropout so maybe the problem is coming from the embedding line? But that is just a guess as I am not familiar with Python code.

The other option is to write your pre/post processing in JS and then just use the TFJS api instead directly to make your model rather than trying to convert Python ones.

Also check this related post that shows the op support TFJS has:

https://tensorflow-prod.ospodiscourse.com/t/tensorflow-js-op-support-matrix-if-you-are-having-python-model-conversion-issues-check-this-first/4000

#4

Thanks for the suggestions. I’ll look into them and get back to this thread.
I have a question regarding classifying poses ( like yoga poses classification). As far as I’m concerned, this is the dev pipeline, which I’m using:

1. Data collection: Set up the web application with movenet to extract 17 keypoints 
2. Preprocess keypoints (normalize)
3. Training: Define a model to train for classification 
4. Deployment: import the model, and we will need to import movenet to feed the trained model with keypoints.

Here is the question: What are the options for classification? I know there is Python Keras classifier which can be converted to TFJS, and ml5 library to train. Is there any guideline to classify poses in TFJS directly and import that costumed model without conversion?

#5

Did you get chance to check out this demo tfjs-models/pose-detection at master · tensorflow/tfjs-models · GitHub ?

#6

Yes, movenet is basically what I’m using for data collection.

#7

So you may have seen a website called Teachable Machine that allows you to classify custom poses. They use the older PoseNet model which I do not recommend anymore but essentially I believe they take the outputs one of the higher layers of that model and then add a simple multi layer perceptron to the end of that which is then trained to classify the poses (the original model is not re-trained of course as that would need all the original training data which they dont have so its frozen).

Essentially you have a trained pose model that you dont want to retrain per se, but you just want to retrain some final custom layers yourself to detect poses from the knowledge the pose model already has - a form of transfer learning if you will.

I actually am in the process of writing an EdX course for TensorFlow.js and just completed writing an example that shows how to load the raw MoveNet model from TF Hub which theoretically you could inspect with model.summary (in JavaScript) and then build upon. See this link:

Also check this example I wrote which shows how to grab some intermediate layer which you could then feed into your own custom trainable layers as inputs.