TFLITE does not support LSTM?

General Discussion
, ,
#1

1. When converting LSTM to tflite, the following WARNING message appears :thinking:.

WARNING:absl:Found untraced functions such as lstm_cell_16_layer_call_and_return_conditional_losses, lstm_cell_16_layer_call_fn, lstm_cell_17_layer_call_and_return_conditional_losses, lstm_cell_17_layer_call_fn, lstm_cell_18_layer_call_and_return_conditional_losses while saving (showing 5 of 40). These functions will not be directly callable after loading.

2. first, ignore the message and measure test accuracy on computer with the tflite model

it shows 14% accuracy…! Before the conversion, it was 94%.

3. Here’s the code to convert the model

h5_model = load_model('//content//multi_lstm.h5')
converter_h5 = tf.lite.TFLiteConverter.from_keras_model(h5_model)
converter_h5.optimizations = [tf.lite.Optimize.DEFAULT]
converter_h5.experimental_new_converter=True
converter_h5.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, 
                                          tf.lite.OpsSet.SELECT_TF_OPS]
tflite_model = converter_h5.convert()

I don’t know why this is happening. Any help would be greatly appreciated. :face_with_spiral_eyes:

+ My model is Multi LSTM model with multiple inputs.

#3

Hi @philip, If possible could you please provide the standalone code to reproduce the issue. Thank You.

#4

@Kiran_Sai_Ramineni Thanks for commenting!

The data has 12 features and the output is seven classes.
the model is a multi-input LSTM with 4 inputs of (batch size, 300, 3). 

x_test = pd.read_csv('.\\data\\x_test.csv')
y_test = pd.read_csv('.\\data\\y_test.csv')
#%%
def tflite_test(model_path, x_test, y_test):
    right_answer = 0

    interpreter_multi = tf.lite.Interpreter(model_path=model_path)

    interpreter_multi.allocate_tensors()

    input_details = interpreter_multi.get_input_details()
    output_details = interpreter_multi.get_output_details()
    
    output = []
    x_test = x_test.to_numpy(dtype=np.float32)
    x_test = x_test.reshape(-1,300,12)

    for i, input_data in enumerate(x_test):
        
        input_data = input_data.reshape(1, 300, 12)
        
        x1 = input_data[:,:,0:3]
        x2 = input_data[:,:,3:6]
        x3 = input_data[:,:,6:9]
        x4 = input_data[:,:,9:12]
        
        interpreter_multi.set_tensor(input_details[0]['index'], x1)
        interpreter_multi.set_tensor(input_details[1]['index'], x2)
        interpreter_multi.set_tensor(input_details[2]['index'], x3)
        interpreter_multi.set_tensor(input_details[3]['index'], x4)
        
        interpreter_multi.invoke()

        output_data = interpreter_multi.get_tensor(output_details[0]['index'])
        output.append(output_data)
        
    out = np.array(output)

    predict_label = []
    for i in out:
        predict_label.append(i.argmax())
    for index in range(len(predict_label)):
        if predict_label[index] == y_test.to_numpy()[index]:
            right_answer += 1

    accuracy = (right_answer / len(predict_label)) * 100
    print(f'accuracy : {accuracy:.3f}%')
#5

Hi @philip, If possible could you please provide the model that was converted to tflite and also sample data from the dataset you are using. so that i can reproduce and understand the issue better. Thank You.

#7

Hi @Kiran_Sai_Ramineni, sorry for the late reply.

if you possible, Send me your email address and I’ll share it with you on Google Drive.

Thank you.

#9

Hi @philip, You can make it public and share the link here so that i can access it. Thank You.

#11

Hi, @Kiran_Sai_Ramineni , I have some good news!
For this problem, I visualized the model structure, I’ve found the cause.
I noticed that the order of the inputs was reversed during the conversion, and once I fixed that, it worked fine.

but, Compared to other models (multi CNN etc…), LSTM is exceptionally slow, do you know why?