After converting the model to .tflite and running it on Android, the accuracy drops

Hello everyone!

This is my first neural network, so there are often problems. And now there is a problem that I can’t solve.
My network produces a binary classification (patient is healthy, patient is sick). The input layer is fed 12 numeric values. I created and trained a neural network in Collab, it trained well and shows acceptable results on the validation sample (val_accuracy: 0.95
val_loss: 0.13), but after converting the model to .tflite and running it on a smartphone, it can’t predict anything.
I changed the number of layers, converted the model with tf.lite.TFLiteConverter.from_saved_model and tf.lite.TFLiteConverter.from_keras_model, viewed .tflite in Netron, tried to change the data input in Android, but nothing helped.

I think the problem is the wrong data transfer to the input layer of the tflite model in Android, but this is just a guess. And if so, please tell me how to fix the error?

This is my Colab code

raw_dataset = pd.read_csv('data.csv')
dataset = raw_dataset.copy()

train_dataset = dataset.sample(frac=0.8, random_state=0)
test_dataset = dataset.drop(train_dataset.index)

X = train_dataset.values
Y = test_dataset.values

Y = np.array(Y).astype("float32")

test_x, test_y = X[:,0:12], X[:,12]
train_x, train_y = Y[:,0:12], Y[:,12]

model = models.Sequential()
model.add(layers.Dense(64, activation = "tanh", input_dim=12))
model.add(layers.Dense(32, activation = "tanh"))
model.add(layers.Dense(16, activation = "tanh"))
model.add(layers.Dense(1, activation = "sigmoid"))
model.summary()

model.compile( optimizer = "adam", loss = "binary_crossentropy", metrics = ["accuracy"])

results = model.fit( train_x, train_y, epochs = 100, batch_size = 10, validation_data = (test_x, test_y))

# Save model
mobilenet_save_path = "my_SavedModel"
tf.saved_model.save(model, mobilenet_save_path)

# Convert the SavedModel
converter = tf.lite.TFLiteConverter.from_saved_model("my_SavedModel") # path to the SavedModel directory
tflite_model = converter.convert()

# Save the tflite_model.
with open('modelSavedModel.tflite', 'wb') as f:
  f.write(tflite_model)

This is my Java code in Android

ByteBuffer byteBuffer = ByteBuffer.allocateDirect(48);
                byteBuffer.putFloat(valueInt1);
                byteBuffer.putFloat(valueInt2);
                byteBuffer.putFloat(valueInt3);
                byteBuffer.putFloat(valueInt4);
                byteBuffer.putFloat(valueInt5);
                byteBuffer.putFloat(valueInt6);
                byteBuffer.putFloat(valueInt7);
                byteBuffer.putFloat(valueInt8);
                byteBuffer.putFloat(valueInt9);
                byteBuffer.putFloat(valueInt10);
                byteBuffer.putFloat(valueInt11);
                byteBuffer.putFloat(valueInt12);

                try {
                    ModelSavedModel model = ModelSavedModel.newInstance(context);

                    // Creates inputs for reference.
                    TensorBuffer inputFeature0 = TensorBuffer.createFixedSize(new int[]{1, 12}, DataType.FLOAT32);
                    inputFeature0.loadBuffer(byteBuffer);

                    // Runs model inference and gets result.
                    ModelSavedModel.Outputs outputs = model.process(inputFeature0);
                    TensorBuffer outputFeature0 = outputs.getOutputFeature0AsTensorBuffer();

                    // Releases model resources if no longer used.
                    model.close();
                    float preResult = outputFeature0.getFloatArray()[0]*100;
                    int result = (int) preResult;
                    System.out.println(preResult);
                } catch (IOException e) {
                    // TODO Handle the exception
                }
1 Like

Hello @Van_Gil Prior to deploying the model, have you considered the following by any chance to improve generalization?

Or, perhaps, the issue here is with the Java code.

A few more sources:

1 Like

Have you found any solutions? I have the same problem: python - Poor tensorflow-lite accuracy in Android application - Stack Overflow – my model works in python but performs poorly in Android app. 1/3 of predictions are wrong, while python version predicts 100% correctly.

BTW, I wrote a script to load .tflite model in python and it works well, too, so the problem is not in .tflite file. The only problem is Android part and I have no idea how to solve it…

Have you by now have found any solutions? I am dealing with the same kind of problem…

Hi @Janneke_van_Hulten @burbilog @Van_Gil , thanks for the questions. Checking with the team :+1:

Do you use the same test set in Android (with Java) as with the Python code you used to train the model before converting to .tflite? This is to make sure we’re having an apples to apples comparison.

Looping in @xhae :+1:

In addition, as per @markdaoust 's suggestion, check out:

TensorFlow inference APIs support the signature-based executions:

  • Accessing the input/output tensors through the names of the inputs and outputs, specified by the signature.
  • Running each entry point of the graph separately, identified by the signature key.
  • Support for the SavedModel’s initialization procedure.

Java, C++ and Python language bindings are currently available.

Java:

try (Interpreter interpreter = new Interpreter(file_of_tensorflowlite_model)) {
  // Run encoding signature.
  Map<String, Object> inputs = new HashMap<>();
  inputs.put("x", input);
  Map<String, Object> outputs = new HashMap<>();
  outputs.put("encoded_result", encoded_result);
  interpreter.runSignature(inputs, outputs, "encode");

  // Run decoding signature.
  Map<String, Object> inputs = new HashMap<>();
  inputs.put("x", encoded_result);
  Map<String, Object> outputs = new HashMap<>();
  outputs.put("decoded_result", decoded_result);
  interpreter.runSignature(inputs, outputs, "decode");
}
1 Like

Looping in @yyoon and @Thai_Nguyen :+1:

I’m not currently, because in my application i make use of real-time video input of poses, which is first translated into 33(x3) keypoints which i then feed to my model. I will try to feed it my test data first, that’s a very good idea!

Tried this, and the test data which gives a correct result in colab when testing it on my keras model, gives incorrect classification result in java. Weird!

Hi @Janneke_van_Hulten

To help you more upload Colab notebook and android app to build, test and debug.

Best

Trying to upload everything to git to be able to share it easily, not working yet as my project is too big. Here the keras model:


# Define the model
inputs = tf.keras.Input(shape=(99))
# embedding = landmarks_to_embedding(inputs)

layer = keras.layers.Dense(128, activation=tf.nn.relu6)(inputs)
layer = keras.layers.Dropout(0.5)(layer)
layer = keras.layers.Dense(64, activation=tf.nn.relu6)(layer)
layer = keras.layers.Dropout(0.5)(layer)
outputs = keras.layers.Dense(5, activation="softmax")(layer)

model = keras.Model(inputs, outputs)
model.summary()
import pandas as pd

model.compile(
    optimizer='adam',
    loss='categorical_crossentropy',
    metrics=['accuracy']
)

# Add a checkpoint callback to store the checkpoint that has the highest
# validation accuracy.
checkpoint_path = "weights.best.hdf5"
checkpoint = keras.callbacks.ModelCheckpoint(checkpoint_path,
                             monitor='val_accuracy',
                             verbose=1,
                             save_best_only=True,
                             mode='max')
earlystopping = keras.callbacks.EarlyStopping(monitor='val_accuracy', 
                                              patience=20)


pose_embedder = FullBodyPoseEmbedder()
# # Calling DataFrame constructor on list
# df = pd.DataFrame(X_train)
# # print(df.shape)
# df.astype('float64')
# print(df.shape)
booly = 0
landmarks = X_train.to_numpy()
validation = X_val.to_numpy()
val_landmarks = []
norm_landmarks = []
for x in landmarks:
  if(booly==0):
    print(x)
  y = np.reshape(x,[-1,3])
  # print(y.shape)
  embedding = pose_embedder(y)
  if(booly==0):
    print(embedding)
    booly=1
  # print(embedding)
  norm_landmarks.append(embedding)

for z in validation:
  p = np.reshape(z,[-1,3])
  # print(p.shape)
  embedding = pose_embedder(p)
  # print(embedding)
  val_landmarks.append(embedding)

# embedding = landmarks_to_embedding(X_train)
# embedding_val = landmarks_to_embedding(X_val)
norm_landmarks = np.array(norm_landmarks)
norm_landmarks = np.reshape(norm_landmarks, [-1,99])
val_landmarks = np.array(val_landmarks)
val_landmarks = np.reshape(val_landmarks, [-1,99])
print(norm_landmarks)
print(y_train)
# Start training
# estimator = KerasClassifier(build_fn=baseline_model, epochs=200, batch_size=5, verbose=0)
# history = model.fit(norm_landmarks, y_train, epochs = 200, batch_size=5, verbose=0, validation_data = (val_landmarks, y_val))
history = model.fit(norm_landmarks, y_train,
                    epochs=200,
                    batch_size=16,
                    shuffle = True,
                    validation_data=(val_landmarks, y_val),
                    callbacks=[checkpoint, earlystopping],
                    verbose = 2)

Used ‘new → other → tensorflow lite model’ to add it to androidStudio

Using this code to run inference:


            Jupyter1 dlClassifier = Jupyter1.newInstance(context);
            // Creates inputs for reference.
            TensorBuffer inputFeature0 = TensorBuffer.createFixedSize(new int[]{1, 99}, DataType.FLOAT32);
            List<PointF3D> landmarks = extractPoseLandmarks(pose);
            System.out.println(landmarks);
            if (landmarks.isEmpty()) {
                return getStringFromResult(history.getResult());
            }
            List<PointF3D> embedding = getPoseEmbedding(landmarks);
            ByteBuffer buf = ByteBuffer.allocateDirect(99 * 4);
            boolean justOnce = true;

            for (PointF3D landmark : embedding) {
                if (justOnce) {
                    System.out.println("landmark X: " + landmark.getX() + " landmark Y: " + landmark.getY() + " landmark Z: " + landmark.getZ());
                }
                buf.putFloat(landmark.getX());
                buf.putFloat(landmark.getY());
                buf.putFloat(landmark.getZ());
            }
            buf.rewind();
            inputFeature0.loadBuffer(buf);

            // Runs model inference and gets result.
            Jupyter1.Outputs outputs = dlClassifier.process(inputFeature0);
            TensorBuffer outputFeature0 = outputs.getOutputFeature0AsTensorBuffer();
            float[] data = outputFeature0.getFloatArray();
            System.out.println("output: " + Arrays.toString(data));

When i have everything on git, i’ll share the complete code!

Oh, btw: this is the first time ever I’m trying anything like this, so it will most probably not be perfect :wink:

runSignature doesn’t seem to be a method of the Interpreter class anymore, unfortunately

Is your training/test set data also collected from the same environment? (i.e. it’s your dataset and not an off-the shelf one)

I’m not 100% sure what you mean. I got a dataset from kaggle, consisting of 2 folders (train&test) with each 5 folders for each yoga pose, filled with images of those poses. I translated all these images in the same way, creating 2 csv files (train&test) filled with 99 datapoints and a number (0-4) indicating which pose it is. I used the first csv file to train my model, the second one to test it (gives around 96% accuracy). The second csv file (test) i also uploaded to android studio, to test the classifier there on the exact same data. There it gives completely different (and shitty) results. But yeah, I guess it’s all from the same environment?

please verify sizeof input tensors - itseems like TFliteconvertor from keras generates wrong default shapes there

val predictionName = tflite?.signatureKeys?.first()
        logMsg("Running inference '$modelFileName' predict:$predictionName()")

        val inTensorNames = tflite!!.getSignatureInputs(predictionName)
        (0 until tflite!!.inputTensorCount).forEach { inputIndex ->
            tflite!!.getInputTensor(inputIndex).apply { //inputTensor->
                logMsg("Input[$inputIndex]=(${dataType()}) [${
                    shape().indices.map { shape()[it] }.joinToString(",")
                }] [${
                    shapeSignature().indices.map { shapeSignature()[it] }.joinToString(",")
                }] ${numElements()} ${quantizationParams().scale} ${quantizationParams().zeroPoint} ${name()} ${inTensorNames[inputIndex]}")
            }
        }
        val outTensorNames = tflite!!.getSignatureOutputs(predictionName)
        (0 until tflite!!.outputTensorCount).forEach { outputIndex ->
            tflite!!.getOutputTensor(outputIndex).apply { //outputTensor->
                logMsg("Output[$outputIndex]=(${dataType()}) [${
                    shape().indices.map { shape()[it] }.joinToString(",")
                }] [${
                    shapeSignature().indices.map { shapeSignature()[it] }.joinToString(",")
                }] ${numElements()} ${quantizationParams().scale} ${quantizationParams().zeroPoint} ${name()} ${outTensorNames[outputIndex]}")
            }
        }

@Van_Gil @Janneke_van_Hulten Hello
I am facing same issues, in python environment the model works fine. But the same tflite fails to predict correct output in c environment.

@Janneke_van_Hulten and @grtsahil ,
I am facing the same issue with my tflite classification model.
I used pretrained EfficientNetB0 model from Tensorflow.keras.applications
i trained the model on my custom dataset.
for test dataset it gives 88% accuracy but when i converted that model into .tflite format model only gave 26% accuracy.

does anyone found solution?