Input tensor 0 is missing TensorMetadata

hissain · November 27, 2023, 4:34am

This article shows how to add meta_data into a tflite model before running in Android. However, it only shows example for Image classification. I am trying to add meta data for Audio classification. How can I get guide.

I am getting error in Android when running a model.tflite which has no meta data.

Error getting native address of native library: task_audio_jni
java.lang.IllegalArgumentException: Error occurred when initializing AudioClassifier: input tensor 0 is missing TensorMetadata.
at org.tensorflow.lite.task.audio.classifier.AudioClassifier.initJniWithModelFdAndOptions(Native Method)
at org.tensorflow.lite.task.audio.classifier.AudioClassifier.access$000(AudioClassifier.java:74)
at org.tensorflow.lite.task.audio.classifier.AudioClassifier$1.createHandle(AudioClassifier.java:143)
at org.tensorflow.lite.task.audio.classifier.AudioClassifier$1.createHandle(AudioClassifier.java:136)
at org.tensorflow.lite.task.core.TaskJniUtils$1.createHandle(TaskJniUtils.java:70)
at org.tensorflow.lite.task.core.TaskJniUtils.createHandleFromLibrary(TaskJniUtils.java:91)
at org.tensorflow.lite.task.core.TaskJniUtils.createHandleFromFdAndOptions(TaskJniUtils.java:66)
at org.tensorflow.lite.task.audio.classifier.AudioClassifier.createFromFileAndOptions(AudioClassifier.java:134)
at org.tensorflow.lite.task.audio.classifier.AudioClassifier.createFromFile(AudioClassifier.java:91)
at com.samsung.classification.MainActivity.startAudioClassification(MainActivity.java:86)
at com.samsung.classification.MainActivity.lambda$showActivitySelectionDialogue$0$com-samsung-classification-MainActivity(MainActivity.java:51)
at com.samsung.classification.MainActivity$ExternalSyntheticLambda0.onClick(Unknown Source:2)
at com.android.internal.app.AlertController$AlertParams$3.onItemClick(AlertController.java:1485)
at android.widget.AdapterView.performItemClick(AdapterView.java:376)
at android.widget.AbsListView.performItemClick(AbsListView.java:1295)
at android.widget.AbsListView$PerformClick.run(AbsListView.java:3571)
at android.widget.AbsListView$3.run(AbsListView.java:4751)
at android.os.Handler.handleCallback(Handler.java:942)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loopOnce(Looper.java:226)
at android.os.Looper.loop(Looper.java:313)

George_Soloupis · November 27, 2023, 2:12pm

Hi @hissain

Is your python code working ok before trying to convert it to .tflite? What example have you used to train the model? Have you used TF Model Maker or a different approach?
The code for inserting the metadata is the same as the one you have provided. Can you share more info about the normalization and the other info for your model that must be added?

George_Soloupis · November 27, 2023, 2:13pm

Check here some examples with the TF Lite Model Maker that adds the metadata automatically

George_Soloupis · November 27, 2023, 2:24pm

Here is the example how TF Lite Model Maker is adding the parameters inside the model.

github.com

tensorflow/examples/blob/master/tensorflow_examples/lite/model_maker/core/task/model_spec/audio_spec.py

# Copyright 2020 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Audio model specification."""

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import abc

This file has been truncated. show original

You can se also which are the parameters based on if it is YamnetSpec or BrowserFftSpec

hissain · November 28, 2023, 6:29am

It was working OK in the same Notebook as I have found evaluation score at around 90% accuracy. I was trying to install tflite_model_maker however, it is failing to install because of dependency conflicts with tflite_support. Is there any guideline on it?

Please refer to the issue I am facing. python - ERROR: Cannot install tflite-model-maker (The conflict is caused by other modules) - Stack Overflow
@George_Soloupis

George_Soloupis · November 28, 2023, 9:14am

It is an known issue. Please refer to Future of tflite-model-maker and mediapipe-model-maker or take a look a recent example I wrote
Use TensorFlow Lite Model Maker with a custom dataset | by George Soloupis | Nov, 2023 | Medium
Inside there you can find a python notebook with example of using a conda env inside Colad.

Regards

hissain · November 29, 2023, 1:51pm

@George_Soloupis your blog provides examples for unix systems. I was trying to run same but tflite model maker installing for indefinite time and storage gets full. Any other suggestions?

George_Soloupis · November 29, 2023, 2:36pm

@hissain The example is working OK inside a Colab session.
No other suggestions I can think of.

hissain · November 29, 2023, 2:53pm

@George_Soloupis can you share any working notebook? I have same issue in colab notebook as well.

George_Soloupis · November 29, 2023, 3:20pm

Inside the blog post there is a link to a notebook…just before the code snippets

hissain · November 29, 2023, 5:17pm

@George_Soloupis could you please help from where can I get the /gdrive/MyDrive/JWick/gun_shot_wav.zip file? I was trying to run the colab and it failed to unzip because I have no zip file in my drive. Here is my notepad copied from your git.

George_Soloupis · November 30, 2023, 4:01am

All you need to use is inside the blog post!