Android GPU delegate gives same output from intermediate layers outputand

Hi Team

I have trained a mobilnetv2 model using the tfod API. Along with the normal detector output (bbox, score,num_detection, confidences), I also wanted an intermediate layers output, so while converting the model to tflite i specified the intemediate node from where i needed the output (‘FeatureExtractor/MobilenetV2/expanded_conv_9/add’) as illustrated below :

tflite_convert --graph_def_file=$1/tflite_graph/tflite_graph.pb --output_file=$1/tflite_graph/detect.tflite --input_shapes=1,300,300,3 --input_arrays=normalized_input_image_tensor --output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3','FeatureExtractor/MobilenetV2/expanded_conv_9/add' --inference_type=FLOAT --allow_custom_ops

When I run the tflite model on the android with the XNNPack or CPU, I am able to get the different output of the intermediate layer which is coming at index 4
code snippet :

try {
      Interpreter.Options options = new Interpreter.Options();
      options.setNumThreads(NUM_THREADS);
      options.setUseXNNPACK(true);
      d.tfLite = new Interpreter(modelFile, options);
      // 4th index for intermediate output
      int[] shape = d.tfLite.getOutputTensor(4).shape();
      d.tfLiteOptions = options;
    } catch (Exception e) {
      throw new RuntimeException(e);
    }

But when I run the same model on GPU delegate, I am getting the same output for every frame I am passing. The other output like (bbox, score,num_detection, confidences) changes but the intermediate output remains the same

try {
      Interpreter.Options options = new Interpreter.Options();
      CompatibilityList compatList = new CompatibilityList();
      GpuDelegate.Options delegateOptions = compatList.getBestOptionsForThisDevice();
      GpuDelegate gpuDelegate = new GpuDelegate(delegateOptions);
      options.addDelegate(gpuDelegate);

      d.tfLite = new Interpreter(modelFile, options);
      // 4th index for intermediate output
      int[] shape = d.tfLite.getOutputTensor(4).shape();
      d.tfLiteOptions = options;
    } catch (Exception e) {
      throw new RuntimeException(e);
    }

System information

  • Mobile device: OnePlus 6T, Mi Note5 pro
  • TFOD Api TensorFlow version : 1.15
  • Android Tensorflow Lite version: 2.4.0

Need help to figure out what may be going wrong
more info on this github issue link : https://github.com/tensorflow/tensorflow/issues/51800