I am trying to deploy my model using tflite on Android, but their speed is a bit slow and seems to be running on the CPU. What should I do

[jun_yin1]

I am currently trying to port the YOLOv8 pose model to my phone. I am trying to use tflite to speed up the process. When I use the HexagonDelegate agent, both my 870 device and 8gen1 device prompt that my device does not have DSP acceleration function. When I choose GpuDelegate, its effect is good, but it does not meet expectations at all. It feels like not all is working. When I use NnApiDelegate, whether it is rk3588, Both Snapdragon 870 and Snapdragon 8gen1 devices have prompted me that the device does not support NnApiDelegate. If there is Dalao, could you help me guide me， I would greatly appreciate it

[Kiran_Sai_Ramineni]

@jun_yin1

I am currently trying to port the YOLOv8 pose model to my phone

Hi @jun_yin1, could you provide the details of the android version of the mobile?

@jun_yin1

When I use the HexagonDelegate agent, both my 870 device and 8gen1 device prompt that my device does not have DSP acceleration function

Because the Hexagon delegate can be used to accelerate models on Android devices with Qualcomm Hexagon DSP only.

@jun_yin1

When I use NnApiDelegate, whether it is rk3588, Both Snapdragon 870 and Snapdragon 8gen1 devices have prompted me that the device does not support NnApiDelegate

The NNAPI delegate can be used to accelerate models on Android devices with GPU, DSP and / or NPU available. It is available in Android 8.1 (API 27+) or higher.

@jun_yin1

When I choose GpuDelegate, its effect is good, but it does not meet expectations at all.

This may be due to only some ops are supported by GPUs. If some of the ops are not supported by the GPU delegate, the framework will run those ops on the CPU. Due to the high cost of CPU/GPU synchronization, a split execution mode like this often results in slower performance than when the whole network is run on the CPU alone.

Thank You.