Using the benchmark tool to analyze tflite model run time and memory, found that when setting use_gpu=true, "initialized session" time is very long,while setting false the time is short.
so i want to know how to reduce Initialized session time when to use_gpu? thanks
lgusm
January 26, 2022, 5:17pm
#3
Do you have any code snippet of what you tried?
are you using the cmd line tool?
Bhack
January 26, 2022, 5:27pm
#4
I don’t know if it could be related to:
opened 12:13PM - 21 Apr 20 UTC
comp:lite
type:performance
TF 2.2
**System information**
- Have I written custom code (as opposed to using a stoc… k example script provided in TensorFlow):
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): android
- Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
- TensorFlow installed from (source or binary):
- TensorFlow version (use command below): 2.2 rc2
- Python version:
- Bazel version (if compiling from source):
- GCC/Compiler version (if compiling from source):
- CUDA/cuDNN version:
- GPU model and memory:
i compile tflite.a (2.2 rc2) from source and use ndk c++ api to run tflite model as follow:
`
#ifdef V2
TfLiteGpuDelegateOptionsV2 tOptions = TfLiteGpuDelegateOptionsV2Default();
if (m_bGPUAllowFP16)
{
tOptions.is_precision_loss_allowed = 1;
}
tOptions.inference_preference = 1;
m_pGPUDelegate = TfLiteGpuDelegateV2Create(&tOptions);
#else
TfLiteGpuDelegateOptions tOptions = {.metadata = nullptr, .compile_options = {.precision_loss_allowed = 0, .preferred_gl_object_type = TFLITE_GL_OBJECT_TYPE_FASTEST, .dynamic_batch_enabled = 0,},};
if (m_bGPUAllowFP16)
{
tOptions.compile_options.precision_loss_allowed = 1;
}
m_pGPUDelegate = TfLiteGpuDelegateCreate(&tOptions);
#endif
auto iRetCode = m_pInterp->ModifyGraphWithDelegate(m_pGPUDelegate);
if (iRetCode != kTfLiteOk)
{
return -1;
}
`
but the time cost is very different, v1 load time cost is only 10% of v2 load time. the model has Conv2DTranspose op, if use v1 api the inference time is 4x of v2 api, so why has this performance different?
yes,i uesd the cmd line tool like this “/benchmark_model --graph=fp16.tflite --num_threads=4 --use_gpu=true” the Initialized session in 940.822ms. however when i use the line “/benchmark_model --graph=car_fp16.tflite --num_threads=4 --use_gpu=false” the Initialized session in 91.076ms.
i want to know how to reduce Initialized session time when to use_gpu?
lgusm
January 27, 2022, 11:32am
#7
@Bhack 's comment has a very good discussion on the topic, I guess it can help you