Hi, I have converted my model for a computer vision application to be deployed onto a raspberry pi 4 4GB. I have converted it from PyTorch into a tflite format. Model works but its slower than running it in an onnx format via raspi.
Any way to improve the speed. I believe it’s supposed to be faster
onnx: 1 fps, 25 mb
tflite: 0.5 fps, 6mb
Hi @Mukesh_Narendran,
Is the tflite model is quantized? If not quantizing the tflite file please do it, as it improves the speed. Let us know your response.
Thank You
This is a common challenge when deploying machine learning models on resource-constrained devices like the Raspberry Pi.
Try uantize your model from FP32 to INT8. This can significantly improve inference speed without sacrificing too much accuracy. TensorFlow Lite offers built-in quantization tools to help you with this process.
Moreover, you can use TFLite’s delegate for ARM processors to leverage the NEON instructions for faster inference.
And if none of the above works, try to use the TensorFlow Lite Micro Benchmarker to identify bottlenecks in your model. Prune the model to remove unnecessary operations and reduce its size and consider using a lighter model architecture like MobileNet or EfficientNet.
Do let me know if that helped. @Mukesh_Narendran
Hi, Thanks for getting back. I stopped working on the project sometime back but I hope the discussion will be helpful to someone looking for a similar search
Hi, Thanks for your support. I am not working on it now but will try to keep the above messages as a future reference resource
@Mukesh_Narendran anytime. Feel free to get in touch if you think I can be of any help in the future.