Tensorflow lite inferencing is too slow on risc-v platform

General Discussion
, , ,
#1

Hi, do any one use tensorflow lite on risc-v platform?
I use both python tflite_runtime and c++ tensorflow lite environment on risc-v platform to do image detecting, however it shows that the performance is really bad.
as the table shows below, I use default param to do benchmark(tensorflow-r2.5\tensorflow\lite\tools\benchmark)

image
image1194×313 59.6 KB

in order to find whether software matters more to the result, I also compare the coremark scores on U540 and Raspberry 3B， it turns that coremark score gap is not that big as inference benchmark time(both cpu freq 1GHz).

image
image1122×265 47.7 KB

> Can the above test prove "tensorflow lite is bad fitted on risc-v platform " ?
**> **
> Or other low-level software support like math lib have not as good performance as ARM platform done?

#3

What risc-v board do you have?

ARM has neon delegate:

In the new MLIR compiler stack they are working on:

#4

I use sifive U740（RISC-V）

#6

You can probably upvote, comment and subscribe:

1 Like
#7

A recent update on the compiler stack:

#9

ARM has neon implementation in TFLite to accelerate inference.
SiFive also has RVV implementation for TFLite, but not open-sourced yet.