I’m wondering if, as phones like Pixel 6 become common, whether TFJS in a browser will run much faster than TFJS currently does on modern phones. If the answer is yes are there any benchmarks? If not, is it because the TPU isn’t compatible with a GPU? Or is it?
I don’t think that currently we have a W3C standard to access to TPU.
Also the WIP webnn, in the current status, It is still limited to CPU/GPU devices:
https://www.w3.org/TR/webnn/#programming-model-device-selection
For Coral TPU that is a quite different use case see:
https://tensorflow-prod.ospodiscourse.com/t/tfjs-and-coral-tpu/5137/3
2 Likes
So I actually have just bought the P6 Pro and while some demos do run faster this is not due to any extra access to the chip itself. Essentially JS on the client side at least is dependent on what the browser can expose etc. Right now that is mostly Web Assembly for CPUs and WebGL for GPUs. In the future there are emerging web standards like WebGPU that may offer even better results. Certainly as far as I know right now you can not access TPU from browser without WebUSB and doing a whole lot of jumps flashing new images to the TPU etc once detected before being able to use it. Also TPU does not support TFJS format, we could only push TFLite models there for now I believe.
2 Likes