Does anyone know how far along formal efforts are to utilise tfjs code directly on the TPU?
This was the most updated status:
1 Like
See also:
1 Like
Thank you for the GH links, Bhack. Unfortunately, I’ve seen them both before and they don’t help. The second one is a great idea, but refers specifically to the USB accelerator. My device is the mPCIe TPU. The first one gave me hope though, which is why I’m asking here; that I might get some direct feedback.
Ok, let’s see if the Coral team can disclose something more about this.
AFAIK PCIe is not accessible from JavaScript in the browser at least. Maybe some options in Node.js if someone has ported something to run there, but from the browser I am unware of any efforts to support that as an official standard at least. Only WebUSB which is what the above github repo demonstrates where by you can take a TFLite model, convert to the TPU Edge format, and then run via browser to USB Coral device, and do inference on arbitrary images etc.
Also right now WebUSB as far as I am aware is only supported on Chrome (and browsers built upon that stack) and due to then nature of the Coral USB TPU will require you to push firmware to the device when inserted (only need to do once while USB is inserted, but would need to redo if you unplug or power off), to access the chip that does the processing as detailed in the Github which means the user will have to allow access to 2 USB devices via the standard WebUSB popups.
My gut instinct right now is that Node.js may be easier to communicate with the Coral device right now as I think I saw some libs for Node that replicate WebUSB too and support USB Reset etc too. This would probably work well on devices like Raspberry Pi etc which are the sorts of platforms that need this acceleration anyhow as most modern laptops etc will run very fast with standard TensorFlow.js inference via WebAssembly or WebGL in browser
Jason, thanks for your feedback. Much appreciated. I too believe Node.js to be the best way forward here, and to that end have already coded with tfjs-node. The problem is that it’s using the CPU and not the TPU.
Would something like this enable you to port the work in the github repo above to work with node for Coral:
?
This are some discussions internally on how to support Coral within TFJS. I think the best solution forward is to compile TFLite + Coral runtime + libusb natively for RaspberryPI, and provide the node (NAPI buiding) API similar to tfjs-node for TensorFlow (CPU and GPU).
It can use the same API as TFJS-TFLite library but everything runs natively.
2 Likes