Transferring tensors and models between Web Workers?

Special Interest Groups Networking
,
#1

I am new to Tensorflow.js and may have horribly wrong assumptions, please correct me.

I am testing out tfjs in Deno, but I think this question applies to Web Workers in general.

I know that I can transfer buffers, i.e. io.ModelArtifact.weightData, to Web Workers back and forth.

Assuming the weight data, once allocated, lives inside my GPU. It should be way quicker if I can transfer only the handle instead of reading the model into CPU and then transfer it as a buffer.

Is it possible at the moment?

#3

You are correct - keeping everything on GPU would be faster than bringing data back to CPU for inspection. Keeping all on GPU is an active piece of research the team is looking into. Right now you must be somewhat of a GPU wizard to know how to do all that manually yourself but we are investigating ways to make that easier in the future.