Minifying tensorflow for deployment?

General Discussion
, ,
#1

I am trying to deploy a desktop application with Tensorflow (with Python and Briefcase), but the installed wheel has almost 900MiB on disk. Are there ways to make this lighter?

  1. I experimented erasing the entire /include subdirectory, which saved more than 100 MiB, apparently without ill effects. Will that explode in my face later on? I’m also considering deleting /lite , gaining another 15 MiB.

  2. More than half the space is occupied by a single file, _pywrap_tensorflow_internal.so, at 515 MiB. Is there any way of making this smaller (maybe recompiling Tensorflow from the source with custom options?)

  3. The only other “obvious” candidate is another binary, _pywrap_tfcompile.so, at 58 MiB. I removed it without immediate effects. Will that explode in my face later?

#3

Yes the TF library is still quite monolithic:

See
https://tensorflow-prod.ospodiscourse.com/t/llvm-updates-and-bazel-cache/2060

And this (also if it is a little bit in an unknown status):

In the meantime, if you need this just for inference and without python, you can explore:

Or

Then probably we will have some improvement in the future running a pretrained networks in the new TFRT runtime. But it is hard to understand the timing:

1 Like
#4

If you can rely on a client-server approach you can always remove the model from your desktop app and use:

Or