With the new tensorflow-directml-plugin project that’s come out, I’m back on the hunt for a method to accelerate tf2.x with non-CUDA gpu’s. I was wondering if it’s possible to build the c api / libtensorflow package (specifically tensorflow.dll) for windows using/including this plugin? thanks for any help or ideas!
I really want to know about that.
Hey @wl9300,
This is not a scenario that we explicitly support yet, but it should be possible to do by using the DLLs from the pypi package. After installing tensorflow-directml-plugin and tensorflow-cpu, you will need site-packages/tensorflow-plugins/tfdml_plugin.dll and site-packages/tensorflow-plugins/directml/DirectML.<SHA>.dll. Since tfdml_plugin.dll also depends on _pywrap_tensorflow_internal.pyd, you’ll also need site-packages/tensorflow/python/_pywrap_tensorflow_internal.pyd.
You should then be able to call TF_LoadLibrary("tfdml_plugin", status); to register the DML device and the kernels.
Remember that this is not something that we explicitly support or tested thoroughly, so it may be flaky. This is something that we might want to support more officially in the future though through a c_api package, like we did for TF 1.15.
Let me know if you have any questions or problems setting it up!
My use case right now is that i have a precompiled application (*.exe, not owned by me) that uses tensorflow.dll. I’m fairly new to and unfamiliar with tensorflow, so apologies if it seems like I’m asking dumb questions. If I understand correctly, one would call the TF_LoadLibrary() you mentioned from within a C++ program? If so, I don’t think that would be possible in my use case. What I currently have is a folder containing an application and tensorflow.dll, which was originally distributed as part of the c_api package. I am hoping to be able to compile a new tensorflow.dll to call the plugin the way the tf-dml c_api or cuda-enabled c_api would, and this is what I was wondering would be possible or not.
If this is not possible, I will see if I can contact the developer of the application and see about altering the application’s source code to call TF_LoadLibrary() or equivalent. Seeing that I’m pretty new to tf and programs using tf, could you clarify for me how to set everything up? I have both dll’s and the pyd file you described, but I’m not sure what to do with them once I have them?
Thanks for your help and sorry for the troubles! -wl
Modifying the application will definitely be necessary since the TensorFlow C API doesn’t know how to automatically load plugins. Furthermore, the workaround I outlined earlier may not work as easily as I thought, as you would also need to link to a punch of python dependencies just to be able to load tfdml_plugin.dll.
We will investigate the best way to provide a C API package for tensorflow-directml-plugin internally and we’ll keep you posted!
Thank you. I look forward to the c_api package!
Also, would you mind telling us which application you are trying to integrate tensorflow-directml with?
that would be starnet, a neural net star removal tool for astrophotography (https://starnetastro.com/) and its pixinsight integration (https://pixinsight.com)
1 Like
Hey @wl9300, short update on the progress for this feature.
We got it working on WSL (Windows Subsystem for Linux), but it turns out that the TensorFlow Windows C API doesn’t export some symbols that are necessary for pluggable devices. Unfortunately the TF 2.10 branch has already been cut so it’s too late to propose a change to add those symbols, but I’ll be working with the TensorFlow team to possibly include it in TF 2.11.
Thanks for the update!
This would be really helpful. I make two ML-based products, StarXTerminator and NoiseXTerminator, for processing astronomical images (www.rc-astro.com), with builds for both PixInsight and Photoshop. Users who have CUDA-capable GPUs are in luck w.r.t. acceleration, but many users have other models or just integrated graphics. Any acceleration is better than no acceleration. 
2 Likes
Hi @PatriceVignola,
now that TF 2.11 is officially out, any updates regarding being able to use the tfdml_plugin.dll with the C/C++ API (managed to successfully build the TF2.11 C++ API from source on windows in CPU-only mode)?
I have copied _pywrap_tensorflow_internal.pyd to be next to tfdml_plugin.dll (with ./DirectML folder containing the DirectML..dll).
called
TF_Library* lib = TF_LoadLibrary(“tfdml_plugin”, status);
with status OK (DLL seems to be loaded).
however, it is not registered. only CPU recognized.
is there something I am missing here (e.g. call to registration function)?
your help is greatly appreciated
We had this feature working internally, but unfortunately the latest keras doesn’t work with TF 2.11+pluggable devices together, so we have to keep the plugin hard locked against TF==2.10 for the time being. We hope it will get resolved by the time keras 2.12 rolls out. Even though Keras support isn’t related to the C API, we want to keep the releases in sync since they use common code internally.
Hi @PatriceVignola,
I’m a bit new to using the C API on Linux, but I’d like to try using the program with WSL. I’m not quite sure how to get started: the program comes bundled with libtensorflow.so.2 and libtensorflow_framework.so.2 . How should I go about installing/moving around the c_api so that I can get the program to talk to tensorflow to talk to the plugin? I tried moving files around as one might with the original tf-dml, but evidently this one doesn’t come packed with tensorflow.so . I tried installing libtensorflow-cpu-linux-2.10.0 in the folder as I would have with the original tf-dml, but I’m assuming it doesn’t really work in the same way. Thanks for your help!
Here you can find an example for how to load a .pb model and run it through the C API: tensorflow-directml-plugin/c_api_tests.cc at main · microsoft/tensorflow-directml-plugin · GitHub
To make it work, you need to download the TFDML C API and the TensorFlow 2 C API and add the include directory to your include path. The CMakeLists.txt file in the TFDML repository shows how to do it using cmake.
Is there a link to a pre build Windows binary for the plugin?