I want to replica some nodes on different devices in Grappler optimization pass. In this situation, there are some cases when I need to share the Variable node on different devices.
Let’s say I have a node
GPU :0 , it’s type is an _Arg node. I want to replicate the same node on
GPU:1, let’s call it
x1. How could I share this variable on graph rewrite pass? If I directly replicate the
x node and keep everything but the name the same, it will get an error of something like variable is not initialized.
In addition, tf1 could get a graph that how variable is assigned , it have a graph with
Variable/Assign node, but tf2 seems delete this, how could I get how variable is initialized and passed to device?