Instantiating new variables using a tf.function

General Discussion
#1

Hi!
I’m quite new to TensorFlow; as a first experiment, I’m trying to implement a python script that manipulates tensors. My goal is to compare the performances using TensorFlow vs using NumPy.
To keep the code most general (thus switching between TensorFlow and NumPy by modifying a flag only), I’m using a flag to implement the same function either using TensorFlow or using NumPy; e.g.:

if use_tf:
  @tf.function
   def maximum(X,Y):
      return(tf.maximum(X,Y))

else:
   def maximum(X,Y):
      return(np.maximum(X,Y))

using the same paradigm, I’m trying to do the same for tf.Variable, as follows:

if use_tf:
  @tf.function
   def Variable(X,name=None):
      return(tf.Variable(X,name=name))
else:
  def Variable(X,name=None):
    return(X)

thus, when use_tf=False this simply returns the input argument, while when using TensorFlow (use_tf=True) this initialises a TensorFlow variable (where X is a NumPy array)

however, when I call the new function (use_tf=True):

U=Variable(np.zeros(shape=(3,3),dtype=np.float32),name="U"

I obtained:

 /home/cc14/Codes/anaconda3/envs/tf-cpu/lib/python3.9/site-packages/tensorflow/python/eager/def_function.py:1007 fn_with_cond  *
        functools.partial(
    <ipython-input-3-9b20263024b5>:3 Variable  *
        return(tf.Variable(X,name=name))
    /home/cc14/Codes/anaconda3/envs/tf-cpu/lib/python3.9/site-packages/tensorflow/python/ops/variables.py:268 __call__  **
        return cls._variable_v2_call(*args, **kwargs)
    /home/cc14/Codes/anaconda3/envs/tf-cpu/lib/python3.9/site-packages/tensorflow/python/ops/variables.py:250 _variable_v2_call
        return previous_getter(
    /home/cc14/Codes/anaconda3/envs/tf-cpu/lib/python3.9/site-packages/tensorflow/python/ops/variables.py:67 getter
        return captured_getter(captured_previous, **kwargs)
    /home/cc14/Codes/anaconda3/envs/tf-cpu/lib/python3.9/site-packages/tensorflow/python/eager/def_function.py:764 invalid_creator_scope
        raise ValueError(

    ValueError: tf.function-decorated function tried to create variables on non-first call.

is there a way to create a function that, given a NumPy array, creates and returns a TensorFlow variable?

Thanks for the help.

Cesare

#3

https://www.tensorflow.org/api_docs/python/tf/function?version=nightly#variables_may_only_be_created_once

#4

Thanks.
I re-defined the function as a class and now it works.
I copy here an example code in case anyone will encounter the same problem. The flag
use_tf is used to choose between TensorFlow and NumPy.
I did not define any class destructor, however: do I need one? Or does TensorFlow have garbage?

import numpy as np

use_tf = True

if use_tf:

  import tensorflow as tf  
  
  class Variable(tf.Module):
    def __init__(self,X,name=None):
      self.X = None
      if self.X is None:
        self.X = tf.Variable(X,name=name)

    @tf.function
    def __call__(self):
      return(self.X)
  

else:

  class Variable():
    def __init__(self,X,name=None):
      self.X = None
      if self.X is None:
        self.X = np.array(X)

    def __call__(self):
      return(self.X)

  
  
  
if __name__=='__main__':
  Y = Variable(np.zeros(shape=(3,3),dtype=np.float32),name="Y")()
  X = Variable(np.random.normal(size=(3,3)).astype(np.float32),name="X")()
  print("X:")
  print(X)    
  print("X type:")
  print(type(X))    
  
  print("Y:")
  print(Y)  
  print("Y type:")
  print(type(Y))    

  
  Z=X+Y
  print("Z=X+Y")
  print(Z)
  print("Z type:")
  print(type(Z))

EDIT: the operation Y = Variable(np.zeros(shape=(3,3),dtype=np.float32),name=“Y”)() returns a tensor rather than a tf.Variable. Is there a way to force to yield a variable?

#6

I wrote a 3 parts article about the correct usage of tf.function.

part 1 contains the explaination of the behavior and let you go trough some examples for understanding it deeply.

part 2 and part 3 go even deeper and explain to you what happens at the various operations when you use tf.function.

Overall the 3 articles should tech you how to write idiomatic code using Tensorflow.

I hope it helps

1 Like
#7

EDIT: the operation Y = Variable(np.zeros(shape=(3,3),dtype=np.float32),name=“Y”)() returns a tensor rather than a tf.Variable. Is there a way to force to yield a variable?

You need to change it a do something like: