What can you use within tf.data.Dataset map functions/on symbolic tensors?

General Discussion
,
#1

The following bit_pack_arg function can be applied as a map function on a tf.data.Dataset:

def bit_pack_arg(*data):
packed_data = []
current_byte = 0
bit_count = 0

for bit in data[:-1]:
    #current_byte |= (bit << bit_count)
    current_byte |= tf.bitwise.left_shift(bit, bit_count)
    bit_count += 1

    if bit_count == 8:
        packed_data.append(current_byte)
        current_byte = 0
        bit_count = 0
    
if bit_count > 0:
    packed_data.append(current_byte)

#return np.array(packed_data, dtype=np.uint8)
return(packed_data)

I first got the error:

TypeError: unsupported operand type(s) for <<: ‘Tensor’ and ‘int’

on the ‘<<’ operator. After replacing the ‘<<’ operator with tf.left_shift I got the error:

NotImplementedError: Cannot convert a symbolic tf.Tensor (or_7/BitwiseOr:0) to a numpy array. This error may indicate that you’re trying to pass a Tensor to a NumPy call, which is not supported.

I uncommented out the numpy.array call and when bit_pack_arg is mapped to the dataset I now get the output:

[(0,
0,
48,
216,
47,
16,
0,
0,
0,
0,
0,
0,
52,
74,
164,
0,
0,
0,
0,
0,
0,
0,
0,
0),

Close but it is not a nested array of numpy.arrays yet. So I guess what happens is that the dataset creates a symbolic tensor, the ‘+’ operator works on symbolic tensors but ‘<<’ and numpy.array not. So, are these bugs or are there rules for what you can use within tf.data.Dataset map functions? The np.array call converts the packed bytes to np.uint8. How can I convert the packed_data to uint8 without the np.array call?

Thanks,
GW

#3

ChatGPT 3.5 at work:

Q: Thank you for this example! More generally, could you tell me what operators can be applied to symbolic tensors?

A: Symbolic tensors in TensorFlow represent computations rather than concrete values. As such, you can apply various TensorFlow operations (also known as ops) to symbolic tensors. These operations include, but are not limited to:

  1. Arithmetic Operations:
  • Addition: tf.add()
  • Subtraction: tf.subtract()
  • Multiplication: tf.multiply()
  • Division: tf.divide()
  1. Mathematical Functions:
  • Exponentiation: tf.pow()
  • Square root: tf.sqrt()
  • Trigonometric functions: tf.sin(), tf.cos(), tf.tan(), etc.
  • Element-wise absolute value: tf.abs()
  1. Reduction Operations:
  • Summation: tf.reduce_sum()
  • Mean: tf.reduce_mean()
  • Minimum and Maximum: tf.reduce_min(), tf.reduce_max()
  • Variance: tf.math.reduce_variance()
  1. Logical Operations:
  • Logical AND, OR, NOT: tf.logical_and(), tf.logical_or(), tf.logical_not()
  • Comparison operators: tf.equal(), tf.not_equal(), tf.less(), tf.greater(), etc.
  1. Bitwise Operations (for integer tensors):
  • Bitwise AND, OR, XOR: tf.bitwise.bitwise_and(), tf.bitwise.bitwise_or(), tf.bitwise.bitwise_xor()
  • Left and right shifts: tf.bitwise.left_shift(), tf.bitwise.right_shift()
  1. Matrix Operations:
  • Matrix multiplication: tf.matmul()
  • Transposition: tf.transpose()
  • Matrix inverse and pseudo-inverse: tf.linalg.inv(), tf.linalg.pinv()
  1. Shape and Slicing Operations:
  • Reshaping: tf.reshape()
  • Slicing: tf.slice(), tf.strided_slice()
  • Concatenation: tf.concat(), tf.stack()
  1. Neural Network Layers:
  • Dense (fully connected) layers: tf.keras.layers.Dense()
  • Convolutional layers: tf.keras.layers.Conv2D()
  • Recurrent layers: tf.keras.layers.LSTM(), tf.keras.layers.GRU()
  1. Custom Operations:
  • You can create your own custom operations using tf.function() or by defining custom layers and models.

These are just some examples of operations that can be applied to symbolic tensors in TensorFlow. TensorFlow provides a wide range of mathematical and machine learning operations to manipulate and transform tensors. You can combine these operations to build complex computational graphs for various machine learning tasks.