AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute '_keras_history'

General Discussion
, ,
#1

Hi everyone,

I’m trying to implement a simple feed-forward neural network with a modification based on this paper arxiv:2007.11207 (See figure 3b).

Basically the inputs (x) are scaled by a vector, e.g., (1,2,3) producing new inputs (1x, 2x, 3x). These are then passed into 3 sub-networks that are independent of each other. The outputs of these networks are then summed together at the end to produce an output.

I have the following TF2 code below but when I try to plot the model using tf.keras.utils.plot_model(model) I get the following error.

AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute '_keras_history'

The code I am using to build the model is:
(for simplicity the sub-networks just have a single layer)

import tensorflow as tf

def build_model():
    input_shape = 1
    output_shape = 1
    units = 128
    scales = tf.convert_to_tensor([[1],[2],[3]], dtype=tf.float32)

    input_layer = tf.keras.Input(shape=(input_shape,))

    # create sub-networks
    xs = []
    for scale in scales:
        scaled_input = tf.keras.layers.multiply([input_layer, scale])
        xs.append(tf.keras.layers.Dense(units, activation='relu')(scaled_input))

    # add the outputs of each sub-network
    output_layer = tf.keras.layers.add([x for x in xs])
    output_layer = tf.keras.layers.Dense(output_shape, activation="linear")(output_layer)

    model = tf.keras.models.Model(inputs=input_layer, outputs=output_layer)

    return model

model = build_model()

# then the following throws the error.
tf.keras.utils.plot_model(model)

Thanks in advance for any help!

1 Like
#3

Hi all,

I think I managed to track down the problem to this line:

scaled_input = tf.keras.layers.multiply([input_layer, scale])

TF2 doesn’t like this multiplication. Is it because scale is not a layer?

I guess to fix this I could make a new layer or use a Lambda layer that does the multiplication between the inputs and a scalar.

Thanks!

1 Like
#4

I managed to get this working now and there was a similar problem on SO here.

I’ll post the working code here in case anyone is interested.

import tensorflow as tf

# create a new layer that simple multiplies the input by a scalar
# got this from
# https://www.tensorflow.org/api_docs/python/tf/keras/layers/Lambda
class ScaleLayer(tf.keras.layers.Layer):
    def __init__(self, scale):
        super(ScaleLayer, self).__init__()
        self.scale = scale
    def call(self, inputs):
        return inputs * self.scale


# need a function to create sub-networks
def build_subnetwork(input_tensor, units, activation='relu', n_hidden=1):
    x = tf.keras.layers.Dense(units, activation=activation)(input_tensor)
    for _ in range(1, n_hidden):
        x = tf.keras.layers.Dense(units, activation=activation)(x)

    # single output here?
    x = tf.keras.layers.Dense(1, activation='linear')(x)

    return x

def build_model(
    input_shape = 1,
    output_shape = 1,
    units = 128,
    activation = 'relu',
    n_hidden_subnetwork = 2,
    scales = [1]
    ):
    input_layer = tf.keras.Input(shape=(input_shape,))
    # create sub-networks
    xs = []
    for scale in scales:
        scaled_input = ScaleLayer(scale)(input_layer)
        xs.append(build_subnetwork(scaled_input, units, activation, n_hidden_subnetwork))
    if len(xs) > 1:
        output_layer = tf.keras.layers.add([x for x in xs])
    else:
        output_layer = xs[0]
    output_layer = tf.keras.layers.Dense(output_shape, activation="linear")(output_layer)
    model = tf.keras.models.Model(inputs=input_layer, outputs=output_layer)
    return model
1 Like