Adapt TensorFlow 1 Variational Autoencoder example from a notebook

This is based on: tensorflow-mnist-vae/mnist_vae.ipynb at master · CharlesD1ng/tensorflow-mnist-vae · GitHub

In my model.py, I have:

class model():
  def __init__(self):
    self.X = tf.placeholder(dtype=tf.float32, shape=[None, 28, 28], name='X')
    self.keep_prob = tf.placeholder(dtype=tf.float32, shape=(), name='keep_prob')

  def encoder(self): #I'm using self.X here to find z
    ...
    return self.X, self.keep_prob, z, mn, sd

  def decoder(self, z): #I'm using z (generated in encoder) to find image
    ...
    return image

  def loss(self, image, sd, mn):#I'm using image generated by decoder and the original image self.X
    ...
    return loss

  def optimizer(self, loss, lr): #Call optimizer by using loss and learning rate.
    ...
    return optimizer

In my training.py, I want to train the model that I defined in model.py. Therefore I have:

mnist = input_data.read_data_sets('MNIST_data')
lr = 0.005

model = model()
X, keep_prob, z, mn, sd = model.encoder()
decoder = model.decoder(z)
loss = model.loss(decoder, sd, mn)
optimizer = model.optimizer(loss, lr)

sess = tf.Session()
sess.run(tf.global_variables_initializer())

for epoch in range(epoch_number):
    batch = [np.reshape(b, [28, 28]) for b in mnist.train.next_batch(batch_size=batch_size)[0]]
    loss_, _ = sess.run([loss, train_optimizer], feed_dict={X: batch, keep_prob: 0.8})

    print("iteration: {}, overall_loss: {}".format(epoch, loss_))

Actually my question is related to that point:

model = model()
X, keep_prob, z, mn, sd = model.encoder()
decoder = model.decoder(z)
loss = model.loss(decoder, sd, mn)
optimizer = model.optimizer(loss, lr)

I’m new on tensorflow and not sure if it is the right way to train a model in tensorflow 1. I saw something scope, variable, get_collection things and a little bit confused. Is it the right way of training a model? Because, if it is the case. After saving my model, to generate new images using this decoder that I trained, I have some difficulties. In a similar way, to test my model that I saved, in test.py, I do:

model = model()
X, keep_prob, z, mn, sd = model.encoder()
decoder = model.decoder(z)

saver = tf.train.Saver()
sess = tf.Session()

saver.restore(sess, 'model.pk')

randoms = [np.random.normal(0, 1, intermediate_dim) for _ in range(100)]
images = sess.run(dec, feed_dict = {z: randoms, keep_prob: 1.0})
k = [np.reshape(imgs[i], [28, 28]) for i in range(len(images ))]

In this writing style, to use decoder I need to call encoder. If I do not call it, it will not know the variable z. See:

X, keep_prob, z, mn, sd = model.encoder()
decoder = model.decoder(z)

Do you have any idea about what is the right way to use different .py files while training a deep learning network using tensorflow 1.

Thank you in advance.

2 Likes

I suggest you to start with TF 2 directly. Take a look at

4 Likes

thank you for your suggestion but I need to solve this problem by using tensorflow 1. It is some how obligatory for me.

1 Like

Do you know that TF 1 is not actively supporter anymore?

2 Likes

yes but by the way, I need to do it

1 Like

I strongly suggest you to use TF 2 cause it will be quite hard to receive support on TF 1.x.

In any case this was an old TF1 tutorial on VAE/MNIST.

3 Likes

Hi @john123 Do you have the full project maybe stored in a public repo or can you post it all here? Maybe we could try to provide feedback / debug it for you if we can replicate fully. Thanks!

2 Likes

hi @8bitmp3, actually I’m trying to separate this ipynb file (https://github.com/CharlesD1ng/tensorflow-mnist-vae/blob/master/mnist_vae.ipynb. ) into model.py, train.py and test.py.

1 Like

For TF 1, maybe you can adapt this code: VAE-Tensorflow/src at master · ChengBinJin/VAE-Tensorflow · GitHub

And, in comparison, this TF 2 code circa December 2019 is quite clean: Deep-Learning-with-TensorFlow-2-and-Keras/VAE.ipynb at master · sujitpal/Deep-Learning-with-TensorFlow-2-and-Keras · GitHub

1 Like