CNN net much slower than DNN

I am a newbie and was going through examples in book ISBN9781492032649.
in page 297.py it uses MINST 28x28 images to train the neural network using regression model. (example 1 pasted below). This took about 167 seconds for 30 epochs on radeon GPU.

But same MNIST images are performy far more slowly on CNN network but i thought CNN would be much more faster and efficient?

EXAMPLE1:

Using neural net to do a classification task.

import tensorflow as tf
import pandas as pd
import matplotlib as plt

from tensorflow import keras
print(tf.version)
print(keras.version)

CONFIG_ENABLE_PLOT=0

fashion_mnist = keras.datasets.fashion_mnist
(X_train_full, y_train_full), (X_test, y_test) = fashion_mnist.load_data()
print("X_train_full.shape: ", X_train_full.shape)
print("X_train_full.dtype: ", X_train_full.dtype)

X_valid, X_train = X_train_full[:5000] / 255.0, X_train_full[5000:]/255.0
y_valid, y_train = y_train_full[:5000], y_train_full[5000:]
X_test = X_test / 255.0
class_names = [“T-shirt/top”,“Trouser”, “Pullover”, “Dress”, “Coat” , “Sandal”, “Shirt”, “Sneaker”,“Bad”,“Ankle boot”]

model=keras.models.Sequential()
model.add(keras.layers.Flatten(input_shape = [28, 28]))
model.add(keras.layers.Dense(300, activation=“relu”))
model.add(keras.layers.Dense(100, activation=“relu”))
model.add(keras.layers.Dense(30, activation=“softmax”))

print("model summary: ", model.summary())

model.compile(loss=“sparse_categorical_crossentropy”, optimizer=“sgd”, metrics=[“accuracy”])
history=model.fit(X_train, y_train, epochs=30, validation_data=(X_valid, y_valid))

pd.DataFrame(history.history).plot(figsize=(8, 5))

if CONFIG_ENABLE_PLOT:
plt.pyplot.grid(True)
plt.pyplot.gca().set_ylim(0, 1)
plt.pyplot.show()

model.evaluate(X_test, y_test)

print("model layers: ", model.layers)
weights, biases = model.layers[1].get_weights()
print("weights, biases (shapes): ", weights, biases, weights.shape, biases.shape)
model.save(“p297.h5”)
X_new = X_test[:3]
y_proba = model.predict(X_new)
print(y_proba.round(2))

y_pred = model.predict_classes(X_new)
print("y_pred: ", y_pred)

EXAMPLE2:

Using CNN to do a classification task.

import tensorflow as tf
import pandas as pd
import matplotlib as plt
import time

from tensorflow import keras
print(tf.version)
print(keras.version)

CONFIG_ENABLE_PLOT=0

fashion_mnist = keras.datasets.fashion_mnist
(X_train_full, y_train_full), (X_test, y_test) = fashion_mnist.load_data()

X_train_full = X_train_full.reshape(-1, 28, 28, 1)
print("X_train_full.shape: ", X_train_full.shape)
print("X_train_full.dtype: ", X_train_full.dtype)

X_valid, X_train = X_train_full[:5000] / 255.0, X_train_full[5000:]/255.0
y_valid, y_train = y_train_full[:5000], y_train_full[5000:]
print("X_test shape: ", X_test.shape)
X_test = X_test.reshape(-1, 28, 28, 1)
X_test = X_test / 255.0
class_names = [“T-shirt/top”,“Trouser”, “Pullover”, “Dress”, “Coat” , “Sandal”, “Shirt”, “Sneaker”,“Bad”,“Ankle boot”]
‘’’
model=keras.models.Sequential()
model.add(keras.layers.Flatten(input_shape = [28, 28]))
model.add(keras.layers.Dense(300, activation=“relu”))
model.add(keras.layers.Dense(100, activation=“relu”))
model.add(keras.layers.Dense(30, activation=“softmax”))
‘’’

model=keras.models.Sequential([
keras.layers.Conv2D(64, 7, activation=“relu”, padding=“same”, input_shape=[28, 28, 1]),
keras.layers.MaxPooling2D(2),
keras.layers.Conv2D(128, 3, activation=“relu”, padding=“same”),
keras.layers.Conv2D(128, 3, activation=“relu”, padding=“same”),
keras.layers.MaxPooling2D(2),
keras.layers.Conv2D(256, 3, activation=“relu”, padding=“same”),
keras.layers.Conv2D(256, 3, activation=“relu”, padding=“same”),
keras.layers.MaxPooling2D(2),
keras.layers.Flatten(),
keras.layers.Dense(128, activation=“relu”),
keras.layers.Dropout(0.5),
keras.layers.Dense(64, activation=“relu”),
keras.layers.Dropout(0.5),
keras.layers.Dense(10, activation=“softmax”)
])

print("model summary: ", model.summary())

model.compile(loss=“sparse_categorical_crossentropy”, optimizer=“sgd”, metrics=[“accuracy”])
history=model.fit(X_train, y_train, epochs=10, validation_data=(X_valid, y_valid))

pd.DataFrame(history.history).plot(figsize=(8, 5))

if CONFIG_ENABLE_PLOT:
plt.pyplot.grid(True)
plt.pyplot.gca().set_ylim(0, 1)
plt.pyplot.show()

model.evaluate(X_test, y_test)

print("model layers: ", model.layers)
weights, biases = model.layers[1].get_weights()
print("weights, biases (shapes): ", weights, biases, weights.shape, biases.shape)
model.save(“p297.h5”)
X_new = X_test[:3]
y_proba = model.predict(X_new)
print(y_proba.round(2))

y_pred = model.predict_classes(X_new)
print("y_pred: ", y_pred)

Its running on CPU and that’s why its slow. Deep leanring models don’t support AMD Radeon GPU. They only work on Nvidia GPUs. This is the same for both Tensorflow and PyTorch.

Because the network is training using CPU, Deep models (with many CNN layers as here) will be slower than shallow networks.

I would suggest you try this in Google’s Colab environment. Please choose Runtime → Change RUntime as GPU in https://colab.research.google.com/

1 Like

It Is supported, see tensorflow-rocm · PyPI

1 Like

Thanks @Bhack. I didn’t realize that Radeon GPU’s are supported. @guen_gn - Did you install tensorflow through this package - https://pypi.org/project/tensorflow-rocm

Also, is your OS one listed in the compatibility for AMD GPUs - AMD ROCm™ Release Notes v4.3 — ROCm Documentation 1.0.0 documentation

1 Like

yes tensorflow-rocm installed.

GPU Radeon support question is shot down as candidate. Any other ideas?

Have you compared the flops of the two models:

how do I do?
Just pass model to this:? get_flops(model)

E.g. you can use something like: