Hello everyone, I’m trying to build a single layer neural network in the function below. For different reasons I’m trying to avoid using Keras because I need to run over 400K instances of this network. Loading 400K keras objects is very slow. So I’m doing my own implementation. I have forward propogation finished.
I don’t quite remember how do backpropogation. I can figure out most of it, but I’m not sure if it’s different with a leaky_relu activation. Would anyone be able to help me leaky_relu backward?
Thank you!
import tensorflow as tf
import numpy as np
@tf.function
def predict(x, b, w):
z = tf.tensordot(w, x, axes=1) + b
G = tf.nn.leaky_relu(tf.tensordot(w, x, axes=1) + b)
return G
weights = tf.random.uniform((9, 9), dtype=tf.float32)
bias = tf.random.uniform([9], dtype=tf.float32)
signal = tf.random.uniform([9], dtype=tf.float32)
model_predicted = []
for i in range(446436):
model_predicted.append(predict(signal, bias, weights))