I’m working on my first machine learning project in Python - using TensorFlow to try and syllabify words using the Moby Hyphenator II dataset.
I am treating this as a multi-label classification problem in which words and their syllables are encoded in the following format:
T e n - s o r - f l o w
0 0 1 0 0 1 0 0 0 0
When reading through this guide as a starting point, I saw that the author used a custom function - they averaged weighted binary cross-entropy with the root mean squared error in PyTorch as such:
def bce_rmse(pred, target, pos_weight = 1.3, epsilon = 1e-12):
# Weighted binary cross entropy
loss_pos = target * torch.log(pred + epsilon)
loss_neg = (1 - target) * torch.log(1 - pred + epsilon)
bce = torch.mean(torch.neg(pos_weight * loss_pos + loss_neg))
# Root mean squared error
mse = (torch.sum(pred, dim = 0) - torch.sum(target, dim = 0)) ** 2
rmse = torch.mean(torch.sqrt(mse + epsilon))
return (bce + rmse) / 2
I have tried to implement this in TensorFlow in the following way:
def weighted_bce_mse(y_true, y_prediction):
# Binary crossentropy with weighting
epsilon = 1e-12
positive_weight = 4.108897148948174
loss_positive = y_true * tf.math.log(y_prediction + epsilon)
loss_negative = (1 - y_true) * tf.math.log(1 - y_prediction + epsilon)
bce_loss = np.mean(tf.math.negative(positive_weight * loss_positive + loss_negative))
# Mean squared error
mse = tf.keras.losses.MeanSquaredError()
mse_loss = mse(y_true, y_prediction)
averaged_bce_mse = (bce_loss + mse_loss) / 2
return averaged_bce_mse
On doing so, I receive the error ValueError: 'outputs' must be defined before the loop. and I’m not sure why as I define this function before I build and compile my model.
I’m using the Keras Functional API, and my compilation and fit stages are:
model.compile(optimizer="adam", loss=weighted_bce_mse, metrics=["accuracy"], steps_per_execution=64)
history = model.fit(padded_inputs, padded_outputs, validation_data=(validation_inputs, validation_outputs), epochs=10, verbose=2)