Questions about Teachable Machine

General Discussion
#1

We are using Google Teachable Machine for teaching ML in K-12 as part of our initiative Computação na Escola at the Federal University of Santa Catarina/Brazil and it has been demonstrated to be a very intuitive and motivating tool in this context.

Yet, in order to teach also a little more about how it works, we find it difficult to encounter documentation. Is there any more detailed documentation available?

We are also in doubt with respect to the following issues:

  • “Teachable Machine splits sample into training sample (85%) and “test” sample (15%). Test samples are never used to train the model, so after the model has been trained on the training samples, they are used to check how well the model is performing on a new, never-before-seen data.”
    Following this definition, we assume that in fact a “test” set is separated and not a “validation” set (being used to evaluate the model performance at each training epoch)?

  • Data augmentation
    We didn’t find any indication that data augmentation is done, so we assume that no data augmentation is done?

  • It is not clear whether, at the end of the training, the result is the model of the last trained epoch or the model obtained at the epoch with the highest accuracy/lowest loss during training.
    We assume that the result is the model resulting from the last training epoch? Which is then used for the performance evaluation with the test set (accuracy by category/confusion matrix)?

Thanks for any help!

#2

I believe you can use the test sample for both validation during training and evaluation of the trained model. My understanding is that, during training, the model does not learn from the validation records but merely computes and reports the loss and metrics on those records. In cases of “overfitting”, the training metrics will often improve while the validation metrics deteriorate.

Unless you use some sort of automatic hyperparameter optimization, such as KerasTuner, the trained model will be based on the last training epoch.

EDIT: I don’t know anything about Teachable Machine, and my comments above are based purely on my use of Keras and TensorFlow.

2 Likes
#3

Thanks for the response in general! Yet, we are still trying to find out about this questions specifically with regard to Teachable Machine.